From William Howard Taft being the first U.S. president to play golf to Lyndon B. Johnson using golf to help pass the Civil Rights Act, the above video offers a brief history of golf and the presidency.
For a complete history, read First Off the Tee: Presidential Hackers, Duffers, and Cheaters from Taft to Bush by Don Van Natta Jr. Van Natta's lively and authoritative story of presidential golf was a New York Times bestseller and Sports Illustrated book of the year.
I'm not sure which president coined the phrase, "Fore more years." (OK, I made that up.