With renowned interior designer William Haines and his partner Ted Graber in charge of the project and University of Southern California architecture professor A. Quincy Jones on their team, construction for the Sunnylands estate began in 1963. The estate includes 12 man-made lakes, an 18-hole golf course with nine greens and hundreds of eucalyptus trees, olive trees and oleanders, plus thousands of shrubs and smaller trees. Beyond the house are the soaring San Jacinto and Santa Rosa Mountains.

Sunnylands is rich with historical significance. President Dwight D. Eisenhower and his wife Mamie often joined the Annenbergs at Sunnylands to fish and golf. The Annenbergs, who both held the rank of ambassador - he as Richard Nixon's ambassador to the Court of St. James's, she as Ronald Reagan's chief of protocol - have hosted presidents of countries and universities, prime ministers, princes, ambassadors, civic and cultural giants and military and business greats. President Nixon used the estate to work on his State of the Union message in 1974 and was a guest there on September 8, 1974, when he was pardoned by President Ford. Among their other guests have been President Ford, President Reagan, President Bush, President Clinton and President George W. Bush.

Sunnylands also has served as a refuge. In 1979 when the sister and mother of the exiled Shah of Iran were being tormented by hostile demonstrators, the Annenbergs made Sunnylands available to them.

In March 1990, the city of Rancho Mirage declared Sunnylands an historic site in recognition of an official dinner at the estate hosted by President George H.W. Bush in honor of Japan's Prime Minister Toshiki Kaifu. Sunnylands remains Mrs. Annenberg's home, but she has made it available for meetings and forums sponsored by the Trust.