One of Long Island’s most famed residences, the mansion at Old Westbury Gardens, has a rich and storied history. There remains, however, an untold story from inside the gated Gold Coast mansion. That is the first-hand account of a young boy who grew up a witness to the abundance of wealth and affluence, yet far removed; he was the butler’s son.
“It was a very unique childhood being the son of the butler,” explained 85-year-old Albert Brown.
Brown’s childhood setting was apparently meant to be. His parents, Basil and Agnes Brown met at Old Westbury Gardens while both working on the domestic staff in the mid-1920s. Brown’s father had came to the U.S. from Newfoundland, his mother immigrated from Scotland. They quickly fell in love and were married.
As a wide-eyed young boy Brown vividly recalls encounters with mansion owners John S. Phipps and his wife. “Mr. Phipps was very friendly to all of the children of the domestic staff,” said Brown. “But Mrs. Phipps was always tucked under Mr. Phipps right shoulder and never spoke to us.”
Being the son of the butler, the main rooms in the “Big House,” as it was known, were off-limits. He did, however, have access to all of the domestic staff rooms, a privilege only afforded the butler’s son. His favorite room to visit was the pantry. “It was great fun when dad had pantry duty. I could get my favorite Millard’s Mints and Coca Cola,” said Brown. “All of the domestic staff treated me like I was their nephew.”
While much of the house was forbidden, Brown was free to explore the lush 200-acre property. It became a virtual wonderland to the adventurous young Brown, his younger brother Daniel and all of the children of the domestic staff.
“I rode my balloon wheeled bicycle all across the property,” Brown fondly recalled. “In the summers I would fish in the pond, in the winter we would use the pond for ice skating.” Brown said the Phipps frequently opened up the gates to allow neighbors to ice skate on the pond as well.
It wasn’t just locals Brown remembers visiting the estate. Many famous people from the era often were guests of the Phipps. One in particular stands out in Brown’s memory.
“Dancer Fred Astaire knocked on the door to the estate, and when my family member answered, she proclaimed, ‘You’re Fred Astaire!’ Fred said yes and came dancing through the doorway,” Brown recalled with a laugh.
Of course Brown and his family took full advantage of what the mansion is arguably best known for, its plush and abundant gardens. In fact, Brown’s first job as a young boy was in the gardens. “I got paid 35 cents a week to weed the greenhouse gardens.”
The industrious Brown went on to other jobs at the gardens. His most profitable at the time was when he hatched his own chickens in an incubator he kept under his bed. “I was selling my own eggs for Hicks Nursery,” said Brown. “I was only about 13 or 14, but making pretty good money.”
Brown is a retired chemist with five children and 11 grandchildren. Just as he did when he was a little boy, Brown can still be found exploring the grounds on the setting of his unique childhood.
“I just love it at the Gardens,” said Brown. “It brings back so many memories.”
