Around 1947, Maude Terry looked at the land along Sag Harbor Bay on Long Island and had a vision—to create a summer community there for African Americans. Terry, along with her sister Amaza Lee Meredith, an art professor at what was then called Virginia State College for Negroes, developed 120 lots into what they called Azurest North, now known simply as Azurest. The lots sold for $1,000 on the bay; less inland. Although black folks had been summering on the East End for some years, Azurest spurred a steady tradition of vacation-home ownership there that continues to this day.
Drive around Sag Harbor Hills or any of the other four historic black communities in Sag Harbor—Azurest, Ninevah Beach, Eastville, and Hillcrest Terrace—and note the street names. They hold the history. There is Terry Drive and Meredith Avenue, for the enterprising sisters; Richards Drive was named after Terry’s son-inlaw. There is also a Pickens Place. “It’s a sign on our property, and it’s unofficial,” says William (Bill) Pickens III, chuckling. Pickens has been living on the East End for 65 years and can trace his local roots to an aunt, an English teacher at Tuskegee Institute, who knew Booker T. Washington and began vacationing in Sag Harbor in 1902. His grandfather, Dr. William Pickens Sr., was one of the first black graduates of Yale Law School; his son also made his way to Sag Harbor.
The 76-year-old Pickens is a keen historian, not only of his family, but also of the area. “There is a Paul Robeson Street. Robeson came here,” says Pickens of the singer and civil rights activist. “Several black artists and musicians did. Langston Hughes used to come out in the 1950s and stay at my parents’ house, read poems on their porch. He was a college roommate of my father’s and an old pal of my mother’s. He even wrote her a poem back in 1925.”
There is also a Cuffee Drive, named after the African American Cuffee family. Whaler Jeremiah Cuffee lived in Sag Harbor and shipped out on the vessel Ann Maria in January 1832. It was the golden age of whaling, and black sailors were ardent participants.
Although there is evidence that British colonists settled in Sag Harbor in the early 17th century—a place, then, of hills and meadows wound through with streams and sunk with swamps—the town itself was not incorporated until 1846. With its deep waters, Sag Harbor (the name possibly derived from the nearby village of Sagaponack) was second only to New York City as a seaport and was at the pinnacle of the whaling industry.
Winding through the traditionally black communities, one can see the generations that have lived there displayed in the architecture; modest cottages are neighbored by larger, modern homes, different styles built over decades and even centuries. There are whaling-era homes on Hempstead Street, the most historic black street in the area, according to Pickens. David Hempstead was one of the founders of the 172-year-old St. David’s African Methodist Episcopal Zion Church in Eastville, reputed to be a stop on the Underground Railroad. He is listed, along with his wife, Mary, in the 1880 census. “Little frame homes were put up by whalers and early settlers who lived a quiet, sedate life,” says Pickens. “Some of those homes are still in the hands of black families.”
Keeping property in the hands of black families has been a concern to residents of these neighborhoods as land values have boomed on the East End over the past 30 years. “One of the reasons I got myself involved here is because it’s not just about real estate, it’s about helping the community stay true to its history,” says Prudential Douglas Elliman Real Estate agent Dianne McMillan Brannen, who, along with her husband, John, specializes in the properties of Sag Harbor Hills, Azurest, and Ninevah Beach. They not only make their livelihood here, but also are long-term residents. “We have an area not like other places in the Hamptons. I work with the sellers and find out their motivation for moving. Can we work with people in the area first? We’ll do an e-mail blast, we’ll send out letters, and we’ll ask other members in the community, ‘Do you have an interest in the sale of this property?’ ‘Do you have a friend or relative?’ Because that’s largely how we operated 60 or 70 years ago, and I think that’s very important to continue.”
This word-of-mouth tradition has kept turnover much lower in these neighborhoods than throughout the rest of the Hamptons. “I would say that between 10 and 20 percent of the overall home ownership in the three beach communities is what we consider to be a changing population,” McMillan Brannen notes. “In the last year, out of the five sales that I did, three were buyers from what we call the outside community, however, I’m finding the buyers I work with are all like-minded people. They appreciate the sense of community, the friendliness, and that when they walk down the beach they get the ‘Good morning’ and ‘How are you?’ They are all here because they like the beauty.”
The allure of the area is evident: clear skies and blue water, sand and sun, and the scent of the warm woods. “It’s a wonderful place for artists,” says Pickens. “The colors are beautiful, the summer light—artists love it. It’s so quiet with sometimes only the birds singing. You can see why a writer would like it. It’s a great place to restore one’s soul.”
In Nina Tobier’s 2007 book, Voices of Sag Harbor: A Village Remembered (with a foreword by E. L. Doctorow, who also spent summers in Sag Harbor), longtime resident Michael A. Butler shared his childhood memories of the area: “I can recall the sound of the bell from the Whalers Church, which was adjacent to our backyard; visits to the 5-and-10 with my grandmother; bridge parties and piano playing at the house on Division Street; and Red Cross swimming lessons at Havens Beach.”
Butler’s uncle James “Jimmy” Harris purchased a home on Division Street in the 1930s and bought cottages in Eastville. Part of the appeal was having a place of the family’s own, to lay down roots and not have to worry about living in a world that was mostly white, and often hostile. But while the loveliness of the East End and the security of being within a mainly African-American community were two lures, blacks like Maude Terry also chose to summer there for another, more pragmatic reason. New York’s mixed-race roots date back to its European settlement in the 17th century, yet the colony played an integral part in the slave trade for more than a century. But even after the Civil War, New York, like many Northern states, practiced de facto segregation. Unwelcome among whites, black whalers settled in communities considered less desirable—in the case of the East End, Sag Harbor’s waterfront and the Eastville neighborhood. Blacks continued the tradition into the early 20th century by taking over the old whaler cottages and building summer homes there.
“We knew where our neighborhood began because that’s where the map ended. The black part of town was off in the margins,” states Benji, the main character in Colson Whitehead’s Sag Harbor. Semiautobiographical, it tells the story of a black teen and his brother over one summer in the 1980s at their family beach house. As Benji walks to his job at an ice cream shop, he ruminates on the black vacation spot: “Certainly the first generation claimed and settled on Sag Harbor Bay because the south side was off-limits—the white people owned the coastline, Southampton, Bridgehampton, East Hampton.”
Today it’s difficult to believe that any prime waterfront real estate could be anything but highly desirable, and over the past decade, the demographics in the traditional black communities have shifted to include other cultures and ethnicities. Still, the turnover rate has been traditionally slow—not because no one wants to live here, but because those who do, want to keep their property in the family. “Families from Brooklyn and Queens did come out in the 1940s, put down claims and stakes, and paid off mortgages,” says Pickens. “They realized that rooting was important. We as a people were so itinerant for 300 years; this gave African Americans a nice place to come. You’ve got four generations out here, and some have five.”
Enterprising is another word associated with blacks in Sag Harbor. From the opportunities of the whaling industry to the entrepreneurial spirit that led Maude Terry and Amaza Lee Meredith to create a vacation community out of the remnants of an old whaling village, it is something that is embodied in current Sag Harbor residents and restaurateurs Barbara “B.” Smith and her husband of 20 years, Dan Gasby. “We knew folks who had houses here,” says Smith, whose list of accomplishments include TV personality, lifestyle maven, and author, and who, along with Gasby, owns three successful namesake restaurants in Sag Harbor, Midtown Manhattan, and Washington, DC. “Dan and I took a look and liked what we saw. Our daughter was five or six at the time, and we saw lots of young children. We concentrated on that area because we wanted her to be with kids who looked like her.”
The Sag Harbor restaurant was not even an initial thought. “B. and I were running, and during our run we kept seeing a fellow resident, Candace Malloy,” Gasby says. “Four times she drove past us and honked her horn. I saw her more in one day than I’ve seen some relatives in 20 years.” A few weeks later, Malloy called. “She said her husband, Pat, who owns the marina in Sag Harbor, wanted to know if we’d like to take over a restaurant at his marina.” The place was derelict but offered the most amazing views. “We said yes, just like that.”
While they treasure the black history and culture in Sag Harbor and would like to see more minority businesses opening up there, Smith and Gasby are also working to nurture the cross-culturalism they believe will move Americans forward. “We hire all sorts of young people out here: black, white, Jewish, Latino, first-, second-, third-generation Sag Harbor residents of all different socioeconomic backgrounds,” says Gasby. “They have to learn how to work together, make it work.”
Standing on the pier by B. Smith’s restaurant, one can look across Sag Harbor Bay to the mansions in North Haven—quite different from the homes and cottages a bay’s width away. A laundry list of the notable and affluent African Americans join Smith and Gasby: Colson Whitehead; Earl Graves and Susan L. Taylor, who blazed trails in publishing with Black Enterprise and Essence magazines, respectively; American Express executive Kenneth Chenault; and General Colin Powell, who spent summers there as a youth. But Smith and Gasby, like Pickens, will be quick to tell you that Sag Harbor is not merely about “who’s who.” Regardless of race or socioeconomic status, there is that ownership of something special, a place of history and inheritance and natural treasures that connects the residents.
“The story is not that it’s elite,” says Pickens. “We’re all out here, but all related by the love of the area. That’s the key to this place.”