Charlie Toy: Milwaukee’s Chinese Connection

by

Standing alone in the cold unrelenting rain, Charlie Toy wept as he watched a wrecking crew demolish the beautiful Chinese-inspired structure that he built in 1912. — The Milwaukee Journal, 1939

Moi Toy Ni, 20, stands on the ship’s deck as the city of San Francisco emerges from the dense ocean fog. He carries a change of clothes and less than five dollars wrapped in a small bundle. New friends in the Chinese community get him work on a potato farm where he earns two cents for every bag picked. To assimilate himself in this new culture, he calls himself Charlie Toy.

Charlie was born in 1860 in a poverty-stricken village near Canton. For years he and his friends were told that their futures lay in the New World. When the time came, only Charlie was brave enough to take the plunge. After a year of backbreaking labor on the potato farm, he found work as a houseboy in San Francisco’s wealthy homes where he learned to prepare American and Chinese dinners. When he finally earned the money for traveling expenses, Charlie moved to Chicago where he hoped to open a restaurant. An acquaintance suggested he choose a small town in Wisconsin where start-up costs were considerably lower. Taking the advice, he opened a tiny café in Oshkosh. It quickly became a favorite gathering place for businessmen, corporate executives, and elected officials. In 1900 Charlie doubled the size of his restaurant and added a gift shop with porcelain tea sets, silk fabrics, and other items imported from China. Soon he was ordering hundreds of crates of merchandise on credit and resold their contents to merchants in the Chicago and New York City Chinatowns. Within two years, Charlie was the nation’s largest wholesaler of Chinese goods. 

In 1903, Charlie moves to Milwaukee and leases a large storefront on Plankinton Avenue. His Shanghai restaurant attracts passersby from the street, and while they eat, he operates the import business from behind the kitchen. There are some organizations in town that believe all Chinese people gamble, run opium dens, and sell white women into slavery. But the racism doesn’t extend itself to Charlie because he’s a people person. The chief of police, the fire chief, several aldermen, and newspaper reporters take to him instantly and dine at the Shanghai at least once a week. 

email newsletter iconStay on top of the news of the day
Subscribe to our free, daily e-newsletter to get Milwaukee's latest local news, restaurants, music, arts and entertainment and events delivered right to your inbox every weekday, plus a bonus Week in Review email on Saturdays.

SIGN UP

Five years later Charlie’s net worth was $4 million in today’s currency. He commissioned an architect to design the six-story Shanghai building that recalled the grand structures in his homeland. Green terra cotta tile, red Chinese characters, decorative lanterns, and golden dragons at the roofline made for a striking addition to Second Street near Wisconsin Avenue. On opening day, the Toy Restaurant’s second floor dining room was packed from noon to midnight. As hundreds of people in the street tried push their way in, Charlie left the receiving line and waited tables for hours alongside his staff. Attached to the restaurant was a large ballroom where popular dance bands played every evening from 6 p.m. to midnight. The third floor held a billiard hall, and a dozen apartments where the restaurant’s managers could live. The fourth floor housed three major clothing retailers while above them Paramount, Universal, MGM and Fox executives managed their Wisconsin movie theaters. Charlie and his family lived in tasteful penthouse suites on the top floor.

Movie Theater

In 1915, he opened the 460-seat Toy movie theater on the building’s ground floor. He was able to compete with the Strand, Merrill, and other downtown theaters until 1920 when he could not increase the number of seats. He leased the theater to independent operators who ran it as the Paradise and then the Imperial. Charlie bought the neighboring Crystal theater and presented vaudeville shows there for the next ten years. 

When the Depression comes in 1929, many people are deprived of disposable income, and cannot afford to eat out. Charlie’s businesses are affected just like everyone else’s, and he compensates in a variety of ways. He scales back the import business and brings in 75% less merchandise from China. Instead, he exports American sinks, toilets, bathtubs, and other plumbing products to Canton to modernize the public as well as privately held buildings. Seeing the dining room in the Toy restaurant half-empty every evening, Charlie closes it and moves to a smaller space across the street. He is hit again when the owners of the land upon which his Shanghai building sits decide a parking lot would generate more revenue. Rather than buy his way out of this latest downturn, Charlie sells his 99-year lease back to them in 1939. On a cold, rainy day in May, pieces of the most exotic building in the city plummet to the ground.

Charlie’s oldest son, Moy Toy, convinced his father to keep the smaller restaurant on Second Street but open a much larger one in the heart of the theater district.  Moy leased the second floor of the Walgreen’s building at Third and Wisconsin and began construction on Toy’s Chinatown Restaurant. Charlie, already in his late 50s, left the day-to-day management to his son while he made several trips to China. With the economy bouncing back during World War II, Moy’s restaurant prospered for decades. 

Charlie Toy left Milwaukee for good in 1946 and lived in China for the rest of his life. He continued to be an economic and philanthropic force, helping resident families live better lives. The entrepreneur who amassed a fortune in 60 years of business passed away on January 4, 1955, at the age of 95.

In 1968 Moy Toy moved the Chinatown Restaurant to 830 N. Old World Third Street where it remained until 2000.