Kyle Massey, 19, a former child star on the Disney Channel show "That's So Raven," was runner-up.
Grey, who danced through much of the season with injuries and said she suffered a ruptured disc in her back on Monday, had been the front-runner after scoring consistently high marks from the judges.
Jennifer Grey in the Lead Going Into 'Dancing With the Stars' Finale
But her claim on the mirror ball trophy seemed at risk as Palin, the daughter of conservative politician Sarah Palin, waltzed into the finals on a tide of public support despite coming bottom with the judges for more than six weeks.
"This has been a life-changing experience," the 20-year-old single mom said of her third-place finish. "I am very blessed to be here. I've had a ball."
Palin again scored the lowest with judges in her final tango and cha cha cha dances on Tuesday, but was praised for her improvement over the last three months.
The winner was determined by a combination of scores from the three judges and what producers said was a record amount of email and phone votes from the audience at home of more than 20 million viewers.
The record voting activity briefly crashed the ABC TV show website on Monday night, producers said, but "affected each finalist equally".
Palin's progress to the finals had sparked a furor over whether she was benefiting over the popularity of her mother at the expense of other more talented contestants.
Last week, one angry man blasted his TV set with a shotgun while watching the show, and an envelope filled with what turned out to be talcum powder turned up on the show's Los Angeles set.
Palin shrugged off the controversy, calling her critics "haters" who were waiting for her to fail.
Grey, on the other hand, is a trained dancer who proved an early favorite. Best known for playing the naive teen Frances "Baby" Houseman in the coming-of-age movie "Dirty Dancing," she won hearts with emotional recollections of performing with Swayze more than 20 years ago. Swayze died of pancreatic cancer in September 2009 at the age of 57.
Brandy Eliminated from 'Dancing With the Stars'
On Tuesday, Grey scored another perfect 30 for her Viennese waltz, which judge Bruno Toniolo called "blissfully beautiful".
Asked what she would miss most about "Dancing with the Stars", Grey replied simply, "Dancing."
"Dancing With the Stars" has enjoyed one of the best seasons in its five year history on U.S. television. More than 23 million viewers tuned in for Monday's finale, making it the most-watched show in the United States that night.
(Reporting by Jill Serjeant)
COPYRIGHT: (c) Reuters 2010. All rights reserved. Republication or redistribution of Reuters content, including by caching, framing or similar means, is expressly prohibited without the prior written consent of Reuters. Reuters and the Reuters sphere logo are registered trademarks and trademarks of the Reuters group of companies around the world.