Each episode, "X Factor" continues to surprise, and this week was no different. Two weeks ago, the show's schedule was rearranged for baseball and last week, Simon eliminated Melanie Amaro, only to quickly bring her back after realizing his mistake. On last night's episode -- the first of the live shows -- Fox announced that five contestants would be eliminated, rather than the expected two.

With one out of four contestants (and two out of five in the Girls category) going home last night, the performers had to give it their all, because one mistake could be the end.

'X Factor': 17 Contestants Move to Live Round & Simon Gets Soft

First up was the Boys category, starting with 14-year-old rapper Astro (formerly Brian Bradley) who performed an original song, rapping about the new generation of hip-hop, "X Factor," and having L.A. Reid on his side. Astro's performances had been getting weaker, but he redeemed himself with this one.

Video: Astro's first live "X Factor" performance

The next performer in the Boys category was Chris Rene, singing "Love Don't Live Here Anymore" by Rose Royce. He was raw and emotional, and Simon praised Rene for being "100 percent authentic."

Video: Chris Rene sings "Love Don't Live Here Anymore"

Frank Sinatra-clone Phillip Lomax was next, and L.A. Reid pushed him out of his Rat Pack comfort zone with "I'm A Believer" by the Monkees. Simon had little to say about Lomax, but he was not impressed with the song choice, calling it "cheap," "throwaway" and "cabaret."

Video: Phillip Lomax, "I'm A Believer"

Marcus Canty performed Culture Club's "Do You Really Want To Hurt Me" with so much energy and stage presence that his vocals could have been subpar -- but they were equally stellar and Nicole said he "made it look too easy" because he was "born to be on that stage."

Video: Marcus Canty sings "Do You Really Want To Hurt Me"

While last week the task of eliminating 16 contestants was so vast that it warranted its own two-hour episode, tonight the eliminations were rushed through in seconds. Although the rapid-fire eliminations did have the benefit of finally giving Steve Jones something to do, as he pressured the judges to make speedy decisions.

L.A. kept Astro, Marcus and Chris, while Phillip was sent home.

Next up were Paula's Groups, beginning with the Stereo Hogzz, who sang Otis Redding's "Try A Little Tenderness." The five-man group improves every week, and Simon told them, "There's nothing right now in the charts like you, but there should be a band like you in the charts right now."

Video: The Stereo Hogzz, "Try A Little Tenderness"

Brother duo the Brewer Boys were next, singing Hall & Oates' "Rich Girl." Both boys showed that they could play a mean guitar in addition to singing, but none of the judges were overwhelmed at the relatively flat performance.

Video: Brewer Boys perform "Rich Girl"

InTENsity, a massive teen group formed from 10 solo contestants at bootcamp, sang Kenny Loggins' "Footloose," performing much better as a cohesive group than they did at Paula's home. Simon pointed out how well they had grown together, in what was "the equivalent of a music miracle."

Video: InTENsity's "X Factor" live shows performance

The final group to perform was also created by the judges from solo contestants at bootcamp. Lakoda Rayne, comprised of four young women, sang Dexys Midnight Runners' "Come On Eileen." The girls finally loosened up and ditched their over-choreographed moves. L.A. Reid told the girls: "If the four of you walked into my office and did what you just did, you'd be signed to a worldwide recording contract." Enough said.

Video: Lakoda Rayne sing "Come On Eileen"

The Brewer Boys were eliminated, and Stereo Hogzz, InTENsity and Lakoda Rayne will continue on.

NEXT: The Over 30s and Girls Perform;
Simon Is Forced To Eliminate Two More Contestants