Owen Coyle's visitors - 22nd in the Sky Bet Championship ahead of the clash - belied their lowly position in the standings to record just a second away win of the campaign.
Defender Charlie Mulgrew notched the decisive goal, volleying home from Liam Feeney's cross after Craig Conway's corner was played short.
United remain well-placed at the top of the table, though their lead over Brighton & Hove Albion has been cut to just two points following the Seagulls' comeback victory over Fulham.
And Yedlin - who returned to the side after missing Sunday's win at Leeds United - admitted that Newcastle never really got going in front of a capacity crowd at St. James'.
"We didn't have a good start, and I think that kind of lead into the rest of the game," he told nufc.co.uk. "It just shows that we're not invincible and we can get beat. We're obviously disappointed, but we have got to stay positive and look forward to the next game.
"If anything, this is a bit of a wake-up call. It's good that everybody is disappointed because it shows what level our team is at."
Ironically, the game's pivotal moment came hot on the heels of United's brightest spell, Matt Ritchie and stand-in skipper Jonjo Shelvey both having come close to breaking the deadlock at the other end.
Yedlin was disappointed the Magpies let their guard down for Rovers' winner, but feels his teammates will be raring to go for their next assignment - Tuesday's EFL Cup quarter-final at Premier League outfit Hull City.
"When you're putting the pressure on, the other team can obviously counter against you, and those are the times where you have to be the most careful," he said. "They had a few corners and got the better of us on one of them. Sometimes, that's the way the game goes.
"Defensively, I thought they put some pressure on us, but in terms of keeping the ball out, we did alright. We could have finished our chances a bit better, because we did have a lot of chances as well. The game could have gone either way, but unfortunately for us it didn't go our way today.
"Now the focus goes to the Hull game, and we'll all be ready to go for that, especially after this one."
