The young Magpies sealed their play-off qualification in Premier League 2 but were made to come from two goals down after Gabriel Kyeremateng and Daniel Jarvis netted for the Potters, with Bailey missing a penalty with the scoreline at 1-0.
However, the United skipper began the comeback with a powerful header just past the hour mark before Yannick Touré levelled to quickly turn the tie around for the United.
Bailey went on to grab his brace and United's match-winner, ending the night on a high after a difficult start for the Newcastle-born defender.
"It was a ridiculous game and it sums up what football is all about," he told nufc.co.uk. "At half-time, I've probably never been as low for a long time. The lads were the same as we weren't playing our normal stuff.
"Missing a penalty affected me and we were a bit lost after going two goals down at half-time. The way the lads have responded in the second half was unbelievable. It's a massive result. Promotion has been something that we've been aspiring for in recent seasons and a win to secure a play-off spot is a massive step towards that. Hopefully, we can continue that and try to go up.
"All we can do is concentrate on ourselves. It isn't in our hands anymore so all we can do is try to win our remaining games and see where that will take us. We have to rely on other results going in our favour but hopefully they can go in our way.
Bailey was full of praise for his team-mates' support towards the captain after an unfortunate first-half but was left jumping for joy after netting the winning goal in the closing stages.
Surprisingly, the centre-back had also netted and missed a penalty in United's previous match - a 3-2 victory over Derby County in the Premier League Cup - but was pleased to achieve three points in their bid for promotion to Division One.
He added: "After missing the penalty and going behind by two goals, we have shown great character and resilience to get the win after it looked like we were out of the game.
"The management said at half-time to try and respond and make sure the players help me out after I missed the penalty. All the lads did that and I can't thank them enough for their brilliant reaction. It shows what great character we've got in the team.
"I've signalled to Matty (Longstaff) to put one towards the back post and, after he was another great ball. I've managed to peel away from the defender and head it across goal but, luckily, it's got straight into the goal and it's helped to get us a result we wanted.
"I can't remember much after scoring my second. All the lads were jumping on top of me and were buzzing but it was a great feeling for everyone involved.
"We have two games left (against Manchester United and Middlesbrough) in the league along with the semi-final of the Premier League Cup (against Leeds). The momentum from this win is massive and hopefully we can use that and kick on to get some positive results and get something good out of the season."
