United had led at St. James’ Park through Aleksandar Mitrović, but a quickfire double from Jason Cummings had put a dominant Forest ahead before Rolando Aarons’ stunning leveller just before half time.
With game having gone to extra time after a goalless second period, Walker’s close range finish made it 3-2 and won it for the visitors, sending the four-time winners of the competition into the third round at the expense of Rafa Benítez’s men.
Benítez made a raft of changes to his side – ten in all – after Sunday’s defat at Huddersfield Town. Only defender Chancel Mbemba kept his place, with Jacob Murphy being handed a home debut and youngsters Jamie Sterry and Dan Barlaser among those who were handed their first starts of the campaign.
And they got off to a fine barely two minutes into the contest, when Aarons powered forward and squared the ball for Mitrović, who sidestepped his man before thumping a shot past a helpless Jordan Smith.
Aside from a speculative Mohamed Diamé effort from just inside the Forest half, Newcastle didn’t test Smith too much after their opener, and they were pegged back just before the half hour.
Mark Warburton’s side – who had enjoyed plenty of possession since falling behind – levelled when Zach Clough’s cross was met by Cummings, who stooped to nod past stand-in skipper Karl Darlow.
And barely two minutes after drawing level, Forest took the lead. United were opened up by a long ball from Danny Fox split the defence, and former Hibernian forward Cummings raced onto it before lifting his shot over a stranded Darlow to turn the tie on its head.
A mix-up in the area between Darlow and Grant Hanley led to another chance for Warburton’s men, but the Magpies goalkeeper – a former Forest player – produced a fine stop to deny Clough as he went one-on-one after beating his man.
But just before the break, and somewhat against the run of play, Newcastle equalised. With the Championship side on the back foot momentarily, Aarons took full advantage, shifting the ball onto his unfavoured right foot before planting an unstoppable drive into the top left-hand corner from just outside the box.
Six minutes after the restart, Murphy almost had a goal to remember his home debut by, curling in a shot with his left foot which Smith did well to get to and tip over the crossbar.
Henri Saivet then came close, running on to a Mitrović flick on, but he saw his close-range effort blocked and then scrambled away by the visitors’ backline, and Murphy had another go – this time his shot, which followed an excellent run, was too high.
The tide hadn’t turned completely, though, as Forest retained a counter-attacking threat. Cummings was played in on the right by Clough but after cutting inside in search of his hat-trick, his shot flew over, with Kieran Dowell well-placed in the middle.
The tie began to drift towards extra time, with Murphy going close once again, this time when played in by Mitrović, but his shot flew high into the Gallowgate End. The Serbian forward then met a cross from substitute Achraf Lazaar with his head and another sub, Matt Ritchie, took aim but still, United could not find a way through.
Forest almost snatched it at the death with just seconds left of normal time, and it was Cummings again – this time with a curling free kick that cleared the wall but cannoned off the crossbar and over.
Former Magpies forward Daryl Murphy was introduced in place of Cummings by Warburton in the first period of extra time, and immediately after his introduction the winning goal was scored, as Walker got a touch to Jack Hobbs’ header to divert it past Darlow to send Forest through.
