After a goalless 90 minutes, which saw Dwight Gayle hit the crossbar for the hosts, the visitors prevailed via penalties as Allan Saint-Maximin and Miguel Almirón saw their spot-kicks saved by Clarets goalkeeper Wayne Hennessey.
Freddie Woodman, who saved a penalty on his Premier League debut against West Ham United, kept out Josh Brownhill's spot-kick in the shoot-out before Almirón's miss left Charlie Taylor to convert the winning penalty.
Steve Bruce made nine changes to the side which suffered a 2-0 defeat at Aston Villa on Saturday and faced a Burnley side who were also targeting a maiden competitive win of the 2021/22 season following league defeats against Brighton and Liverpool.
Playing in front of over 30,000 spectators, the hosts created the first goalscoring opportunity through Jeff Hendrick as the Irish midfielder, who joined the Magpies from Burnley last summer, controlled Dwight Gayle's pass before firing a low effort narrowly wide of the left-hand post.
Forward Jay Rodriguez created the Clarets' sole attempt of the first-half after the one-time England international controlled Phil Bardsley's lofted pass with his chest before arrowing a wild volley over the crossbar.
The Magpies, who reached the quarter-finals of last year's competition, continued their push for an opener as Joelinton saw an effort crucially blocked after Sean Longstaff's intended strike towards goal fell kindly into the Brazilian's path.
In the second-half, attacking the Leazes End, Newcastle went agonisingly close to breaking the deadlock through Hendrick.
After Emil Krafth's cross from the right wing was flicked into Jamal Lewis' path by Hendrick, the left-back's low pass found Joelinton inside the box but after the Brazilian's scuffed shot rolled kindly into Hendrick's path, the midfielder's close-range strike was turned over the crossbar by Hennessey.
Newcastle struck the woodwork minutes later as Gayle raced onto Ciaran Clark's ball in behind the Burnley defence, advancing beyond Clarets defender Nathan Collins before his strike clipped the underside of the bar.
United saw another chance in front of goal go begging as Gayle was denied from close-range by Hennessey. After Javier Manquillo set Joelinton free on the right wing, the forward's cross into the penalty area found the 30-year-old striker but his sliding effort was kept out by the former Crystal Palace shot-stopper.
Manquillo almost registered his first goal for the club in fine fashion after latching onto Joelinton's pass before arrowing a ferocious effort towards goal, forcing Hennessey into tipping the ball over the crossbar.
Steve Bruce deployed Allan Saint-Maximin, Joe Willock and Miguel Almirón from the substitutes' bench but were unable to find a breakthrough in the dying stages as the cup tie headed into a penalty shoot-out.
In front of a fixated Leazes End, Saint-Maximin stepped up first for United in the shoot-out but the Frenchman's effort was saved by Hennessey, who guessed the correct way to parry the ball away from goal.
After successful conversions from Willock, Joelinton and Longstaff, Woodman then produced a vital stop to deny Brownhill from 12 yards but Almirón was unable to dispatch the following penalty as the Paraguayan's effort was kept out by Hennessey.
Taylor then confirmed Burnley's passage into round three after his attempt from 12 yards squeezed past Woodman into the bottom right-hand corner.
Newcastle United: Freddie Woodman, Javier Manquillo (Allan Saint-Maximin 76), Emil Krafth, Jamaal Lascelles (c), Ciaran Clark, Jamal Lewis, Jeff Hendrick, Sean Longstaff, Ryan Fraser (Miguel Almirón 83), Joelinton, Dwight Gayle (Joe Willock 76).
Subs not used: Mark Gillespie, Jacob Murphy, Fabian Schär, Matty Longstaff.
