The hosts had lost by the same scoreline four days earlier against Portsmouth and, just as against Pompey, the Red and Whites will have felt hard done by.
An inspirational substitution and a stroke of bad luck proved costly for the north-east side. But just as against Portsmouth the chief architect of their downfall were the Black Cats themselves and their inability to turn possession into goals.
There was no repeat of the slick first-half show against Tony Adams' side but the home team had edged proceedings against a Rovers eleven that didn't have a single shot on target before substitute Roque Santa Cruz found the net.
Martin Olsson's 65th-minute cross was flicked on by Christopher Samba and Santa Cruz nodded home with his first touch of the game, having come off the bench to replace Andy Haworth just two minutes earlier.
Things went from bad to worse for Sunderland five minutes later. Matt Derbyshire's cross was cut out by Marton Fulop but the Hungarian keeper could only push the ball into the path of team-mate Phil Bardsley and the defender could do nothing to stop it from bouncing off him and into the net.
There was hope for the hosts a minute later when surprise inclusion Kenwyne Jones slid the ball under Paul Robinson to pull a goal back.
Just 24 hours earlier Sunderland manager Roy Keane had given the striker no chance of making the cup-tie, saying he was struggling with a swollen knee.
Keane had also hinted that he would blood a number of the club's youngsters and give several fringe players a chance to shine. But the Wearsiders' starting line-up included nine of the 11 that started against Pompey on Saturday.
However, that didn't help them get off to a flyer as both sides played in fits and starts throughout the first half.
In truth the hosts probably just edged it in terms of possession but they were unable to build on this slight advantage ahead of the break, even if they did come closest to opening the scoring.
Just after the half-hour, Danny Collins crossed in from the left and Jones rose to crash a header down off the bar but the ball somehow stopped out and Rovers managed to clear their lines.
Apart from a number of long-range efforts from both Sunderland and Rovers, the first half lacked any real impetus from either side in the final third.
In fact the game looked to be heading for extra-time before the introduction of Santa Cruz sparked his side into life and caught the Wearsiders off guard.
Keane's mean pressed for a late equaliser and Rovers were encamped in their own half for long spells in the dying minutes.
However, they failed to find the breakthrough they craved and slumped to their fourth successive defeat and out of the Carling Cup.
The two sides meet again in the League on Saturday. This time Rovers have home advantage and Sunderland will be hoping the shoe is on the other foot.