It seemed certain that the match would be settled by a penalty shoot-out until the South Africa international struck in the closing stages of extra-time.
Zurab Khizanishvili hit the post for Sam Allardyce's side before Northern Ireland international David Healy put the visitors ahead with their first attack of the game.
Healy reacted quickly after Daryl Murphy's shot had spun off Ryan Nelsen and slotted the ball past Paul Robinson to register only his second goal of the season.
Black Cats keeper Craig Gordon, starting his first game after a ten-week injury absence, kept out efforts from Keith Treacy and Jason Roberts as Rovers pushed for an equaliser.
Steed Malbranque almost doubled Sunderland's lead before Roque Santa Cruz wasted a glorious chance, heading wide from four yards.
Rovers did eventually equalise in the 37th minute when Aaron Mokoena scored a rare goal with a brilliant shot.
Santa Cruz saw Gordon tip his header from six yards over the bar while at the other end Grant Leadbitter narrowly missed Robinson's goal with a well-struck right-foot effort.
Carlos Villanueva almost snatched a winner for Rovers with a free-kick that Gordon tipped over, while Andy Reid went close for Ricky Sbragia's side in the closing stages of normal time.
Just when it seemed spot-kicks were on the cards substitute McCarthy converted Villaneuva's cross to snatch victory for the home side and earn them a tie with Coventry City in the last 16.