Jason McAteer, back in Sunderland's midfield after a 16-week absence due to a double hernia operation, was adjudged to have unfairly challenged Heidar Helguson and referee Mike Dean pointed to the penalty spot.
But Sunderland's players and fans alike were further incensed when the referee awarded the spot-kick to be retaken after Thomas Sorensen had beaten out Tommy Smith's first effort.
Watford's leading scorer made no mistake at the second attempt and sent his side through to the last eight with even manager Ray Lewington admitting he was at a loss to explain the bizarre decision.
"Sunderland will feel unlucky," he said. "You don't normally get those decisions when you are away from home. I half turned away when the penalty was saved and one of the subs said it had to be taken again.
"I am grateful we got it, but I don't know why the penalty was given. It was a fantastic result. We got the rub of the green a couple of times, but I think we deserved the bit of luck we got."
Chances were few and far between, but Kevin Phillips should have sent Sunderland on their way in the ninth minute when released by Tore Andre Flo only to miss the target with only goalkeeper Alec Chamberlain to beat.
Jody Craddock had a close-range shot blocked from another good opportunity but it was until the latter stages that the home side exerted any real pressure on the Hornets goal.
A disappointing Flo was replaced by David Bellion and Moroccan international Talal El Karkouri came on for Julio Arca, but apart from Chamberlain saving in style from a Craddock header following a series of corners, Watford held on reasonably comfortably.