The French U21 international snubbed the Black Cats and Villa, in favour of a move to Pompey.
Although his transfer was dragged out for weeks, Kaboul stated that he was always going to sign for the south coast club.
"It wasn't a difficult choice to make coming here," Kaboul told the club's official website. "Everyone in the papers was saying I was going to Sunderland or Aston Villa but I wanted to come here straight away.
"As soon as I knew they were interested I said: 'I want to come to Pompey'. I came here because the manager has wanted me for two years. I met him two years ago when I was at Auxerre.
"I like the man because he gives you confidence. It's a good opportunity for me because the club has a lot of ambition. I think Pompey are a top-five Premier League side.
"I believe I can achieve what I want to achieve at this club. I enjoyed my year at Tottenham and it taught me many things. I'm still young and I made progress but now I hope to make more progress here."