The victory at the Riverside gave 'Boro, who have never won a major trophy, a 3-1 aggregate win and they will play Bolton Wanderers in the final at the Millennium Stadium in Cardiff on February 29.
Premier league leaders Arsenal fielded a weakened team and had defender Martin Keown sent off just before halftime for a professional foul.
Dutchman Boudewijn Zenden gave Middlesbrough the lead after 69 minutes. Brazilian Edu equalised for Arsenal with a header but Reyes' own goal secured Middlesbrough's long-suffering fans their ticket to Cardiff.
Middlesbrough, who lost in the League Cup final in 1997 and 1998, have waited 128 years for a piece of silverware.
"We've been so close before and we're there again," manager Steve McClaren told Sky Sports television.
"It's going to be a fantastic day out for the fans but the players have got a job to do. The players were magnificent, absolutely fantastic."
Arsenal manager Arsene Wenger left out Thierry Henry, Dennis Bergkamp, Robert Pires and Freddie Ljungberg, although captain Patrick Vieira returned to stiffen the midfield.
However, Middlesbrough dominated the first half. Italian striker Massimo Maccarone fired one chance wastefully wide
CLUMSY KEOWN
On the stroke of halftime Arsenal did their cause no good when Keown clumsily pulled down the Italian and earned a straight red card, the 54th Arsenal dismissal in Wenger's seven-and-a-half year reign.
Reyes vainly claimed a penalty straight after halftime when he went down under a challenge from 'Boro keeper Mark Schwarzer.
Schwarzer then blocked a Kolo Toure shot on the line but Arsenal went behind when Gaizka Mendieta touched a deft through ball for Zenden who dinked the ball over Stack.
The visitors, whose last League Cup triumph was in 1993, hit back quickly, with Edu netting after Vieira's header rebounded off the post eight minutes later.
However, 'Boro killed the Londoners off five minutes from the end when Reyes tried to tackle Stuart Parnaby but succeeded only in diverting the ball into his own net for an unfortunate first goal for his new club.
Wenger said he thought Reyes should have had a penalty but was pleased with the way his 10-man team battled.
"I'm very proud of the players, they gave absolutely everything but things turned a little bit against us," said the Frenchman.
"They (Middlesbrough) tried to cut the fluency of our game and they did it well."