To applause, he said he wanted a judiciary cleaned of its political affiliations.
Marta Gabriela Michetti was sworn in as vice-president.
Mr Macri then travelled to the presidential palace to receive the sash and baton of office.
Ms Fernandez had insisted that the handover of the symbols of office should also take place in Congress, where her party holds a majority of seats.
She argued this was a tradition established by her and her late husband and predecessor in office, Nestor Kirchner.
'Best and worst' of Fernandez - media round-up
The Buenos Aires Herald says Ms Fernandez's last day showed her "at both her best and her worst" with a "display of petty political mischief" but with "a grip on the popular imagination". She is "leaving not with nostalgia but with a rallying cry", it concludes.
La Prensa describes Ms Fernandez's failure to manage the handover ceremony as "a loss without precedents" and "a bitter lesson learned too late", adding that new president is now firmly "in control".
Left-wing newspaper Pagina 12 is more sympathetic, showing images of the huge crowds of supporters gathered in the Plaza de Mayo, along with the headline: "The Hug."
In Brazil, right-wing magazine Veja writes of "the follies of Cristina Kirchner" while Chile's El Mercurio says Mr Macri is taking power "surrounded by controversy".
Challenges for the new president
Mr Macri argued that according to presidential protocol, the handover should be held in the palace, as it was before 2003.
Local media reported that Mr Macri's decision was probably driven not just by tradition but also by a concern that followers of Ms Fernandez could disrupt the ceremony in Congress.
After Ms Fernandez declined to attend the ceremonies, Mr Macri's party sought a court injunction affirming that her term ended at midnight on Wednesday.
Ms Fernandez told her supporters on Wednesday evening: "I can't speak long because at midnight I turn into a pumpkin."