Formally, Prime Minister-designate Giuseppe Conte presents his cabinet to the president, who must endorse it. Conte, a little-known law professor with no political experiences, met the president on Friday without resolving the deadlock.
Mr Mattarella, however, objected to the choice of Paolo Savona, 81, as economy minister, forcing the prime minister-designate to abandon his efforts to form a government, more than 80 days after the elections.
"I have given up my mandate to form the government of change", said lawyer and political novice Giuseppe Conte, 53, plunging the country into a political crisis almost three months after March's inconclusive general election.
"I agreed to all the ministers except the finance minister", Mattarella told reporters at the presidential palace in Rome.
Italian bonds surged the most since September and stocks rallied after President Sergio Mattarella vetoed populist leaders' choice of a euro-skeptic candidate for finance minister, fueling investor optimism that there are checks and balances in place that are working.
Italy's anti-establishment 5-Star Movement is considering campaigning together with the far-right League if the nation goes back to the polls, a 5-star source said on Monday.
Yields on 10-year Italian government bonds jumped by more than 20 basis points last week.
Mr Cottarelli, known as "Mr Scissors", has been summoned for talks with the president later Monday.
After winning elections in 1994, the media magnate, who was already in the crosshairs of the Italian justice system, proposed to president Oscar Luigi Scalfaro to nominate his personal lawyer Cesare Previti as Minister of Justice. Giuseppe Conte.... who returned the mandate given to him on May 23 to form the government.
He will struggle to gain the approval of parliament with Five Star and the League commanding a majority in both houses.
Uncertainty over Italy's political situation has rattled financial markets, driving up the country's borrowing costs and this "increases our public debt and reduces the funds available for public spending", Mattarella said. Mainstream center-left and center-right parties were seen losing further ground in the face of voter anger over the sluggish economy.
Politics in Italy took another dramatic turn overnight, as the tentative right-wing coalition which formed government in early May unraveled. Under that clause, parliament can demand a president leave office if a simple majority of lawmakers votes in favor.
Adding an element of constitutional risk, members of the Five Star party subsequently called for Mattarella's impeachment for betraying the Italian people.