"This is our land. We will not give an inch of it. Russia and its president should know that," Yatsenyuk told a rally of thousands of people to mark the 200th anniversary of the birth of 19th-century national hero Taras Shevchenko.
Rallies were also planned in Simferopol and Sevastopol in Crimea, where pro-Russian forces have taken control.
A pro-Russian counter-rally was called in the eastern city of Donetsk with predictions that clashes could take place with Ukrainian nationalists.
They insisted on the "need for Russia to withdraw forces sent to Crimea since the end of February and to do everything to allow the deployment of international observers," a statement from the French presidency said.
"If there's a lack progress in this direction, new measures will be taken which would noticeably affect relations between the international community and Russia, which is in no-one's interest," it went on.
The US has already imposed visa bans and set the stage for further sanctions agaust Russia, which denies Ukraine's claim that it has sent 5,000 more troops than allowed under existing agreements to Crimea.
Hollande and Obama called fro the formation of a contact group for Ukraine and Russia to engage in dialogue and declared their support for the government in Kiev.
Obama also called British Prime Minister David Cameron and Italian leader Matteo Renzi.
Forty gunmen wearing balaclavas and combat fatigues fired shots to prevent military and civilian observers from the Organisation of Security and Cooperation in Europe entering Crimea to monitor the 16 March referendum on joining Russia, the OSCE said late Saturday.