"We will remain here until all of us have received our papers and until those who come after us, have the right to stay," said representatives of the Black Vests, who took their name from the Yellow Vest protest movement that disrupted France for seven months.
Their sit-in was called by the migrants rights group Chapelle Debout, which said it had written to the prime minister "weeks ago" to complain about the conditions for those seeking asylum, failed seekers and undocumented workers.
Police were dispatched to the area in Paris' Latin Quarter, but no incidents were reported.
Inside the Pantheon, a necropolis of France’s greatest citizens and a popular national monument, the atmosphere remained peaceful.
"There are plenty of speeches about racism, and our rights," one member of the collective Chapelle Debout said, insisting that "all tourists had been evacuated."
The incident unleashed a torrent of reaction on social media, with far-right leader Marine Le Pen saying it was unacceptable for illegal migrants to invade a building as prestigious as the Pantheon "in total impunity."
The Paris prefecture ordered the site to be evacuated, although risked clashing with protesters who warned they would not leave until they were met by Prime minister Edouard Philippe.
Friday's sit-in came a day after President Emmanuel Macron welcomed members of the African diaspora to the Elysée palace for talks on building a new partnership with Africa.
The gathering was slammed by critics as being an elitisit affair that failed to represent the entire African community.