About 8,500 people gathered in Paris's Place de la République on Saturday afternoon, according to police estimates.
There were smaller rallies in Nantes, Bordeaux, Strasbourg, Toulouse and Lyon.
A poll last week showed 56 per cent of the French were opposed to taking in more migrants and refugees.
Participants in yesterday’s meetings wanted to be the voice of the other 44 per cent, showing solidarity with migrants and echoing the “refugees welcome” slogan in Germany.
“I’m a citizen, a French woman and I’m here to protest against our government’s silence about the refugees,” Michèle, a writer and member of the group that came together to initiate the rally, told RFI.
“This movement began especially on Facebook, it began with a conversation on Facebook, a little talk like 'OK it’s terrible, let’s do something'. The "Let’s do something' was like a match and it set Facebook alight. People were ready to do something, to protest against the situation.”
“I’m against what’s happening," said 75-year-old Gisèle, holding a leaflet saying “Yes to refugees”. "That little three-year old that we found dead on the beach died. It’s shameful to do nothing. Here in the West, we just close our eyes. We shouldn’t.”
“People who say France can’t take in migrants are bloody idiots;” she added.
Engineer Alexandre Lepcheri brought his children to the rally.
“It was a question of solidarity to be together today without specific requests and it’s very important to send a signal to politicians and our government to show that we fully support the refugees," he said. “Also to be with our children to show them there are situations outside Europe in order for them to understand the situation."
He was keen to tell the Socialist government not to let the fear of losing voters to the far-right Front National dictate its policies.
“We saw that some initiatives are coming from Germany and we have to send a signal to Mr Hollande to have the same way of speaking as Mrs Merkel," he said. “France probably can’t [take in as many refugees as Germany] but the question of quotas has to be taken into account.”
Among Syrian refugees present, Ammar Kharboutli, a 35-year old engineer from Damascus, arrived in France six months ago, applying for asylum.
He was brandishing a photo of Aylan Kurdi.
“I could have been in his place because I took the same journey from Turkey to Greece," he said. "I’m lucky to be alive but a lot of people didn’t have this chance to be here.
“I’m here today to say give these people the chance to live.”
He, too, called on France to follow Germany's example.
“I want especially France to take a step. Germany took the step and Austria but we’re waiting for France to do their mission, because they didn’t do their mission four years ago and this is the result. They didn’t stop our dictator Bashar Al-Assad. Aylan is the result of that.”
According to European Commission president Jean-Claude Juncker, France could be called upon accept 27,000 asylum-seekers if the proposed quota system is accepted.
After receiving "several dozen" messages from mayors ready to take refugees in their towns, French Interior Minister Bernard Cazeneuve has invited them to a meeting on Saturday 12 September to discuss how to do so.
France's Chief Rabbi Haïm Korsia called for a complete change in policy in the light of the crisis.
"France, the country of exile and welcome, the cradle of human rights, cannot continue to close its eyes to these women and men who collapse at the door of our borders," he declared at a ceremony in Paris in memory of the "martyrs" of deportation under Nazi occupation.
Former French president and right-wing opposition leader Nicolas Sarkozy called for "administrative retention centres ... under European control in countries surrounding Europe, because the statute of political refugee must be given or refused before the Mediterranean is crossed" at a meeting of his Les Républicains party in western France on Saturday.
As President François Hollande is reported to be considering allowing French airplanes to carry out air strikes on Syria, former prime minister Alain Juppé, a rival to Sarkozy for Les Républicains presidential nomination, backed the idea.
Pointing out that the air force is already carrying out air strikes against the Islamic State armed group in Iraq, Juppé said they should be extended to Syria.
"The question is whether they should be accompanied by a land deployment and that I'm completely against," he said on Sunday.