"Islam still provokes misunderstandings, prejudices and is rejected by some citizens," Valls said. "Yet Islam is here to stay in France. It's the second largest religious group in our country."
About 150 Muslim community leaders attended the meeting with top government officials and ministers.
Although the conference did not discuss radicalisation, judging the question too sensitive, Valls was at pains to distinguish between mainstream Islam and behaviour he deems unacceptable.
"We must say all of this is not Islam," said Valls. "The hate speech, anti-Semitism that hides behind anti-Zionism and hate for Israel... the self-proclaimed imams in our neighbourhoods and our prisons who are promoting violence and terrorism."
Muslim groups refused to attend a meeting of former president Nicolas Sarkozy's Républicains party - formerly the UMP - on the "question of Islam" in France, claiming that it "stigmatised" the religion.
There are about 2,500 mosques in France and another 300 projects under way, sometimes giving rise to local opposition, as in the Front National-controlled town of Mantes-la-Jolie.
The meeting was to discuss the construction and running mosques, as well as their security and Islam's image in the media.
The government hopes to establish "forums for regular discusson" with a community that has no unified leadership, despite Sarkozy's attempts to create one when he was interior minister.