Leading members of France's mainstream right opposition - formerly the UMP, now Les Républicains - got together on Thursday for a meeting entitled "Islam in France or Islam of France" or, as Sarkozy put it recently, "not what France can do for Islam but what Islam can do to become the Islam of France".
Statements like that have angered Muslim representatives, leading the country's two Islamic umbrella groups to decline the invitation to attend, even though one of them, the CFCM was set up by Sarkozy when he was interior minister.
"It is unacceptable to participate in this convention," CFCM executive committee member Abdallah Zekri told Le Monde newspaper.
In March Sarkozy told CFCM leaders that French Muslims must change some of their practises, calling for the ban on the Islamic headscarf to be extended from schools to universities and for halal meals in schools to be scrapped.
Sarkozy has toughened his stance since January's Charlie Hebdo attacks and, with the far-right Front National stealing his party's votes, he recently declared that talk of "integration" is out of date, calling for "assimilation" of Muslims into French society.
The spokesperson for the Socialist government, Stéphane Le Foll, said he was "extremely worried" by the meeting, asking "where this party that claims to be republican intends to go".
Last month the vice-president of what was then the UMP, Nathalie Kosciusko-Morizet declared the meeting a "bad idea because this question is not the only subject".
And Sarkozy's competitors for to be party candidate for the presidency in 2017 - Alain Juppé, François Fillon and Bruno Le Maire - gave to understand they would not be going.
Sarkozy on Wednesday told a public meeting that the party should not "run away from debate".
And Henri Guaino, one of the MPs charged with drawing up a report of the meeting's conclusions, accused critics of being a "coalition of great consciences and ostriches".
"Shouldn't we talk about subjects that divide," he asked on Europe 1 radio. "If you talk about immigration, you're a xenophobe, if you talk about security, you're a fascist, if you talk about Islam, you're an Islamophobe."