While discussing different kinds of boats with an official at a sea rescue centre in Brittany on Thursday, Macron was caught on camera saying, "the kwassa-kwassa doesn't do much fishing, it carries Comorian".
Kwassa-kwassa are small fishing boats that are often used by migrants to sail from the independent Comoros islands to the French territory of Mayotte.
Mayotte Senator Thani Mohamed Soilihi called them the "boats of death" in a recent parliamentary debate because of the huge number of people who have died trying to make the crossing.
In 2015, 19,000 of France's 20,000 deportations of people attempting to enter its territory illegally were from Mayotte.
Comorians demand apology
Macron's wisecrack sparked outrage in some quarters.
The Council of French Citizens of Comorian Origin demanded a public apology for the "racist and dehumanising declarations".
Putting the death toll from sunken craft at 12,000, Green politician and former housing minister Cécile Duflot tweeted that if right-wing former president Nicolas Sarkozy had made such a remark "the outcry would have been gigantic".
Other left-wingers also slammed the new president.
"More than an apology, France has a duty of hospitality," Communist Party chief Pierre Laurent tweeted and former presidential candidate Jean-Luc Mélenchon posted a video of a campaign meeting during which he paid tribute to migrants who have died at sea.
On the right, Republicans parliamentary election campaign chief François Baroin called the remark "shocking", while National Front vice-president Florian Philippot accused the media of being soft on Macron.
National Front leader Marine Le Pen, who lost to Macron in the presidential election's final round, said Sunday that he "should not be laughing at the situation ... not only because there are people losing their lives but also because our compatriots [in Mayotte] are experiencing a hellish situation, a real nightmare, being swamped by illegal immigration".
Sympathetic media coverage
Although Macron has made no public reference to the row, his entourage has admitted to the media that the joke was "not very fortunate" and "untimely".
Apart from allegations of favouritism against Housing and Urban Planning Minister Richard Ferrand, the kwassa-kwassa controversy has attracted practically the only negative media coverage for Macron since he took office on 14 May.
The "juxtaposition of pleasant images, which are contributing to creating a climate of optimism to which the country aspires, ... should not deprive it of lucidity", commented Socialist former prime minister Bernard Cazeneuve in an interview publish Sunday.