The committee called for the women to be compensated and for a review of the 2010 law that forbids people from publicly wearing clothing that conceals their face.
"The French law disproportionately harmed the petitioners' right to manifest their religious beliefs," the committee said in a statement.
It added that it was not convinced by France's claim that the ban was necessary for security and social reasons.
The two French women were convicted in 2012 for wearing the niqab, a veil with an opening for the eyes.
"The ban, rather than protecting fully veiled women, could have the opposite effect of confining them to their homes, impeding their access to public services and marginalising them," the committee said.
The UN Human Rights Committee, made up of independent experts, ensures countries stick to their human rights commitments but it does not have enforcement powers.
It said the French ban was "too sweeping" but that governments could still make people show their faces in specific circumstances.
'Violated' human rights
The committee's decision reignites a debate that has raged in France for years over Muslim headwear and other religious clothing.
The debate has regularly pitted supporters of the country's secular constitution against those who argue for religious freedoms.
The 2010 law had strong public support when brought in under former president Nicolas Sarkozy. But many said it targeted the tiny minority of Muslim women in France who wear Islamic veils.
Condemned by critics for pandering to far-right voters but backed by many women rights activists, the law made France the first European country to ban garments that cover the face.
An estimated five million Muslims live in France and women who ignore the ban can be fined up to 150 euros.
"The vast majority of cases where people have been stopped for checks and have been condemned to fines... have been women wearing the niqab," committee member Ilze Brands-Kehris said.
"In the context of fewer than 2,000 women wearing the full-face veil in France... (the law has) a vast disproportionate effect on those women."
Other EU countries, including Denmark, Austria and Belgium, have also implemented similar full-face veil bans.
"France violated the human rights of two women by fining them for wearing the niqab," the committee said, adding that measures should be taken to prevent similar violations in future, including a review of the law.
France, which has ratified UN rights treaties, is "under obligation" to comply with the committee's recommendations and act in good faith, Brands-Kehris said.
'Oppression of women'
The committee's stance contrasts with a 2014 European Court of Human Rights ruling which upheld the French ban, rejecting claims that it breached religious freedom.
The court found that France was justified in introducing the ban in the interests of social cohesion as it was "not expressly based on the religious connotation of the clothing".
Explaining the committee's decision, Committee Chair Yuval Shany said a general criminal ban did not allow a "reasonable" balance between public interests and individual rights.
"The decisions are not directed against the notion of secularity, nor are they an endorsement of a custom which many on the committee, including myself, regard as a form of oppression of women," Shany said.
In August, the committee also criticised France over the 2008 sacking of a nursery worker who refused to remove her veil at work, arguing it interfered with her right to manifest her religion.
The so-called Baby Loup case, named after the nursery, had already faced multiple legal battles in the French courts.
Debate about the effectiveness of the "burqa ban" also made headlines earlier this month after it was revealed that French gangster Redoine Faid, who broke out of jail in July using a helicopter and was recaptured three months later, had at times worn a burqa as a disguise.
Police say Faid was caught after officers saw someone wearing a burqa but walking like a man