"No tax merits putting the unity of the nation in danger," the prime minister said in a televised address, adding that the anger on the streets "originates from the profound injustice: of not being able to live with dignity from one's work".
Edouard Philippe added that any future protests would have to be declared in advance and "take place calmly".
He also announced that increases in the cost of gas and electricity, also set to take effect from January 1, would be suspended for three months this winter.
Philippe added that a tightening of the technical assessment for cars, which was set to penalise heavily polluting older vehicles, would also be suspended for six months.
Protests over the fuel hikes have blocked French roads and petrol refineries for the last two weeks.
Philippe acknowledged that France had some of the highest taxes in Europe.
He repeated his earlier condemnation of the violence in Paris at the weekend and thanked France's security forces.
Reactions in Bordeaux
The spokesperson for the Yellow Friday Revolution, a yellow vest movement in the Bordeaux region with 6,000 members on Facebook. says "we are absolutely not satisfied with Edouard Philippe's speech. He is only calling for calm.
"Our actions will continue because globally nothing has changed.
"There are no actions to make our lives better today. There are no measures of improvement. They are just suspending the measures that would've made our life worse."
European Finance ministers meeting in Brussels
Finance Minister Bruno Le Maire said on Tuesday that France would stick to its EU commitments to slash public spending, despite the cost of the decision to suspend the fuel tax.
"There is a course set by the French President Emmanuel Macron, which is to respect our European commitments, reduce spending, reduce debt and reduce taxes, and that course will be maintained," Le Maire told journalists in Brussels.
The suspension could lead to a shortfall of nearly two billion euros in France's 2019 budget plans, equivalent to 0.1 percentage points of gross domestic product.