France were just 12-9 ahead with 13 minutes to go before Gael Fickou, Baptiste Serin and Jefferson Poirot slotted tries to secure victory against the underdogs.
The bonus point keeps the pressure on England, who play Argentina on Saturday in another Pool C clash of the heavyweights.
"It was a very difficult match but the important thing is that we won with a bonus point," said fly-half Camille Lopez, who picked up the man-of-the-match award.
USA captain Blaine Scully was delighted his team had been able to push France, three-time World Cup finalists, until the closing stages.
"It's where we want to be. We want to be testing ourselves in the best competitions against the best teams in the world," Scully said.
Barrett brothers to the fore as All Blacks go marching on
Two-time defending champions New Zealand ran nine tries past Canada in a 63-0 romp to strengthen their grip on Pool B at the Rugby World Cup on Wednesday.
All three Barrett brothers -- Jordie, Beauden and Scott -- scored tries as the All Blacks totted up four before half-time and added another five in the second period.