Suicide is the third highest cause of death behind cancer and heart disease, the report, which the ministry commissioned in 2011, found.
With most farmers being men, the male suicide rate is much higher than that among women - 417 men and 68 women took their own lives from 2007 to 2009.
The suicide rate among male farmers was 20 per cent higher than the French average and was particularly high among cattle farmers aged between 45 and 64.
The death toll hit a peak in 2008 with 146 men and 27 women killing themselves, rising 56 per cent among dairy farmers and 127 per cent among meat producers.
Rising production costs for livestock and dairy farming were largely to blame, the InVs concluded.
Farmers' unions agreed, blaming pressure on prices on the world market and adding that solitude and administrative pressure were contributing factors.
Agriculture Minister Stéphane Le Foll on Thursday promised "determined action" to improve rural incomes.