Voters can cast their ballot in ten thousand polling stations which will remain open till 7 pm with the first results expected to be out starting 8:30 pm.
The two rivals voted around the same time around 10:30 in the morning with Fillon, who is a member of Parliament casting his vote in Paris and Juppé doing so in the city of Bordeaux, of which he is the mayor.
“I am waiting for the verdict of the voters, it is they who speak,” Fillon said at the exit of the polling station. Juppé said he has no regrets. “I have defended my ideas”.
Fillion upset predictions by comprehensively winning the first round last Sunday ahead of Juppé. With more than 44% of votes in the first round, Fillon registered nearly 650,000 votes more than Juppé (28.6%) who had a considerable lead before the televised debate involving all seven candidates which took place two days before the first round.
In addition to most of the Sarkozy supporters, Mr Fillon received the support of Bruno Le Maire and President of the Christian Democratic Party Jean-Frédéric Poisson.
Alain Juppé, on the other hand, was supported by Nathalie Kosciusko-Morizet and Jean-François Copé.
While the candidate of the centre-right is a matter of choosing between Fillon and Juppé, there is still uncertainty surrounding the candidate of the ruling Socialist Party.
From Monday, all eyes will turn to President François Hollande who must announce shortly whether he intends to seek a second mandate or not.
Prime Minister Manuel Valls has not excluded from the possibility of candidacy in the left primary. In an interview with the weekly Journal du Dimanche, he said “I will make my decision with a clear conscience”.
Hollande's former protege and economy minister, 38-year-old Emmanuel Macron, is also set to stand for the presidency as a centrist independent, adding another element of uncertainty to the race.