The 32-year-old was arrested Friday over an incident in the early hours of August 8 in which an 11-year-old girl was attacked as she slept with her younger sister in a tent at Saint-Didier-sous-Aubenas in the central Ardeche region.
The suspect, who was due to appear before a judge later Sunday, subsequently confessed to four other recent attacks on girls holidaying in an area popular with nature-loving campers from all over Europe.
Three of the victims were French, one Dutch and one German.
The prosecutor in charge of the case confirmed that DNA evidence had helped police track the man to his home in the tiny village of Brune, suggesting he had some kind of record of criminal activity.
According to neighbours, local police were informed 18 months ago that the suspect, a divorced tiler who lived alone, had been downloading pornographic images of children.
"About a year and a half ago, myself and some friends were trying to find a video on his computer and we came across pictures of rapes and other paedophile images in his browsing history," said Warren, an 18-year-old who described himself as having been a friend of the man until then.
Officers at the nearest police station, in nearby Pouzin, declined to comment when asked to clarify what exactly they had been told at the time.
Police are also expected to face questions as to whether they should have been quicker to alert tourists to the potential threat.
The girl attacked last week and her sister were sleeping on their own next to the caravan occupied by their parents.
Just hours before the suspect's arrest on Friday evening, prosecutor Franck Alzingre had told reporters there was no evidence that a serial attacker was operating in the area.
Local tourist chiefs, apparently concerned about the potential impact of the situation on visitor numbers, also repeatedly expressed scepticism about what they described as mere "rumours".