The chauffeur has been cleared of suspicions that he tipped off the robbers, via his brother, that bodyguard Pascal Duvier was absent, thus allowing them to break in without any obstruction.
Duvier was providing security for Kim's sister Kourtney in a Paris nightclub at the time.
Two other people, a man and a woman, were also released without charge on Wednesday morning.
60-year-old, 72-year-old suspected
Of the 14 who remain in detention, five are believed to have taken part in the heist.
The oldest, who was arrested in the French Alps town of Grasse, is 72 and has a criminal record for drug trafficking and possession of firearms.
Another is 60-years-old.
Traces of his DNA are reported to have been found on the cuffs and gag used to tie Kardashian up in her bathroom during the raid.
Investigators tailed him and he led them to the rest of the gang, according to sources.
A 65-year-old woman was also among the detainees, while the youngest is 23.
Links to organised crime
Several of the suspects have links to organised crime and have already been convicted of counterfeiting, drug trafficking or involvement in a jewellery heist in the south France about 10 years ago.
The other nine detainees, one of whom has been convicted of involvement in hijacking security trucks in 2011, are suspected of receiving the stolen goods.
Two of the suspects were found to have travelled to Antwerp, the centre of the European diamond trade, after the heist.
The jewels themselves had yet to be found on Wednesday morning but, according to Le Figaro newspaper, 300,000 euros in cash were seized during Monday's raids.
Kardashian may be asked to identify her assailants, either in a line-up or by looking at photos.