Froome, who fell into a ditch during the first stage, and other favourites Richie Porte and Adam Yates trailed Gaviria in the overall standings by one minute one second.
"I'm the first Colombian to wear the yellow jersey for 15 years," said the 23-year-old Colombian, who is on his first-ever Tour de France and who already has 33 wins to his name.
Colombia's cycling-mad president Juan Manuel Santos, who visited the Tour de France last year, rushed to congratulate him on Twitter.
Colombia's president congratules Fernando Gaviria
Muchos éxitos a @NairoQuinCo, @Eganbernal, @FndoGaviria, @UranRigoberto, @danifmartinez96 y @elpumaDarwin que desde hoy compiten en el #TDF2018 ¡Ustedes son orgullo colombiano! pic.twitter.com/Kaff7B1qq9Juan Manuel Santos (@JuanManSantos) 7 juillet 2018
US rider Lawson Craddock, who also fell during the first stage, suffered a fracture line in his left shoulder, as well as a cut over his right eye that left his face covered in blood.
It is the 26-year-old's second Tour de France.