About 50 family and friends accompanied Juppé to the funeral at the Givat Shaul cemetery on the western outskirts of Jerusalem.
Four coffins containing the bodies of 30-year-old teacher Jonathan Sandler, his sons Arieh, 5, and Gabriel, 4, and seven-year-old Myriam Monsonego, were flown from Toulouse to Paris on Tuesday before continuing to Israel, where the two bereaved families had asked that they be buried.
Speaking shortly before the funerals, Juppe said the Toulouse attack had bound Israel and France together.
"In some ways, it was the blood of our two countries that flowed on Monday at the Ozar Hatorah school," he told reporters after meeting Israeli President Shimon Peres, saying he had come "to share in the grief of the Monsonego and Sandler families".
"He who looks for a justification for hatred, needs to know that there is no explanation for hatred," Israeli parliamentary speaker Reuven Rivlin told the mourners.
"There is no explanation for the murder of many pupils in Toulouse. There is not, and there will never be any explanation for acts of terror against Jews wherever such things happen."
As an elite unit of French police laid siege to a flat in Toulouse, Interior Minister Claude Guéant reported mid-morning that the suspect had stopped talking to the police after earlier saying that he had agreed to give himself up in the afternoon.
Palestinian representatives in France have condemned the attack, which the suspected killer claimed was revenge for the deaths of Palestinian children in the conflict with Israel.