Delegates cheered and pumped their fists after the results were announced in Addis Abeba. "When you try to do something, you mean that you can do it," said Ahmad after the vote. "If I can't do it, I never stand."
He earlier said his programme was the reform of the administration of CAF to avoid the involvement of politics in the organisation.
Hayatou, 70, is credited with increasing the number of African teams at the World Cup and bringing in extra finance for the continent's competitions.
"Issa Hayatou has done a lot for African football but it is time for him to step back," said George Afriyie, vice president of the Ghana Football Association.
Africa has made an emphatic decision that we are ready for change," said Liberian Football Association president Musa Bility.
Seyi Akinwunmi, vice president of the Nigeria Football Federation, said: "Hayatou is a colossus who has bestrode African football for decades. But it's a new dawn, all should be happy.
"We're Africans and Africans never forget their elders. An elder has been defeated today, but he's still an elder.
"The world has moved on, we really need energy. And the truth is Issa Hayatou is almost 70. There's only so much really that he can do.
"The world, technologically and in terms of the way you see things, has changed. Speaking with Ahmad, I found that he understood those things."
Before the vote, Hayatou called for delegates to vote for unity. In a flurry of last minute campaigning, he said Africa should have 10 teams at the World Cup which will be expanded from 32 to 48 nations from 2026.
Hayatou also underlined his desire to maintain the January-February slot for the the bienniial Africa cup of Nations. The scheduling of the tournament - which started in 1957 - has been frequently criticised because leading European clubs can lose their top stars for up to a month.