Abiy's trip to Germany comes after a visit to France, where he was praised by President Emmanuel Macron for his "unprecedented" reforms.
In six months, Abiy has secured a peace deal with Eritrea, revamped Ethiopia's security apparatus, released political prisoners and recently appointed a 50 percent female cabinet.
It's this reform-minded approach that has got investors in Berlin talking.
"The new prime minister is seen as a reforming person, trying to do something for Ethiopia and open it up a little bit," explains Robert Kappel, a professor at Leipzig University.
Abiy is one of 12 African heads of state to attend the Berlin summit for Compact with Africa project, an investors' conference, that included Egyptian leader Abdel-Fatteh el-Sissi and Rwandan President Paul Kagame.
"German investors have seen Ethiopia as a hub, also because China is investing there, and Germany will also do something quite similar to get German investors to invest more in Ethiopia," Kappel told RFI.
With a GDP growth rate average 10.8 percent annually, compared to a regional average of less than five percent, it's easy to see why Ethiopia is one of the African countries described by Chancellor Angela Merkel as having "huge potential".
Reforms for investment
It is also one of the few countries that has gone out of its way to impose ambitious reforms, a key requisite to receiving development aid.
"The promising steps by Prime Minister Abiy Ahmed" are likely to be supported by a new reform partnership announced by Chancellor Merkel," says Stephan Exo-Kreischer, Director of ONE, a lobby group campaigning against extreme poverty.
Sources are indicating the partnership could be worth as much as one billion euros.
"If a country shows a willingness to reform in the governance sector or in a different sector, then the German development cooperation will help support that reform by additional financial resources," Exo-Kreischer told RFI.
Merkel claims a partnership with Africa will be a win-win situation, launching her Compact with Africa initiative last September, notably to tackle underdevelopment on the continent, which has led to mass migration and intense pressure on her at home.
Critics however complain that the project is one-sided.
"One big problem is that they're too single-minded, focusing on private sector investment and especially foreign direct investment,"continues Exo-Kreischer.
"They neglect that in order to have a healthy and inclusive growth, you need a strong social sector."
Reforms undermined by ethnic conflict
Yet the appeal of Ethiopia to investors, namely its reforms, risk being unravelled by the country's ongoing conflict involving the Oromo people.
Security forces on Tuesday clashed with members of an armed rebel group, the Oromo Liberation Front (OLF) following separate clashes at the weekend.
"It is really important to understand that this is a country emerging from a highly divisive, fractious period for almost 27 years," explains Awol Allo, a law professor at Britain's Keele University.
Abiy has shown restraint in dealing with his country's ethnic tensions, careful not to fall into the trap of his predecessors, who were renowned for their heavy-handed approach.
However, critics say he has his work cut out for him to impose his leadership without tipping into authoritarianism.
"There's a culture in Ethiopia that leaders are decisive, that they use force," says Allo.
"The fact that the government is restraining is being interpreted by some actors that the government is no longer in control and does not have the ability it had in the past."
Authoritarianism may not be what Prime Minister Abiy wants but, the longer the ethnic disputes in his country continue, the more the pressure will be on him to go down that road.
"I don't’ think the government can continue to defer that power that it has to use and make sure that it alone has the monopoly of violence," concludes Allo.