The demonstrators welcomed Sharaf to his post chanting "Lift your head up high, you're Egyptian", as they did when president Hosni Mubarak stepped down on 11 February after mass protests against his 27-year rule.
Previous premier Ahmed Shafiq stood down on Thursday.
After Mubarak's resignation The Supreme Council of the Armed Forces pledged to oversee a transition to a free democratic system.
But protesters had continued to gather in Tahrir Square after overthrowing Mubarak to demand that the caretaker government headed by Shafiq be purged of members of the old regime.
Friday's protest, which was originally meant to press for change, turned into a massive celebration following news that Shafiq had been replaced by Sharaf.
The new premier, who served briefly as a transport minister under Mubarak, had joined protesters in Tahrir Square during the uprising to demand political and economic change.