Firmino started the match as a substitute after suffering an eye injury in a clash with Tottenham Hotspur's Jan Vertonghen at Wembley on Saturday during a Premier League match.
But the Brazilian came on 18 minutes from time and steered a superb finish past PSG goalkeeper Alphonse Areola to send Anfield in ecstasy.
Firmino celebrated by running towards the crowd with one hand covered over his wounded eye.
"You can bring Bobby Firmino on and that helps a lot. I loved his goal celebration," Klopp said.
That it needed a last minute winner was a testament to PSG's resolve. The hosts raced into a 2-0 lead courtesy of goals from Daniel Sturridge and James Milner.
Thomas Meunier halved the deficit just before half-time and Kylian Mbappé equalised in the 83rd minute.
However PSG, boasting more than 400 million euros of attacking riches in Neymar, Edinson Cavani and Mbappé left Merseyside empty handed.
"It was really important we started like we did. We were so strong and good with the ball," said Klopp. "We needed all of our focus and we covered a lot of yards and closed all of the gaps. We scored wonderful goals, they were thunderballs."
Meunier said his side missed a chance to press home their advantage. "After coming from 2-0 down, the momentum is with us. I thought we should have put them under more pressure and go for the three points. But we stayed deep and allowed them to come at us."
PSG will have the chance to relaunch their campaign on 3 October when they host FK Crvena Zvezda who drew 0-0 on Tuesday night with Napoli at the Stadion Rajko Mitic in Belgrade. Liverpool will face Napoli at the Stadio San Paolo.