Thiem returned to court Tuesday for the first time since suffering an ankle injury in the third round of the BNP Paribas Open at Indian Wells.
The two-time French Open semi-finalist came from a set down to book a last-16 clash against either Novak Djokovic or Borna Coric with a 5-7, 7-5, 7-5 victory.
"I had to fight from the first to the last point, which of course I did well. But I was also lucky at the end when he had a match point and missed a forehand by 10 centimetres or so," said Thiem.
The Austrian was 15-40 up on Rublev's serve and clinched victory on his first match point, when Rublev double-faulted with a weak serve into the net.
"I was 10 centimetres from being out of the tournament," a relieved Thiem said. "But I'm happy that I played two hours and 40 [minutes]."
Thiem has reached the French Open semifinals for the past two years.
He next meets 12-time Grand Slam champion Novak Djokovic or Borna Coric of Croatia, who play their second-round match on Wednesday.