The reigning Scottish champions were dealt a setback inside the opening four minutes, when striker Kelechi Iheanacho was forced off through injury.
However, Iheanacho’s premature departure presented an opportunity for substitute Johnny Kenny, who stole the ball high up the pitch and raced into the box, only to drive his shot wide of Matteo Bignetti’s far post.
Just moments later, though, the hosts found themselves a goal behind after Tomi Horvat produced a spectacular effort from distance, rifling his left-foot shot into the top corner to score in back-to-back European matches.
Matters nearly got worse for Celtic when Otar Kiteishvili rattled the frame of the goal as the game drifted past the half-hour mark.
After surviving that scare, the Hoops went close to restoring parity when Arne Engels’s goal-bound header forced Emanuel Aiwu to make a last-ditch goalline clearance to protect the visitors’ slender advantage.
The Austrian side threatened to add to their lead immediately after the restart as Horvat saw his free-kick saved before Jon Gorenc Stanković headed wide from the subsequent corner.
Sturm Graz were left to rue their failure to find a second goal when Celtic staged a dramatic turnaround with two goals in three minutes after the hour mark.
The hosts put together a slick short corner routine to work the ball into the box for defender Liam Scales to steer a left-foot shot into the far bottom corner.
Minutes later, Benjamin Nygren made full use of another dangerous corner, heading in from Arne Engels’ inviting delivery to send the decibel levels soaring at Celtic Park.
The visitors’ hopes of rescuing a positive result were dealt a major blow when Tochi Chukwuani was given his marching orders for a clumsy studs-up challenge on Callum McGregor.
Celtic negotiated the closing stages of the contest to seal three precious points to send them soaring up the standings to 21st spot, while Jürgen Säumel’s charges are a point adrift in 26th spot.
Flashscore Man of the Match: Liam Scales (Celtic)