It could have been worse for the home side as Shakers substitute Glynn Hurst fired into the side-netting when he should have scored.
Norwich kicked off attacking the Barclay End goal and should have taken an early lead when Lee Croft played Darren Huckerby's cross back across the face of goal for Chris Martin to head wide.
In the fifth minute Dion Dublin headed wide from Mark Fotheringham's free-kick before Matty Pattison failed to find the target with a long-range free-kick.
However, it was Bury's David Buchanan who had the first on-target attempt, his header going straight to David Marshall.
Martin then fired wide after being set up by Huckerby and in the 39th minute Jamie Cureton fired wide of the near post after Mo Camara had crossed low from the left.
Ryan Jarvis replaced the out-of-sorts Martin for the second half but the first shot after the break came from Richie Baker.
Jon Otsemobor then replaced Pattison so that Darel Russell could move from right-back to midfield.
New loan signing Ryan Bertrand replaced Croft just after he had given away a free-kick on the Bury left.
When Brian Barry-Murphy delivered the ball, Bishop met it at the far post to score with a powerful header over Marshall's despairing attempt to keep the ball out.
The Championship side were stung into action, but Doherty's goal-bound shot was blocked by Steve Haslam.
However, Doherty did find an equaliser in the 80th minute when he blasted a right-foot shot past Provett from close range after Mark Fotheringham's shot had been blocked.
Provett then saved from Huckerby in a one-on-one situation and kept out Otsemobor's header in stoppage time.