After enjoying the best of the early exchanges Palace fell behind to McVeigh's measured left-foot shot from 16 yards when he was put through by a pass from Roberts, who was booked five minutes later for a foul.
The visitors continued to look dangerous. Julian Gray went close with a lob and Andrew Johnson did likewise with a header.
Phil Mulryne was booked for a foul on Johnson in the 20th minute and Steven Thomson followed him into the notebook for a foul on Mulryne five minutes later.
Following an earlier injury, Craig Fleming had to be substituted by Malky Mackay on the half hour.
David Neilsen had a fierce shot saved by the Palace keeper just before half time.
A minute into the second half Neilsen collided with Gray as he burst into the Palace penalty area, but when the ball broke to Roberts the big Welshman fired well wide of the target.
MacKay got his head to a Mulyrne free-kick but the ball flew straight at the keeper.
Darren Kenton was inches away from scoring with a fierce drive on the hour and Mulyrne's free-kick was just wide of the same post a minute later.
Tommy Black made way for Gareth Williams in the 66th minute and Aki Riihilahti came on for Shaun Derry eight minutes from time.
For Norwich Nicky Southall replaced Mark Rivers seven minutes from time.
Roberts grabbed his goal after 87 minutes when he controlled Gary Holt's headed throughball and his right-foot blast from 16 yards flew past the keeper's outstretched left hand into the back of the net to the delight of the Canaries fans.