I’m not aware of a name, and nothing is shown on a detailed Hexham street map. The alleyway does go from Fore Street to what used to be the Goose Market (the cobbled area before you get to Hallgate), so maybe there was a name associated with that?
The alleyway is between Nos 12 and 14. I believe the Sun Inn was No 36 and there is another alleyway there. The White Heart was around No 44, so again not associated with the “Goose Market” alleyway.
There are a number of photos of both alleyways in the Fore Street Gallery of the Photo Archive: https://www.rogerb129.sg-host.com//historic-hexham/photograph-archive/photo-archive-choices/hexham-choices/streets-choices/fore-street-gallery/