All seems very plausible, dates match up near perfect. Seems that James changed his name back to Armstrong after his mother, Dorothy died in 1868. Because he is stated on the 1871 Census at the same address, same occupation & with Wife Ann. Perhaps Dorothy let him into her secret past on her death bed?
Anyway many thanks for the invaluable information, just wish l had the same access to the records that you have!