Or he was taking a detour to visit a friend....get a pint of milk....had arranged to meet his wife somewhere beyond his direction of travel ? Loads of options.
It is mere speculation but I think it’s likely Totman saw Tanner on her way back to her apartment and that’s why he thought he was Tanner’s sighting. Perhaps because he didn’t see Gerry or Jez that is where the doubt set in ?
I think the pint of milk thing is unlikely; I don't think he would have carried a sleeping child with him to do that.
Depending on the camber of the road and the closeness of Jane would he have necessarily seen the two men even if he had registered Jane? Perhaps? if they had been standing where Gerry thought, but not necessarily where Jane and Jes thought. We don't know what he saw, for all we know it may have been all three from the top of the hill. I don't think it really works, though, and as shining has said the weakness lies in the direction of travel for most of the speculation.
On the other hand, there is another way of considering the situation. Perhaps he wasn't actually walking in the wrong direction at all.
He may have followed what we think is the logical route from the creche to his apartment in block four.
He may have walked uphill and through the junction either just before Gerry and Jes met in the street, or he may even have passed them while they chatted. They didn't see Jane, so what else didn't they see? Particularly if he followed the route Jane did.
He may have seen Tannerman walking out from block 5 as he was walking into block four.
I think that works. It is simple and it cuts out much of the need for speculation.