Jes himself saw it as highly unlikely that he wouldn't have seen her. He remembered seeing her earlier for no reason, so why would he forget if he saw her again? It's possible, but unlikely imo.
The single fly in my ointment is that it appears Jes and Jane had played tennis together. I believe this is in one of Jane's statements or Jes'.
However, I still don't see it as important.
One's brain is purpose built to dump irrelevant trivia. Please don't ask me for a cite, I did all the research perhaps 10 years ago, and AFAIK, it is still on blogspot.
There are many reasons I was interested in this. Let me pick one example. Memory. (An important part of learning).
To memorise something, step 1 is to focus on it 100% for 6 or more seconds. That gets it from your frontal cortex to the rear of your brain.
Step 2 is to go to sleep, without filling up your brain with alcohol, medicinal compounds, or simply more important or exiting things. The rear of your brain is akin to a cup. Once it is full, when you pour more in, something has to come out. That's why it is tough to learn if you have late-night parties, play interactive X Box games to the small hours, or watch TV dramas after the news.
Step 3. Sleep. Your brain physically rebuilds and alters itself if you sleep properly. A simple cheat is to have a siesta long enough to dream. That is your brain doing the work of moving information from short term storage in the rear of your brain to permanent storage in your bonce.
This is a very long winded way of saying I can't see why this encounter should have been misremembered, in any significant way.
Speculation. Gerry does not see Jane, so he does not remember the details. Jes sees Jane, but it does not occupy his focus for long enough to sink in. Then he goes to sleep, and is woken up on early 4 May and cannot remember a man with child in his arms.
I see nothing odd here.