I wonder who else there was that he could have turned to. My neighbour -90+- insists that he was the recipient of the "I must never turn my back on that young man" statement but no mention of a planned killing. Maybe BW was told in a moment of desperation/weakness.
PS I know plenty of men "of a certain age and background" who wouldn't think twice about having an affair with their secretary -just for the record, I've never been a secretary!!!!!- but at least that isn't exhibiting bad manners because presumably nothing personal get discussed 
Yes that's what I said if the relationship went beyond employer/ee and they were lovers but there's no evidence for this.
Page 327 CAL's book:
"Pamela had no memory of the conversation although she was present, and Robert's account was therefore designated heresay, along with a remark Nevill was said to have passed about Jeremy, which Barbara Wilson repeated to Colin: 'I must never turn my back on that young man.' Today, Babara muses: I think that came about because they'd had a row and Jeremy threatened him.
I can easily imagine him after a row, telling Mr Bamber, "You want to watch your back, Dad, when you're out shooting"'.
So BW admits it's all down to her imagination!