Men do not forget such things easily. It has straight connection with a blow to his male ego (that in spite of his presence you thought of somebody. It also gives feelings of inadequacy.) He might have developed paranoia. You should consult psychiatrist for counseling.