Guys have a habit of talking about their affair; including every detail of what they did, with close friends. This is truer when they have already broken off. There are all chances that he may talk about your relationship to his friend so if you do not want to take chance then honestly tell him.