He is clear in his mind for feelings that he has for you, but probably you are unable to accept the fact that he is just physically attracted and no emotional involvement is there. If you are comfortable with this equation then you may continue or else discuss your emotional need openly with him.