You are trying to feel your emptiness and marital unhappiness by establishing relationship with him. However, you have to remember that he may not be unhappy with his marriage. He may not be interested in serious relationship with you but may get in to it just for fun (to have another woman), if you will keep pursuing. In that case, on long run you are going to get pain and I think which you have an ample at present. His attitude suggests that he is not interested in supporting you on emotional front. Stay away from him.