He seems to be exploiting you on your sexual desire. This is how he might be pressurizing you to change the way he wants. If you think that you need to improve your behaviour then work in that direction. At the same time stop demanding or craving for his love or sex. Let him become impatient about why you have stopped craving for sexual relationship. He might be involved somewhere is also a possibility.