Hello!
In my opinion, there's nothing you can do about this because you cannot control someone else's actions. The best you can do if you truly care about him is just tell him how you feel. And then he can decide what to do from there. You cannot force him to do something if he doesn't want to. Maybe he is scared, maybe he is confused too. If he has Asperger's syndrome then he may not know how to communicate his feelings to you.
I'm sorry to hear about your past abusive relationship. But you must grow and not let anyone hold you back. We never know why people do the things they do, you just have to explain your feelings and be honest. That is all you can do! They can take it or leave it--it's just like a break-up, or a friendship falling apart. Things like this are hard and upsetting, but you can't wonder and guess what he is thinking. Just ask, and tell him how you feel.
A little tough love but I hope that this helps!