What should I do.....?

I had casual sex twice with a guy who told me "not to get emotional" over him, I agreed because I really hadn't been interested in a relationship in years. I wouldn't have gotten any feelings had he not began to take the time to get to know me..and ask me personal questions such as what I look for in relationships and told me I was different and cute. He is clearly a charmer and I've tried to stop everything by not talking to him because he's sending mixed messages (relationship..no relationship) but he then proceeds to come back for more. I even told him that I was no longer interested in sex and he stopped talking to me as much..but that lasted about 2 weeks before he texted me out of the blue one day. I'm not sure whether he is interested in me or sex with me....why is he going after me (a shy quiet woman) when there are tons more girls he could easily have because he is very attractive. Do I just stop responding to him all together and ignore advances at work...or keep going on possibly expecting a relationship?