Boy problems; please help?

I have a boyfriend named Dylan. I liked him for a while then he told me he asked out a lot of other girls and got rejected by all five of them and I was the only one who said yes. I was worried by this that he only liked me out of desperation. But after a week or so he kept saying I wanna see you, I wanna see you. I tried to break up with him but he doesn't take it. He wants to kiss me, and I don't want to because I'm not sure if I like him that much. Is it okay that I walk away and start crushing again, or should I wait till Dylan understands I don't want to get serious?