Is he joking or for real?

Okay I'm 19 and I have a crush... thats so strange to say. After a bitter two year relationship and a weird friendship with someone else, I'm still getting those butterflies about someone.
Problem is, I can't tell if he likes me... its not the whole I...

look at him, he looks away sort of thing... let me explain.
He is in a fraternity and while I was going through an intense breakup, I met one of his brothers (Jake) and we hit it off- problem is that we started to be hookup buddies. I wasn't looking for anything serious and neither was he and it was nice to have a friend with benefits. Eventually I asked him if we could stop hooking up because we had gotten so close and it didn't feel right. (When I say hooking up, we just made-out... never went father than that) We talked about how we were only friends, but somehow he eventually confessed that he fell for me and that he needed space away from me and stuff. Basically I haven't talked to him since and I feel badly about how our friendship ended.
The whole time we were just hanging out as friends, I got to be really close with his fraternity and hung out there all the time with my friends. We never acted couply but everyone was under the impression we were dating even though I tried to make it clear we weren't.
Clay and I met at a party for his fraternity where he was tipsy and called me his wife and from then on it came to be a little inside joke between us. Problem is, I really started to like him a lot. He would joke around that he loved me and hug me and call me beautiful, but I'm pretty sure it was all part of our little joke.
Then I hadn't heard from him all summer until last night he left me a voicemail from a loud party up at school and said "Hey! I miss you!!! Come up here right now!"
Then, he called me and he was just talking about how much he missed me and told me he wanted to make love to me (being goofy clay) and saying that I was the most beautiful girl he's ever seen and all this. He asked me if I would go to dinner with him soon (he stayed at school this summer 7 hours away from where I live). He wanted to know when I would be back and all this. I told him sure.
Then today, he sent me a text that said "I hope I wasn't too big of an idiot last night, I saw that I called you. Sorry if I was inappropriate! I was kind of out of my head last night. You should have come!"
I told him it was fine and all that. then he said
" What can I say! I'm crazy about you, wouldn't of married you if I wasn't. Well cme to town sometime and maybe we can have a romantic dinner for two."
I said yeah... you promised you would last night
Then he said "Anything for you beautiful! Looking forward to it"

Okay... is this just our little inside joke or could he actually like me? I don't know whether to play the goofy friendly card or to go for it. Iwouldn't normally post something this sixth-gradeish, but desperate times call for desperate measures
THANKS I really appreciate any help you have to offer!

He does like you and hes sticking with that married and wife thing cause it makes you laugh I bet. you should go to dinner sometime and see how it goes if something was to happen afterwords though it probably wouldnt turn into something you wanted but if you really like him I would say go to dinner and to take it slow and to see what his intentions are.

I would say that he really likes you. He just doesn't want to ruin your friendship by coming out and saying how he feels about you as probably how you are thinking right about now. You both know that you have a good fun relationship right now and don't want to mess it up but if you want to be with him you have to take a chance and at least come out for a little. He really wants to come out and that's why he was saying those things while drunk. Being drunk does not make you say things that you don't mean it just makes them easier to say without thinking about the rebercussions.