Do I just ignore it?

I started dating this guy I met about 2 weeks ago (we went on dates but after the 6th day of knowing him we made it official) and I thought that it was fast but why not? Today, while on the phone, we had a silence and he interrupts it with "babe...i love you". I said "what?" And he said "nothing". That ended the phone call. Now should I just pretend it never happened or should I just finally break it off? (I have suspisions that he has other girls so I kinda dont like how untrusting i am of him)