I told him I wanted more romance

I am a polyamorous person myself, so I can't really say being swingers is a wicked thing. But I am struggling to see how he might have thought that was a solution to the problem of you feeling that you need more romance. Swinging might add more kink, but I think the way it's usually done doesn't really build deeper and more emotional bonds. And I'm assuming those deeper bonds are what you mean by romance. He either confuses kink with romance or he's been thinking about this for a while and was looking for an excuse to suggest it as the answer to something.