what would you do?

I was at this training seminar thing one time for jobs. they asked this question. it's your first day on the job and a co-worker asks you if you want to go lunch. some people said yes. they said y'all would be fired for doing that because apparently that person person is toxic and a bad employee
I call B.S. on that don't buy it because
1. it's my first day on the job and how would I know what kind of a person they are?
2. they can't control what I do or I hang out with if it's outside of work.