Freedom of speech?

Why is that when you criticize Christianity, Islam, Hinduism etc (religion, not individuals). It's a matter of freedom of speech but when you question Judaism and Zionism, it becomes a matter of anti-semitism only? Why not freedom of speech applies in that subject?