Following one's dream, especially when it takes you away from something or someone you've become attached to is difficult. Ask yourself, when did you dream of this journey? Before or after meeting him? How likely is it that you'll be able to go another time, if you delay it now? There are so many questions to ask yourself and no one but you can answer it.
I've made decisions that have taken me away or even agreed with someone I cared about having to leave. It hurt, it was tough, but in the long run I can and have looked back and can say I don't regret any of them. I got to experience things I never would have, if I'd have waited and I know, for those that left me behind, they got to live their dream.
If it's a trip away, maybe he'd wait or at least be open to seeing how things are when you return. If this is permanent, well that's a different story. I wish there was an easy answer, but no matter what you decide, something will always be on your mind of whether it was right or wrong.
My suggestion is, follow your heart, weigh out everything you can think of and do it. Sorry if this doesn't help, but I wish you the very best on this one. I've been there and know it's not easy. 🤗