Eh don't worry, I get you. Japan is a pretty neat country with great technological advancements, beautiful nature (cherry blossoms) and respectful hardworking people. I hate how the internet has this grudge against people like us but they never care if someone is obsessed with a different country, only Japanese enthusiasts get this treatment. But in general you really just can't talk about Japan without a certain sense of irony involved, and that's just… sad… And yeah, Japan is obviously not perfect either. Work hours are hellish, ESPECIALLY for foreigners! And like you said, they aren't very opened to Gaijins, but it just seems like people who "hate" Japan think they're somehow smarter than us, telling us things that are obvious about Japan as if we didn't already know them.
If you ask me, if you want to move to Japan, go for it! Or at least try! I know that "you only live once" is a pretty cliche line, but it's true! You do only live once, so why not spend that time on somewhere you want to be in? Enjoy your time in Japan, cause you won't be able to experience it when we transfer to LFE.