There are two different answers to this, and it’s all based on context and personal exposure:
Rationally, no, because the thing they’re into exists independently of the fandom as a piece of art, as someone’s creation. Just like how a remake of a film doesn’t “ruin” the original (like so many love to claim) because the original still exists and you can watch that version instead. It’s not like they went out and burned every existing copy of the original when the remake is out - there’s simply more out there, and you can choose whichever version you like most.
Irrationally, yes, because exposure to fandom can sour you on just about anything and make it impossible for you to enjoy it without thinking about how gross and shouty and entitled and over-the-top fandom is. If you go in to something blind, with zero knowledge of how dumb the fans are about it, you can enjoy it completely outside that context and you’ll generally be better off. Once you start looking up the internet’s reaction to it, though, it can sour you. Suddenly that cartoon you enjoy leaves a bad taste in your mouth because all you can think about now is how fandom reacts to it, with shipping/porn/screaming at eachother about it. Totally understandable, but it’s only inside the context of “fandom” that something can be “ruined” and it’s entirely a subjective thing based on personal experience.
You do have to be careful, though, not to contextualize the thing itself only inside your personal experience with fandom. It’s better to just say “yeah, the fandom really ruined that show for me” rather than saying “that show sucks because of the fans” (I’m guilty of the latter myself, unfortunately) and then going on and on about how much the show sucks because you don’t like the fandom surrounding it.
It’s like if during your first time watching Lawrence of Arabia a clown ran out in front of the screen and started beating off. All you’re going to think about when it comes to that film is the masturbating clown, but if someone brings up the film and you say “Ugh, that movie is terrible, that clown wouldn’t stop jerking off!”, that context is only relevant to you. Your feelings are totally valid, but when communicating with others about it, it’s best to use language that suggests you understand this is your personal experience and not a judgment on whatever the thing is itself.
Also I laughed at that Lawrence of Arabia joke more than I probably should have.