They get downvotes in the niche subs too because everyone browses Lemmy by All due to lack of content.

It’s not gonna stop until Lemmy has enough content that people stop coming across niche NSFW content they aren’t into in their “All” feeds because they’re gonna automatically downvote it instead of blocking. Any uncommon kinks or male NSFW content is gonna be hurt by this. Disabling downvotes should be considered as a permanent setting for this instance.