I’m aware this is probably because of SEO bs, but I can’t google posts on the Fediverse. You can easily find posts/accounts on twitter or Reddit using Google, but not Mastodon, Kbin, Lemmy, etc. The only way to find ANYTHING about them is by going directly through the sites themselves, and typing in the entire domain like it’s 1999 and you’re on Netscape Navigator after finding them through word of mouth. The first search result for “Kbin” isn’t even the site itself, its a freaking GitHub page.
I tried using other search engines like Marginalia.nu and MetaGer, but no results show up for either of them. None of the in-site search engines are very good at finding anything other than keywords, so people will still probably be stuck using “site:reddit.com” until its possible to do so.
I really like this place, so I really hope there will be a way to fix this, but im worried that it won’t last if people still have to rely on Reddit to get results.
The
robots.txt
file on this instance (https://kbin.social/robots.txt) disallows crawling.By contrast, it appears that lemmy.ml and lemmy.world allow crawling (https://lemmy.ml/robots.txt).
I wonder if crawling is disallowed since a week ago this site was basically still a prototype?
I see, that makes sense.