Appreciate the clarification! I’ve been trying to wrap my head around how the fediverse works but I’m far from a programmer so some parts are still a bit confusing to me.
Is it just a different approach to reach the same goal or is there some inherent limitation to how the fediverse works that prevents instances from using crawlers?
Lemmy a little bit comparable to E-Mail. Both are decentralized systems. I can register a domain ajsldkfjkadsfjkladfsjklfasdjöfda.com and host an email server there. Now, I go to gmail and send a mail to hi@ajsldkfjkadsfjkladfsjklfasdjöfda.com.
Gmail has had no idea that ajsldkfjkadsfjkladfsjklfasdjöfda.com even existed, but gmail and ajsldkfjkadsfjkladfsjklfasdjöfda.com speak the same protocol and can therefore communicate. So, the email reaches me. You see, Gmail did not have to crawl the whole internet to find my new site. It being able to speak the same language (some protocols) is already enough.
Same with lemmy: Different instances (e.g. lemmy.world and feddit.de) don’t need to know which instances exist. If I (from feddit.de) want to post on lemmy.world, it will work, because they are having the same protocols.
What prevents instances from using crawlers?
Nothing. It’s hypothetically possible, but:
It has to be implemented in a good way
In a large network, it would take some time to crawl.
Where will the crawling be done?
On your PC or phone? Then, the crawling will be done for each user. But maybe, this takes too long.
So, let’s do it on the lemmy instances…? What if an instances blocks the IP Address of another instance? Then, the users wouldn’t know about the communities there…
Maybe, there is already some crawling happening. The search of the lemmy app WEFWEF is able to find communities of other instances. Either, WEFWEF shows me posts and stores the community they come from somewhere and seaches them when I use the search… Or it accesses a crawled database or similar to suggest me those communities…
What they are doing is pretty much what Google is doing to find the webpages. It’s called crawling.
Appreciate the clarification! I’ve been trying to wrap my head around how the fediverse works but I’m far from a programmer so some parts are still a bit confusing to me.
Is it just a different approach to reach the same goal or is there some inherent limitation to how the fediverse works that prevents instances from using crawlers?
Lemmy a little bit comparable to E-Mail. Both are decentralized systems. I can register a domain ajsldkfjkadsfjkladfsjklfasdjöfda.com and host an email server there. Now, I go to gmail and send a mail to hi@ajsldkfjkadsfjkladfsjklfasdjöfda.com. Gmail has had no idea that ajsldkfjkadsfjkladfsjklfasdjöfda.com even existed, but gmail and ajsldkfjkadsfjkladfsjklfasdjöfda.com speak the same protocol and can therefore communicate. So, the email reaches me. You see, Gmail did not have to crawl the whole internet to find my new site. It being able to speak the same language (some protocols) is already enough.
Same with lemmy: Different instances (e.g. lemmy.world and feddit.de) don’t need to know which instances exist. If I (from feddit.de) want to post on lemmy.world, it will work, because they are having the same protocols.
Nothing. It’s hypothetically possible, but:
Maybe, there is already some crawling happening. The search of the lemmy app WEFWEF is able to find communities of other instances. Either, WEFWEF shows me posts and stores the community they come from somewhere and seaches them when I use the search… Or it accesses a crawled database or similar to suggest me those communities…
Wow, thank you so much for taking the time to write such a detailed explanation. It really clears up a lot of what I was confused about.
The community across the board on Lemmy has been so refreshing compared to the last few years on Reddit or any of the alternatives I’d tried before.