‮

@ireneista i have done a small amount of research into how this could be done. you can definitely go with https://commoncrawl.org/ to start out with, instead of crawling sites yourself.

i feel like the governance and funding is the bigger issue, though. search engines become shit when they become ad engines. if i were to operate something like this, it would either have to be a money sink hobby thing i give only to friends, or an explicit "pay a subscription for access" type thing.

@ireneista yeah, the search engine that i'd like to build Eventually(tm) would be one that has a really locked down set of allowable domains and paths. honestly something like a web of trust model might be good