this post was submitted on 15 May 2024
118 points (100.0% liked)
Technology
37355 readers
251 users here now
Rumors, happenings, and innovations in the technology sphere. If it's technological news or discussion of technology, it probably belongs here.
Subcommunities on Beehaw:
This community's icon was made by Aaron Schneider, under the CC-BY-NC-SA 4.0 license.
founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
Sigh. AI has basically added a rocket booster to the enshittification train.
Hopefully this doesn't impact DDG.
I mentioned this in another thread, but I do worry that google is eventually going infect the APIs that metasearch engines like DDG, Kagi, searchxng, etc depend on.
In my experience, a lot of the sysadmins who run high traffic sites will treat all bots as scrapers that have to be blocked or slowed to a crawl. Then they make special allowances for googlebot, bing/msnbot, and a few others. That means there is a massive uphill climb (beyond the technical one) to making a new search engine from scratch. With Google and MS both betting the farm on LLMs I fear we're going to lose access to two of the most valuable web reverse indexes out there.
I fear that as well. I use Searx-NG at home, so am expecting that to start dying a death of a thousand cuts soon.
Was thinking about standing up (or contributing to) either YaCY or Stract, but you made a good point about the bot allowances for the Googlebot et al crawler UAs. Wonder how frowned upon it would be to spoof the crawler UA in a self-hosted one?
I've just started using Searxng.. you expect it to die soon? Is it because you expect other search engines to follow suit until there are no search engines anymore, only hallucination machines?
Basically, yeah.
I hope there's enough of a market for non-ai content that it doesn't come to that. I think we already reached the pushback stage with image generation.