I added a throttle for unauthenticated accesses to /tags on our public Mastodon instance. The rate limit of 50 in 10 minutes triggers quickly, and for the first time this year, our media cache usage has been slightly decreasing since.
A toggle to completely disable unauthenticated access to hashtag search would curb the current wave of scrapers quite effectively.
Replying to @galaxis@mastodon.infra.de
@galaxis Awesome. Can you share what you did?
Replying to @galaxis@mastodon.infra.de
@galaxis i noticed all of this was coming from CN ip space. I need up blocking all of china :/
tencet was the biggest problem, but i was playing whack-a-mole so i just blocked all of china and the problem basically evaporated. there are still some bots hitting from an ip and never showing up again, but those are harder to capture