Temporary breakage for all content?

Found an issue with the phpBB system here at NESdev? Use this forum to report problems.

Moderator: Moderators

Post Reply
calima
Posts: 1711
Joined: Tue Oct 06, 2015 10:16 am

Temporary breakage for all content?

Post by calima »

For a minute, all pages returned an "information: the requested page could not be found" page, and the pages that had loaded were failing to load their images hosted here. A bit later things were working again.
Fiskbit
Posts: 803
Joined: Sat Nov 18, 2017 9:15 pm

Re: Temporary breakage for all content?

Post by Fiskbit »

Thanks for the report! I've relayed it to brizzo. We've had a few reports of very slow page loads or timeouts recently lasting a few minutes at a time. So far, there's nothing showing up in the logs that would explain it, unfortunately. We did have an issue with abusive scraping of the site recently, but that's been fixed.

Definitely keep these reports coming, if you see more issues. Time and time zone, as well as whether the problem affects just the forums or also the wiki, are helpful.
calima
Posts: 1711
Joined: Tue Oct 06, 2015 10:16 am

Re: Temporary breakage for all content?

Post by calima »

It was 2-3 minutes before I made the post. GMT+3 Europe.
calima
Posts: 1711
Joined: Tue Oct 06, 2015 10:16 am

Re: Temporary breakage for all content?

Post by calima »

Happened again just now. I think I'm hitting the scraping protection, just by opening a handful of threads with images.
User avatar
Individualised
Posts: 297
Joined: Mon Sep 05, 2022 6:46 am

Re: Temporary breakage for all content?

Post by Individualised »

The site admin may want to look into blocking user agents with the string "Bytespider". ByteDance (the social media company known for TikTok) have recently been crawling the web with a very spammy crawler that seems to have no regard for the rules that web crawlers are usually expected to follow. I've noticed that this site has been slow recently and there's usually a high amount of guest users active when this happens.
Fiskbit
Posts: 803
Joined: Sat Nov 18, 2017 9:15 pm

Re: Temporary breakage for all content?

Post by Fiskbit »

We have ByteDance and ByteSpider blocked now; they have been a nontrivial percentage of our traffic lately.

We've gotten other feedback about the ratelimiting hitting legit users; I suspect brizzo is going to pull back on that soon as we get a better idea of how well the site is working with the changes we've made and bots we've blocked.

Thanks for the feedback and please keep it coming!
Post Reply