This forum is invisible to Google

Found an issue with the phpBB system here at NESdev? Use this forum to report problems.

Moderator: Moderators

Post Reply
User avatar
Dwedit
Posts: 4924
Joined: Fri Nov 19, 2004 7:35 pm
Contact:

This forum is invisible to Google

Post by Dwedit »

It appears that you can't get any Google results at all even if you do a site search on the forum.

But you can find results from "archive.nes.science".

Seems that DuckDuckGo does have results.
Last edited by Dwedit on Fri Mar 24, 2023 3:41 pm, edited 1 time in total.
Here come the fortune cookies! Here come the fortune cookies! They're wearing paper hats!
lidnariq
Posts: 11432
Joined: Sun Apr 13, 2008 11:12 am

Re: This forum is invisible to Google

Post by lidnariq »

The only thing we could do about that is move the TLD back to nesdev.com. Google prefers the oldest apparent-mirror of something to prevent pagerank cheating.


... also, I can't exactly reproduce? if I search google for "site:forums.nesdev.org gtmp3" that works for me
Fiskbit
Posts: 891
Joined: Sat Nov 18, 2017 9:15 pm

Re: This forum is invisible to Google

Post by Fiskbit »

I can't reproduce this, either. I also use Google to search for stuff on the forums pretty frequently and the only difficulty I've encountered is that Google prefers the mirrors because they're older than the .org domain. Otherwise, it works.
tepples
Posts: 22708
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Re: This forum is invisible to Google

Post by tepples »

I could rebuild the XHTML files for archive.nes.science with canonical links so that Googlebot understands the intent of the duplicate pages. They would take this form:

Code: Select all

<link rel="canonical" href="https://forums.nesdev.org/viewtopic.php?t=99999"/>
EDIT: I've written the script to add the canonical link. The 7z file is 40 MiB.

EDIT: I got in touch with cppchriscpp, who runs archive.nes.science, and am sending the update.

EDIT: cppchriscpp received the update and has applied it. Let's see if rel="canonical" helps redirect link reputation where it belongs.
User avatar
Dwedit
Posts: 4924
Joined: Fri Nov 19, 2004 7:35 pm
Contact:

Re: This forum is invisible to Google

Post by Dwedit »

I was looking for some specific thread I created earlier, using the search 'site:forums.nesdev.org legend of kage input lag tester'. Duckduckgo, Bing, and Yandex will find the topic, Google does not.

Maybe Google somehow hasn't crawled it while all the other search engines have?
Here come the fortune cookies! Here come the fortune cookies! They're wearing paper hats!
tepples
Posts: 22708
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Re: This forum is invisible to Google

Post by tepples »

There are three possibilities:
  1. Googlebot hasn't yet had a chance to observe the addition of <link rel="canonical"> to archive.nes.science three and a half days ago.
  2. There was some error in the pages served with <link rel="canonical">.
  3. Google Search is imposing some other penalty.
We'd need someone with access to the domain's Google Search Console to track this down.

In my experience, DuckDuckGo's results resemble Bing's, making me think DDG syndicates Bing results. So it'd also be a good idea to keep an eye on Bing Webmaster Tools.
Bavi_H
Posts: 193
Joined: Sun Mar 03, 2013 1:52 am
Location: Texas, USA
Contact:

Re: This forum is invisible to Google

Post by Bavi_H »

tepples wrote: Tue Mar 28, 2023 2:10 pmIn my experience, DuckDuckGo's results resemble Bing's, making me think DDG syndicates Bing results.
(Yep, DuckDuckGo uses Bing and other sources: DuckDuckGo Help Pages - Results Sources )
Post Reply