There's no robots.txt that I can find, and nothing in the page source that would deter a spider. So why, when I search Google for pithy BBS phrases such as "All three link forests", does nothing come up? Even searching for "Empeg BBS" gives only administrative pages from this site (plus a couple of hits from the old URL). Is this being done deliberately (by us) and, if so, how and/or why? Has Google decided that with all our CGI pages and intra-site links, we're a whacking farm, or whatever the phrase is?

Peter

I don't know if this is the case, but google does not like dynamic pages, and php and asp(.net) do not get indexed, unless you use some url conversion fuction in the forum so that urls become "search engine friendly". I would assume UBBS threads has something like that. That's what I had to do to have my forum indexed (which is asp.net based, however).

Mmm.. yep, nothing there.

At one point in the past year, the BBS Search page even had a Google Search function at the bottom. This also seems to have gone away again, and as a result I cannot find anything here again (the regular "search" and I just don't get along -- it never finds anything for me).

Cheers

It's not just because the BBS has the no spidering tag activated, is it?

Most BBSes and such will activate that tag to prevent spiders from thrashing the database constantly.

In addition to +adding +pluses +on +everything I also find that to find anything useful these days I have to add -minuses -on -crap -stuff.

Google really need to clean the shit out of their database.

I think the usefullness of the search function depends on your choice of search terms, like any engine. The plus signs took a little getting used to, but I prefer it to other engines in some ways. As far as I understand it, the plus makes a word required in a post result, meaning if you have multiple words without plusses, it will give you all posts with either of those words, not both. With that in mind I'm usually able to find anything I'm looking for.

I'm not doing anything specific, not sure why they purged the BBS from their search index. The search index friendly URLs are still on after the upgrade.

I'll look at restoring the google search thing on the search after the 6.5.4 upgrade.

Google does index dynamic content. Not always, though.

http://www.google.com/support/webmasters/bin/answer.py?answer=35287&topic=8522

http://www.google.com/support/webmasters/bin/answer.py?answer=34431

Google shouldn't even be seeing this board as a "dynamic" site anyhow. The trick that UBBThreads did is made it look like the site has a ton of static content across a lot of directories. And they did this in a clever way. If you take a URL like the one to read this thread:
http://empegbbs.com/ubbthreads/showflat.php/Cat/0/Number/283367/an/0/page/0

It makes it look like a folder structure by allowing variables to be passed in via / instead fo the normal & and ?. By default, the board will use that as the URLs for everything, unless it is a true dynamic area like the make a reply section.

Sadly if Google is going to yank the boards out just because I don't touch their sitemap or whatever (I made one once), then honestly it is not worth my time to ensure the boards are listed there. A webmaster shouldn't have to work to keep his search results coming back the same way they once did.

Yeah, I get all that. I was just responding to the dynamic/filetype points.

I don't see why the bbs would disappear, though. That is rather odd.

I wonder if any of the other search engines index this site?

What are the other search engines, anyway??

Heh.. I don't know either, and I work for the place.. I'll poke my friends in the index team when I'm at work on monday.

The only other search engines I can think of are MSN, Yahoo, and Amazon (aol?) A9. All trying to say they're better then Google, when they all seem so similar.

It's odd that we vanished from Google (though I can find a few pages in the results) when we once had such pull. I would post some obscure question, then search the net for answers, only to find my post near the top of the results list.

Altavista seems to still be running, but I don't know how recent their content from this board is. I did a search for empegbbs and got 165 000 hits. I also checked hotbot and they were using msn and google. Excite has turned into a portal, and it seems like they don't have their own search engine any more.

Stig

There were other upstart "Google Killers" that I can't even remember. Something like Atoona, which used the same black/blue/green result text layout as the Goog.

Did Google buy DMOZ yet, not that they're a search engine, but a long-lived independant index.

I wouldn't have anything to do with the recent disappearance of empeg.com would it? Didn't it have pointers to the bbs? Dosen't Google's page rank system still depend on the count of links pointing to a site?

Dunno, Maybe that isn't so important.

The empegbbs.com front page has a page rank of 3, which I think it what it has had for a long time.