Robot Attack, Site Sluggish
Home » Forums » Features » Unblock spiders?

Unblock spiders?

permalink   Sun, Apr 23, 2006 @ 9:46 AM
Unblock spiders from reading the media/files pages. It would be very useful for searching down files.

Is there some reason to not allow the spiders to index those pages?


P.S. Google, Yahoo! and other search engines lead me to believe most of the site is spider blocked due to the absolute minimal ccMixter accessible content done on those web indexes. It’s notable that a vast increase in indexable material will significantly improve ccMixter’s search rankings.
permalink   Mon, Apr 24, 2006 @ 1:56 PM
ccMixter rankings are pretty freakin high for everything but the forums, which, as I’ve said is a low priority because long term plans are to replace the forums completely.