Comments on: Fast & Frequent Spider Indexing

By: Scott

Scott — Tue, 08 Sep 2009 00:29:42 +0000

Yes I’m still a member, and use it often, but do keep it to far less than half of the links being built to any domain. http://www.link-vana.com does have the potential to be misused, and some people have reported drop in rankings after they (ignorantly) pound dozens or hundreds of links with the exact same anchor text in a short amount of time.

By: Brian Bonito

Brian Bonito — Mon, 07 Sep 2009 04:26:17 +0000

Scott, I see you commented on Linkvana… I was wondering if you’re still a member and if you had any follow-up thoughts since you last posted about it?

Thanks

By: Scott Hendison

Scott Hendison — Sun, 08 Feb 2009 19:00:26 +0000

Well yes “they” should grab your pages fast and on their own, but since they DON’T, these ways should all help until their technology improves.

By: Tahir Jamil

Tahir Jamil — Sat, 07 Feb 2009 15:15:22 +0000

I have noticed crawling of few search engine spiders in long intervals of time. All your methods are correct and praiseworthy but one need to become SEO expert to let spiders crawl their websites faster. In my view search engines should crawl every page as soon as it is added or updated. Unfortunately few search engines are taking months to better rank my pages. It means some search engines are using better technology while others need to upgrade their system.

By: Scott

Scott — Wed, 17 Dec 2008 14:02:41 +0000

Thank you Jill for your comment, and I appreciate the time you took to write. You’re certainly right, and repeating information doesn’t make it true.

I’m not quite positive whether you disagree with the fact that frequent cache dates are good, or you disagree with the fact that these are ways that will help freshen pages.
I’m assuming that you’re talking about my reference to the speaker statement at SMX being “spot on”, as the fact you’d like me to give subsequent proof of? Since I didn’t make any bold claims other than the fact that these were all ways to get spidered more frequently, so I’ll assume that’s correct.

When I wrote the sentences “Are their cache dates newer than yours? I’ll bet they are in many cases.” I did five quick searches first, and all five showed me newer cache dates near the top of the rankings, but there WERE exceptions in every single search, just like there would be exceptions in the PageRank.

Besides Aaron writing this in 2007, there are numerous other references around the web to people saying that cache dates indicate what Google thinks of your sites importance.

Here are couple of tests & references I found while looking just now…

This one definititely DOES seem to indicate support for the theory, and shows a chart that reports the top 20 Internet marketing sites based on Google crawl rate, and claims that “Not surprisingly, the overall popularity of the site is negatively correlated with the crawl rate”

Donna Fontenot did a guest post here, making what I think is a good case for the fact that cache dates do equate with site importance, and she ends like this –

In summary, the more often your page is crawled and cached by Google’s spiders, the more important that page is in Google’s eyes. The more important the page is, the more likely it is to show up in Google’s search results for related queries. If the page isn’t being crawled and cached very often, you should focus more efforts on keeping the page updated with useful content, and spend more time promoting the page.

This one does NOT “prove” the theory – but the author claims it doesn’t DISprove it either

However, your comment did pique my curiosity, so I’ll do some research of my own, and I’ll try to cite others more often whenever I write.
Again, thanks for taking the time to comment

By: Jill Whalen

Jill Whalen — Tue, 16 Dec 2008 14:18:55 +0000

While frequent respidering and having newer cache dates have been part of SEO lore for ages now as something that can increase rankings, I’d sure like to see your research and subsequent proof of this fact.

IMO, it’s just another SEO myth from what I’ve seen. How about another article showing your data, how it was accumulated and the results?

By: Jason Capshaw

Jason Capshaw — Sat, 13 Dec 2008 09:58:14 +0000

Scott, nice post. I know you linked to it, so I assume you have tried the 30 minute backlinks. Does it really only take 30 minutes to build and submit software?