Page 1 of 2 12 LastLast
Results 1 to 10 of 13

Thread: googlebot

  1. #1
    Join Date
    Apr 2012
    Location
    Central New Jersey
    Posts
    130
    Thanks
    49
    Thanked 0 Times in 0 Posts

    Question googlebot

    I have five active domains that get decent traffic. Until recently, a frequent visitor was the googlebot spider, frequent meaning maybe 4 to 5 percent of traffic. Of late googlebot does not seem to have stopped by at all. Traffic as a whole does not seem to have fallen off--at least not yet. I am wondering whether I should be concerned over googlebot's absence.

    A.

  2. #2
    Join Date
    Mar 2005
    Location
    SE PA USA
    Posts
    28,926
    Thanks
    43
    Thanked 3,192 Times in 3,155 Posts
    Blog Entries
    12

    Default

    Probably not. Although I don't know for sure, I would think if you're updating your content a lot the bot has to visit more frequently. Once you've settled on things and the site is indexed, the bot doesn't have to visit as often. This would only have an impact on your human visitors if one of the main reasons they're coming is for frequent updates.
    Last edited by jscheuer1; 06-04-2013 at 02:24 AM. Reason: Usage
    - John
    ________________________

    Show Additional Thanks: International Rescue Committee - Donate or: The Ocean Conservancy - Donate or: PayPal - Donate

  3. #3
    Join Date
    Apr 2012
    Location
    Central New Jersey
    Posts
    130
    Thanks
    49
    Thanked 0 Times in 0 Posts

    Default

    Well the bot doesn't know whether it's being updated unless it stops by! I made significant changes to some pages on Saturday that google hasn't picked up on yet. On previous occasions, google reflected these changes within a day. All visits have stopped entirely, on all my domains.

    A.

  4. #4
    Join Date
    Mar 2006
    Location
    Illinois, USA
    Posts
    12,162
    Thanks
    263
    Thanked 690 Times in 678 Posts

    Default

    It might have noticed a change in the frequency of updates, so that it now visits less regularly (because it predicts fewer updates). But that wouldn't explain why it never visits.

    It's hard to say what's going on. I'd guess either:
    1. There's a new IP or other reason you're not recognizing it (but it's still there).
    or
    2. For some reason Google has flagged and rejected your website. I have no idea why that would be, but it's the only thing I can imagine if you get no visits.

    Have you tried logging into your account with google to see what's going on? Webmaster tools, and so forth.
    Daniel - Freelance Web Design | <?php?> | <html>| espa˝ol | Deutsch | italiano | portuguŕs | catalÓ | un peu de franšais | some knowledge of several other languages: I can sometimes help translate here on DD | Linguistics Forum

  5. #5
    Join Date
    Apr 2012
    Location
    Central New Jersey
    Posts
    130
    Thanks
    49
    Thanked 0 Times in 0 Posts

    Default

    The new IP suggestion appears to be not it: I do searches using keywords on the changed page that were not there previously, and they're not reflected in search results. At the same time (saints be praised), searches using preexisting keywords do yield the same good placement that I've long enjoyed, so at least it seems I'm not being blacklisted.

    Concerning google's webmaster tools, no, I've not considered that. I was hoping this problem would resolve without need for deep analysis. My original question was, should I be concerned? So long as my previous placement continues, I can continue for a rather long time. But it's not something that I can let go forever.

    A.

  6. #6
    Join Date
    Mar 2005
    Location
    SE PA USA
    Posts
    28,926
    Thanks
    43
    Thanked 3,192 Times in 3,155 Posts
    Blog Entries
    12

    Default

    When was the last ime you saw the bot?
    - John
    ________________________

    Show Additional Thanks: International Rescue Committee - Donate or: The Ocean Conservancy - Donate or: PayPal - Donate

  7. #7
    Join Date
    Apr 2012
    Location
    Central New Jersey
    Posts
    130
    Thanks
    49
    Thanked 0 Times in 0 Posts

    Default

    I last saw it maybe six weeks ago. That does not mean it's been MIA all the while--I hadn't been checking it.

    My monitoring service allows me details of only the most recent 100 visitors. One of the sites averages about 100 visits daily. Another, about 25. Since I resumed looking, last Saturday, I've not seen the bot.

    I resumed looking last Saturday because I'd made considerable changes to one of the sites, and wanted to start monitoring their effect. Not surprisingly (in view of the absent bot), the Google search engine is still unaware of those changes. A different search engine, DuckDuckGo.com, has picked up my changes. I do not know what DuckDuckGo bot's IP is, so can't do any frequency comparisons.

    A.

  8. #8
    Join Date
    Jul 2008
    Location
    Derbyshire, UK
    Posts
    1,935
    Thanks
    16
    Thanked 303 Times in 302 Posts
    Blog Entries
    10

    Default

    If you sign up for Google Webmaster Tools (its free) and submit an XML/txt sitemap, you can check when your site was last spidered via their dashboard.



    Mine are usually crawled automatically every 7-10 days or so, but you can push URLs through manually to speed-up the process, via their Fetch as Google feature: http://support.google.com/webmasters...answer=1352276

    Google Webmaster Tools and Google Analytics are 2 excellent, feature rich tools that every website owner can learn a lot from. Definitely good things to have in your SEO/Marketing arsenal.

    EDIT: conveniently, this just popped up on my Twitter feed (tips to master Google Analytics): http://www.hongkiat.com/blog/google-...ics-data-tips/
    Last edited by Beverleyh; 06-04-2013 at 05:15 PM. Reason: Analytics tips article link added
    Focus on Function Web Design | Latest News RSS | Facebook | Twitter |
    Fast Edit (A flat file, PHP web page editor & CMS. Small, FREE, no database!) | Fast Edit BE (Snippet Manager) (Web content editor for multiple editable regions!) | Fast Apps |
    The only limit to creativity is imagination: JemCon.org

  9. #9
    Join Date
    Jul 2008
    Location
    Derbyshire, UK
    Posts
    1,935
    Thanks
    16
    Thanked 303 Times in 302 Posts
    Blog Entries
    10

    Default

    I dont know if you already do, but if you just want to check which pages of your site Google has indexed, you could always do a specific "site:" prefix search. To do that, just go to Google and type "site:http://www.mywebsite.com" (with the url of the website to check) into the search bar. The results should just list all of the pages from your website that Google has on record - if a page isn't there, Google hasn't indexed it.
    Focus on Function Web Design | Latest News RSS | Facebook | Twitter |
    Fast Edit (A flat file, PHP web page editor & CMS. Small, FREE, no database!) | Fast Edit BE (Snippet Manager) (Web content editor for multiple editable regions!) | Fast Apps |
    The only limit to creativity is imagination: JemCon.org

  10. The Following User Says Thank You to Beverleyh For This Useful Post:

    marain (06-04-2013)

  11. #10
    Join Date
    Apr 2012
    Location
    Central New Jersey
    Posts
    130
    Thanks
    49
    Thanked 0 Times in 0 Posts

    Default

    OK. You've given me new toys to play with. I took the time to dabble with them and am highly impressed. But I must now put down my toys and return to (what for me is) the real world. I'll be back.

    Thanks! To the lot of you!

    A.

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •