Magento - Google Indexing Page that Doesnt Exist. Why - Search Engine Optimization - eCommerce Software for Growth

Skip to site navigation »
Skip to main content »
Skip to footer content »

Login

Login:

Password:

Remember password
Show my name in the online users list

Forgot password?
Register
Job Board

Magento Forum

From setting up your store to managing your products, pages and promotions to generating detailed reports, the Magento User Guide empowers the user to utilize the platform for all of its vast capacity.
Available in eBook and Print formats – Download Now!!!

Google Indexing Page that Doesnt Exist. Why

brendan. Total Posts: 726 Joined: 2008-07-16 London, United Kingdom Ignore user	Hi All, ive just recently pushed a site live and have been checking out some the pages google has indexed to date. This url : googleindex Goes to a page that that doesnt have any records. For example the product doesnt have these attributes. Hence nothing is returned! Im just curious why the bots are crawling my page like this Also when i look at my Customers online i can see bots indexing pages by the product attributes. This seems weird as ive posted up a sitemaps with URL. Ive attached a screen capture of the page. anyone suggest what im doing wrong here? cheers Image Attachments Click thumbnail to see full-size image Signature magento 1.3.1.2 PHP Version 5.2.6 http://www.getstuff.com.au
	Posted: October 14 2008 \| top

golles Total Posts: 237 Joined: 2008-01-15 Ignore user	brendan. - 13 October 2008 05:09 PM Hi All, ive just recently pushed a site live and have been checking out some the pages google has indexed to date. This url : googleindex Goes to a page that that doesnt have any records. For example the product doesnt have these attributes. Hence nothing is returned! Im just curious why the bots are crawling my page like this Also when i look at my Customers online i can see bots indexing pages by the product attributes. This seems weird as ive posted up a sitemaps with URL. Ive attached a screen capture of the page. anyone suggest what im doing wrong here? cheers Google will crawl and index any pages that have links to it - regardless of the sitemap - we have extensively tested this. To stop google crawling and indexing those pages you need to apply restrictions via nofollow on the links that are leading to those pages and/or apply restrictions in the robots.txt file Google is going to those ‘attribute pages’ via links in the layered navigation, compare products, sort by, etc - you need to control what pages you want crawled and indexed and those you do not. Some people MAY want these attribute pages indexed - that is up to them - we do not so we have taken steps to prevent it. The robots.txt and nofollow changes we have made have been documented elsewhere in this SEO forum but if you need help let me know. With all SEO you get debate about what you should and should not do and what google will and won’t do - we tend not to listen to MOST opinion we test and then act upon the results of our tests - we are comfortable with what we do and know 100% it makes a difference to our results. But everyone is different so you also need to be comfortable and understand the changes you are making. Hope this helps.
	Posted: October 17 2008 \| top \| # 1

brendan. Total Posts: 726 Joined: 2008-07-16 London, United Kingdom Ignore user	hey golles, thanks for your reply, its helped me understand whats happening here.. FYI I have a robots.txt file already, but ill probably have to tune this to remove any pages that i dont think google need to index. `User-agent: * Disallow: /index.php/ Disallow: /checkout/ Disallow: /app/ Disallow: /downloader/ Disallow: /js/ Disallow: /lib/ Disallow: /media/ Disallow: /pkginfo/ Disallow: /report/ Disallow: /skin/ Disallow: /var/ Disallow: /catalog/product_compare/ Disallow: /customer/ Disallow: /wishlist/` cheers brendan Signature magento 1.3.1.2 PHP Version 5.2.6 http://www.getstuff.com.au
	Posted: October 17 2008 \| top \| # 2

Magento Community

Magento Community

Back to top

Product review URLs not SEOd - why???

© Copyright 2010 Varien. Magento, eCommerce software, is a trademark of Irubin Consulting Inc. DBA Varien
Privacy Policy|Terms of Service Magento Community Count
177727 users|1138 users currently online|277233 forum posts