Thank you all for your feedback on this matter. The Magento team reviews every thread on this forum. We look into these issues as well as others in upcoming releases.
OK how about this. Will this ever be fixed AT ALL?
If you cant give any around-about time-table, can someone from the team at least let us know if this issue will be addressed at all? There have been numerous forum posts, and bugs in the bug tracker (that still have “New” status)..
I know sometimes things are the way they are because of a reason..And I have yet to read anywhere from the developers that this is an issue that needs to be, or is getting addressed. I just hope this isnt the Achilles Heal of Magento and needs to stay as it is..
I currently have a web site that is outdated and can not keep up with my business. I need to start moving in some kind of a direction. Id love for it to be Magento, but if I start migrating to Magento and this issue never gets addressed I will lose everything I worked so hard for in the SERPS..
I need someone from the development team to acknowledge that the URL issue is in fact an issue and will be addressed..
Good finds, I just started noticing the duplicate url’s in 1.0. The “Use categories path for product URLs” worked like a charm in 0.8xx if I remember correctly, I hope this gets fixed soon!
I just tried it and I think it works. Have you remembered to refresh the Refresh Catalog Rewrites?
Bit of a late reply, but I think I was confused in the post you quoted. I’m still worried about the multiple urls to products, though the “categories path for product urls” toggle does indeed work.
The productid should be added to the product URL. Right now if you have 2 products with identical names then you have a conflict… but with adding the productid you have a unique id. In addition to this it is easier to handle 301 redirects using regular expression matching.
Not really. If there is a product that uses the exact same and URL you can append a number at the end like “my-product-2.html” and it becomes unique (this should be done automatically if there is a conflict).
There is no need to use product id in the URL.
Having one URL per page however is really really important. Or the spider wont know which one to rank and distributing link juice between several urls with same content is not going to help much in the SERPs.
I’ve mentioned the problem with using categories in URL before. If you have a product in multiple categories they would become duplicate pages aswell. So either you have one primary category being in part of the URL even if the product is displayed in a secondary category, or have no category in the URL.
All SEO’s know that using <title> and <h1-h6> is a good way of informing your search engines what a single page is about. It helps the spiders know exactly which text phrases this unique URL is about. Combined with other text elements on the page, the spiders will get a good picture about what a single page is about. And each single page should be unique. Having thousands of URLs with markup telling search engines the exact same thing is a bad thing, and I can show you how Magento does that.
First of all - layered navigation is a good feature, for any webshop. Being able to set different criteria which users can filter their products is great. And in addition, being able to set and create any attribute as an attribute which will be used in layered navigation is also very cool - and highly dynamic. My problem is only illustrated when you are using “Yes” on “Is Anchor” for your category. If you set that option to “No”, my about-to-be-mentioned issue will not be valid anymore, but in addition, there will be no layered navigation (no filter options on the left side).
Lets move on to my scenario. Imagine I have a webshop with clothing and accessories. I have “Clothing” as a category under the “Default category”. As subcategories of “Clothing”, I have categories like “Dresses”, “Jackets”, “Underwear”, “Raincoats”, “Sweaters” and so on. For the “Clothing” category, the “Is Anchor” attribute is set to Yes. I have the following attributes which are enabled in layered navigation: Price, Color, Size and Designer (which is renamed from Manufacturer).
Now lets say I am viewing the “Clothing” category. I get all the subcategories, as well as price, color, size and designer options as part of “Layered Navigation” on the left side. I get lots of options to filter my search, which is great! On the “Clothing” category, both the <title> and the <h2> of the page is: Clothing - which is fine.
The problem however, occurs, when I choose any of the options in the layered navigation. Lets use subcategory as an example. If I click the “Jackets” subcategory, I will be sent to a different URL. Now, one would expect the <title> and <h2> of the new URL to reflect the choice the user made - but no: <title> and <h2> is still: Clothing. There is no information for the spiders to understand that this single URL is actually about “Jackets”, not only “Clothing”. On the top of the layered navigation, on the left side, it displays a “Currently shopping by: Category: Jackets”. That is fine, and some users MIGHT notice it, but for the spiders, it is useless. It is just a small box of text that has changed some words. The real HTML elements available to tell spiders what this is about (<h2>,<title>) - is not used / untouched.
When you are in a shop, which displays a title, containing “where you are”, you would expect it to change when you go from category “Clothing” to category “Jackets”. It doesn’t. The page is EXACTLY the same (markup-wise), except for the actual list of products, which reflect the choices that has been made (search engines will never understand that, it will just se a shorter list of products).
Considering that I have quite a few attributes enabled for layered navigation (Price, Color, Size, Designer, Category), there will be thousands if not tens of thousands different combinations of these filters. With Magento 1.0, ALL OF THESE URLS WILL CONTAIN THE EXACT SAME <h2> and <title>. Search engines will NOT be able to tell one URL a part from another. And when some people search for eg. “Black jackets”, no search engine will display my shop, because my page for “Black jackets” will contain a <h2> and <title> with “Clothing”. I have a unique URL for “Black jackets” which are perfect for what the user wants, but since the markup only contains “Clothing”, there is no chance we will rank for this phrase.
Unfortunately - it gets worse. What could be even more important than ranking on category - is ranking on designer/manufacturer/brand. Most shops are selling products from different brands or manufacturers. For them selling clothing, designer or brand is common. For selling electronics, manufacturer is common. It seems obvious that shop owners would want to rank for such terms. “Sony TV” or “Adidas Sweaters” for example. With the current state of layered navigation, this will never happen. As with all other attributes enabled in layered navigation, manufacturer gets treated the same way. In my shop we have “Designer” attribute in the layered navigation, with our different designers/brand. One of the brands are named “Cream”. We obviously want to rank for “Cream clothing”, “Cream jackets”, “Cream dresses”, and so on. However, if I choose brand “Cream” from the layered navigation, I do get products from Cream, but the <h2> and <title> is still; you guessed it; Clothing! Still - no way for the search engine to understand that this page is about products from the designer Cream! This, in my mind, is absolutely crucial.
There is an additional consequence to this issue. Lets say you want to rank for just the term “Clothing”. Should be pretty easy with Magento, right? Considering you have thousands of URLs with <title> and <h2> “Clothing”. The downside of this is instead of having a SINGLE url for “Clothing”, which spiders can focus all their link juice on - ours is spread across all thousands of pages. So instead of maybe being displayed on page 1 when searching for “Clothing”, we might be on page 10 instead.
This posts continues below, It was more than 3000 chars.....
I haven’t mentioned another big problem with layered navigation - the actual URLs generated. If you have “Is Anchor” disabled for your category, the subcategories listed on the left side has clean and pretty URLs (this is when layered navigation basically is deactivated for the category). Now, if I enable “Is Anchor” for the category, all of the URLs change. When disabled, the link to the “Jackets” category where myshop.com/clothing/jackets. When I enable it, the SAME request (clicking “Jackets” in the left side) will take me to myshop.com/catalog/category/view/cat/13/id/3/ instead. Another thing worth mentioning is that when you disable “Is Anchor” for the category, and you ONLY get a list of subcategories on the left side, the <h2> and <title> issues disappear. When I choose “Jackets” from the left menu, the <h2> and <title> changes to “Jackets”. Now isn’t that amazing! Too bad it gets thrown away once you actually turn on “Is Anchor” and thereby enable layered navigation.
And I’m sorry to say, there are more. Lets say I am in myshop.com/clothing, with Is Anchor enabled, so I get lots of options on the left side. I click on “jackets”, once again. The site takes me to myshop.com/catalog/category/view/cat/13/id/3/ - Now in at the top of the layered navigation, I see “Currently shopping by: Category: Jackets”. Next to the text is a X-icon, which symbolizes “disable this filter”. In other words, I will get back to the “Clothing” category which should be located at mysite.com/clothing. However, the actual link, is NOT what you would expect. When you click that link, you end up at this URL:
http://myshop.com/catalog/category/view/id/3/
when you would assume you would get back to:
http://myshop.com/clothing
This issue happens basically everywhere - Your original page is a pretty URL, but if you activate some filter or option, and then disable it again, you are NOT back at the original pretty URL, you are now at some ugly URL. There are tons of examples of this issue, and not only in layered navigation. Lets say I’m in the “Clothing” category, still at mysite.com/clothing. I click on the “Grid” view to see products lined up side by side, instead of having them in a list. When seeing them side by side, I regret it, and click back on the “List” view. Now, for some reason, I am sent to mysite.com/catalog/category/view/mode/list/id/3/ - instead of mysite.com/clothing, which was the original URL. So if you are on a page with a pretty URL, and you choose a filter, then disable it, you end up at an ugly URL. This is completely useless - search engines will have several URLs with the EXACT same content and purpose, both with pretty URLs and without pretty URLs. This is a big problem, but it is a SMALL problem compared to the main one made in this post.
Hopefully this will be addressed as I think it should be one of the main SEO concerns when it comes to Magento. The product URLs are of course equally important - having multiple URLs for the same product, just to DISPLAY A DIFFERENT BREADCRUMB is a complete waste of URLs. This could easily be fixed by saving the previous category in SESSION and displaying it from there instead of the URL loaded!
I hope the Magento community will take these issues seriously - as they are highly critical, to say the least! Magento is a lot better than all alternatives I’ve seen, when it comes to SEO, but if these matters could be fixed, it would gain tremendous value as regards to search engines. And it’s not just for the search engines - I believe the main issue made in this post also is harmful from a USER FRIENDLYLESS/ACCESSIBILITY standpoint. A person with a screen reader will have no idea what the current URL is about, unless he lets the screen reader read the COMPLETE page, so he understands it when it reads “Currently shopping by: Category: Jackets”. If the <title> or <h2> would contain the same, it would be much easier.
One solution could be to be able to decide which filters should be “reflected” in the <h2>/<title> portion of the page. So one can decide that for example Category and Designer/Manufacturer should affect the <title>/<h2> when chosen in the filtered navigation. And this of course should be valid for URLs as well. I want pretty and clean urls for categories and designers, but understand that having a cute URL for the filter “Between $500 and $1000” is not as easy.
Some comments from other SEO people on this issue would also be fine - but I assume most of them would agree, as this is just as much about HTML standards and semantic markup, as it is “SEO”.
One last thing. All of the issues I’ve mentioned can be seen on this demo shop: http://ladiesroom.etroll.no/klaer - That URL sends you to the “Clothing” page (just that it’s called “Klær” in Norwegian). The “Jakker” category is the same as “Jackets” in my examples.
What elements are in header tags (H1, etc), whether the logo is an H1 or not, etc., are all controlled by the template. The demonstration template that comes with Magento may not be the best for SEO, but that is something that’s easily customizable.
Similarly, the title tag in categories is customizable in the back end, plus you can edit the code to control how it’s generated if you want it different.
That said, I don’t disagree with most of these issues, just that you don’t have to wait on Varien nor do you have to rely on them.
These issues will kill an established SEO’d site with good rankings.
I’ve looked into ways to block pages in robots.txt with the wildcard options but there’s too many to make it viable.
What we need is an admin page that lists all the urls that can be changed for each product and category and that the magento core will use this strictly.
As the magento store has an SEO feature built in then someone must have noticed the issues.
The store functionality is great but commercially with Search Engines will kill a sites ranking.
I’m going to look into this and will post back. I’m sure there’s a simple code change in a few places that will fix this.
Can people list all the issues that they see and add to the next post by copying and pasting previous suggestions then add your own to a new post.
1) my cart, right column, products need to include “url/category/product name.html” and not just “url/product name.html”
2) recently viewed right column, products need to include “url/category/product name.html” and not just “url/product name.html”
3) you may also be interested in, below product description needs to be “url/category/product name.html” and not “url/catalog/product/view/id/93/category/18/”
4) the shoes category shows products urls with “url/catalog/product/view/id/93/category/18/” and not “url/category/product name.html” not sure what thats all about.
1) my cart, right column, products need to include “url/category/product name.html” and not just “url/product name.html”
2) recently viewed right column, products need to include “url/category/product name.html” and not just “url/product name.html”
...
Regarding 1 & 2 - what do you think should be there if given product belongs to multiple categories? Which category should be used for the URL? I am not sure if remembering from which category a product was added to cart was a good idea - think about adding from a search page or adding one product A from category 1 then adding the same product A from category 2. The product will be just shown in cart as having qty 2 but what category to use then?
Recently I moved back to the default horizontal navigation which means I need the layered navigation. I have also made individual links to every product at the base of my home page.
Regarding the duplicate content problem, WOULD THE FOLLOWING HELP:
1. In my sitemap.xml (of course this is the first file i created) should I make the <changefreq>never</changefreq> and <priority>0</priority> for all categories?
2. Is there any way to disallow robots into the categories pages using the robots.txt file? (not sure how to add this to the robots.txt file created by seoguy http://www.magentocommerce.com/boards/viewthread/6002/ )
My aim is for the search engines to bypass the catalog pages, thereby bypassing the layered navigation and other issues discussed throughout these threads…
