Posting in the Magento forums has been disabled pending the implementation of a new and improved forum solution which should better serve the community.

For new questions please post at magento.stackexchange.com, the community-run support site for the Magento community. We will be providing updates on the new forum solution soon. For questions or concerns please email community@magento.com.

Magento Forum

Page 2 of 2
Unicode (UTF-8) errors… not fully supported? 
 
Grayson
Member
 
Avatar
Total Posts:  58
Joined:  2007-11-20
Netherlands
 

Alas, this issue is still alive in 0.7.14800 though the files listed here have been fixed in that release. This is major--a deal-breaker--for European users.

If I try to type in Dutch characters such as ë in, say, a product description, the text is truncated at the first such character.

The euro symbol € also does not display correctly in the storefront.

Moreover, special characters displayed in lists like the “allowed countries” list on Admin->System->Configuration->General tab come out strange, such as ë instead of ë. (I’m using the Dutch locale setting, so the list is shown in Dutch, hence the special characters.)

The problem occurs for me in both Firefox and IE.

I’ve also added this comment to the bug report(s) for this issue.

 
Magento Community Magento Community
Magento Community
Magento Community
 
Grayson
Member
 
Avatar
Total Posts:  58
Joined:  2007-11-20
Netherlands
 
Grayson - 10 January 2008 12:40 PM

Moreover, special characters displayed in lists like the “allowed countries” list on Admin->System->Configuration->General tab come out strange, such as ë instead of ë. (I’m using the Dutch locale setting, so the list is shown in Dutch, hence the special characters.)

This part of the problem stems from the nl.xml file in magento/lib/Zend/Locale/Data/; the faulty characters are already there. Not sure how I can fix that.

 
Magento Community Magento Community
Magento Community
Magento Community
 
Grayson
Member
 
Avatar
Total Posts:  58
Joined:  2007-11-20
Netherlands
 
Grayson - 10 January 2008 12:40 PM

The euro symbol € also does not display correctly in the storefront.

This part of the problem appears to stem from the lack of a euro symbol in the utf-8 encoding. <rant_not_directed_at_Magento> Nice. The euro’s been in use for six years now; why doesn’t utf-8 have an encoding for it yet? </rant_not_directed_at_Magento>

 
Magento Community Magento Community
Magento Community
Magento Community
 
Grayson
Member
 
Avatar
Total Posts:  58
Joined:  2007-11-20
Netherlands
 
Grayson - 10 January 2008 12:40 PM

If I try to type in Dutch characters such as ë in, say, a product description, the text is truncated at the first such character.

This appears to be a common MySQL problem that occurs when character sets don’t jive up as they should. I don’t know if it has anything to do with the Magento admin code (when I type in ë, does that get properly interpreted and converted to utf-8? beyond my level of expertise) or if it lies elsewhere, but my database is 100% utf8_general_ci.

ETA: I do see that the tinyMCE files use charset=ISO-8859-1. If the input captured here isn’t converted to utf-8, that could be a source of the problem

ETA #2: After reading up on it, I’m pretty sure it is indeed a problem with the tinyMCE charset encoding. The tinyMCE documentation even says “Please try using entities if possible. Like & aring; etc for non a-z characters.” (from magento/js/tiny_mce/langs/readme.txt). So I tried using & euml; for ë in the product description input box, and voila, it worked just fine. Displays properly as ë in the storefront, too.

While this is workable, it’s hardly ideal for European users. It will be so clunky having to type html entities for letters we’re used to typing using just a keyboard stroke or two. I don’t know how much work is involved in changing the tiny_mce distribution with Magento to accept the “international keyboard” input method (e.g., double quotation mark + e = ë), but I’ll look further tomorrow. It’s way past time for this 6-months-pregnant lady to go to bed wink.

 
Magento Community Magento Community
Magento Community
Magento Community
 
redrum
Jr. Member
 
Total Posts:  1
Joined:  2008-01-11
 

hey!

i had the same problem with a fresh install.

the symptom was that descriptions using swedish characters where cut off at the first such char in the string.

running w3c validation showed that my meta tags told the browser to use utf-8 but the http headers from apache told the
browser to use iso-8859-1(or whatever its called).

solution was to change the apache config, adding this
AddDefaultCharset Off
AddDefaultCharset utf-8
to the virtualhost entry.

now, im not sure that you need the first line, but it works for me.. wink

 
Magento Community Magento Community
Magento Community
Magento Community
 
Grayson
Member
 
Avatar
Total Posts:  58
Joined:  2007-11-20
Netherlands
 

Hi Redrum, thanks for your response! Another person (SeL) also suggested this might be the problem in this thread. He also suggests a “path of least resistance” approach—see responses # 8 / 9 -ish there. You can just use

AddDefaultCharset Off

which, as SeL points out in that thread, is the better option (doesn’t force UTF-8 encoding so if you happen to have pages using a different encoding in your magento/ tree they’ll work fine too).

 
Magento Community Magento Community
Magento Community
Magento Community
Magento Community
Magento Community
Back to top
Page 2 of 2