« Lollygag, the Ajax Framework for .NET | Main | Qooxdoo 0.6.2 Released »

Prototype Encoding Issue

Another encoding issue for AJAX libraries after fixing something similar last year with AjaxAC and middle eastern encoding, yesterday I noticed that prototype have similar issue. The problem with prototype is the usage UTF-8 as default encoding even if your page have a different encoding, this will make the data exchanged between client and server incorrect, and in our cases the SQL executed in server side return no results.

I grabbed today the latest prototype 1.5.0_rc1 included with script.aculo.us, but be sure to get the latest one from their website because the script have changed and version remain the same. In the latest version you can notice a new parameters which have been added to the options "encoding" :

In this case be sure to set the appropriate encoding that your page is using for example windows-1252, windows-1256 ... etc. It should solve the problem if ... encodeURIComponent() wasn' t used for escaping special characters.

And that's the second issue for Prototype which is the usage of encodeURIComponent() which replace each instance of certain characters by one, two, or three escape sequences representing the UTF-8 encoding of the character. The problem here that all escaping functions replace non-ASCII characters with Unicode ones, and in this case even if you set the unicode by default to windows-1252 for example, a special character will be replaced by its equivalent in unicode.

The solution

There is two possible solutions, first find a replacement for the escaping method, and the second solution which is more easy to implement is using PHP iconv extension :

So even if Prototype send data in Unicode you can get it again in its original encoding and continue your process. Will face this bug anybody who work with middle eastern or asian encoding, even french, german, dutch and any encoding which using non-ASCII character. If you use already UTF-8 in your application and database, you should be safe, and won't face this issue. Let me know if someone have found similar issue for other AJAX libraries.

Bookmark this article at these sites
Comments
1

Excellent post. This is not an issue of Prototype library - the encoding attribute of Ajax.Request has nothing to do with the way encodeURIComponent() works, which may be surprising to some.

The server-side iconv solution seems indeed the simplest.

2

When you use iconv don't forget to add headers
'Content-type: text/html; charset=windows-1256'
to your controller!

3

it saved my life :)
congratulations
very good.


4

Thank you very much! Your advice and iconv did the job.

5

excelent post! thanks!


header('Content-type: text/html; charset=iso-8859-1');
//+
$texto = iconv("UTF-8", "iso-8859-1", $_POST['texto']);


that saved my life

6

Thank you for your advice.

I tried many times to solve this problem, but it's not working. Still print "garbage data".

I used windows-1256, and I used this :

thank you

7

I am using classic ASP and my pages and db are in windows-1251 encoding. Now I am trying to use AJAX and facing this problem. What am I suppose to do when there is no ICONV equivalent in ASP 3.0. I have been looking all over Internet for somebody who has already created such function without luck.

8

This is the method I call onboady load:

function getInitialTree(){
tree=new dhtmlXTreeObject('treeBox',"100%","100%",0);
tree.setImagePath("../../images/");
tree.enableIEImageFix(true);
tree.attachEvent("onClick",onNodeClick);
url = "/treecontroller?Action=GetDhtmlxTreeXML";
tree.setXMLAutoLoading(url);
tree.loadXML(url+"&framework.actionContext=20");
}

It gets a perfectly valid XML file encoded in iso-8889-1 (but could be anything else);

The problem I get is that the tree doesn't show on screen, instead I get a msg saying that the the XML is incorrect and when I hit "Generate Report" it shows me my XML file with the right encoding in its header, but the special french characters I use get all scrumbled. How is this possible and how can I solve it? I get the impression that whatever the encoding indicated in the header, the tree translates it into utf-8.

Btw, I know the file is ok, because I am able to see it in the browser when consulting the same url (IE). If I change the encoding to utf-8 in the XML file on the other hand, IE throws an exception.


Thanks.

9

Georgi, try to put the following on the ASP page that you call with a ajax request :

Response.CodePage = 65001
Response.CharSet = "utf-8"

this did the trick for me

10

This article is more than GREAT, you saved my time ... I was going to use the charset convert class.
For those who may interest in using a class to do this:
http://www.phpclasses.org/browse/package/1360.html

Best regards
Oras

11

still i have the same problem
i am using protoform script and prototype 1.6.0.2 to validate and pass the data with(windows-1256) from html form to php file put the result is bad characters
, please any idea

this is the protoform site WWW.CSSREVOLT.COM

Post a comment





(Email will remain hidden)





Please enter the security code you see here




Related entries
Email to a friend
Email this article to:


Your email address:


Message (optional):