Jump to content


Check out our Community Blogs

Register and join over 40,000 other developers!


Recent Status Updates

View All Updates

Photo
- - - - -

json_encode() returns unicode

php unicode json

  • Please log in to reply
12 replies to this topic

#1 __ak

__ak

    CC Newcomer

  • Just Joined
  • PipPip
  • 24 posts

Posted 16 March 2011 - 05:05 AM

Hi all,

I'm currently doing a project where it's possible to type in data in HTML to be imploded to and JSON string in the PHP backend.
The user have two input fields: "Title" and "value".
But "value" could contain an JSON object itself which you have "Title" as index in the major object.

I do this on backend:
            foreach($_POST['jsonArguments'] as $key=>$arg) {
    		$val = $_POST['jsonArguments'][$key];
                    $json_parse = json_decode($val);
                    if($json_parse) $val = $json_parse;
                    $json[$arg] = $val;
            }
            $parser_setting_new = json_encode($json);

And when you save and refresh the page, the saved JSON settings should be show as you wrote them.
The problem comes when I try to use Scandinavian (Danish actually) characters like; æ ø å.
The word "måge" (seagull) becomes "måu005ege" because "å" is replaced with "u005e".

My question is, how to I handle this correct so that I can see the actual characters in my frontend instead of unicodes?
  • 0

#2 Orjan

Orjan

    CC Mentor

  • Moderator
  • 2918 posts
  • Location:Karlstad, Sweden
  • Programming Language:C, Java, C++, C#, PHP, JavaScript, Pascal
  • Learning:Java, C#

Posted 16 March 2011 - 01:21 PM

Hello Neighbor!

if it would replace å with u005e, måge would become mu005ege, not måu005ege as you wrote. is it a typing mistake, or is there some other error?
  • 0

I'm a System developer at XLENT Consultant Group mainly working with SugarCRM.
Please DO NOT send mail or PM to me with programming questions, post them in the appropriate forum instead, where I and others can answer you.


#3 Alexander

Alexander

    YOL9

  • Moderator
  • 3963 posts
  • Location:Vancouver, Eh! Cleverness: 200
  • Programming Language:C, C++, PHP, Assembly

Posted 16 March 2011 - 04:53 PM

json_{encode, decode} are fully UTF8 compliant, so you must ensure they are encoded and decoded with that encoding.

Orjan is right to be suspicious, måu005ege is not an artefact of UTF encoding issues (it would not display at all), was the data that you applied json encoding to malformed when transported?

Edited by Alexander, 16 March 2011 - 11:47 PM.

  • 0

All new problems require investigation, and so if errors are problems, try to learn as much as you can and report back.


#4 __ak

__ak

    CC Newcomer

  • Just Joined
  • PipPip
  • 24 posts

Posted 16 March 2011 - 10:59 PM

sorry mate, I meant "mu005ege".

But okay, I just need to utf8_encode all data before saving it to the database?
Do I need to make sure the encoding in the database is UTF-8 as well?

I dot SET NAMES 'utf8' when creating connection.
  • 0

#5 __ak

__ak

    CC Newcomer

  • Just Joined
  • PipPip
  • 24 posts

Posted 16 March 2011 - 11:18 PM

I belive it's json_encode that eats my specialchars.
This scripts does the same as my original:

<?php
$s = 'måge';
echo json_encode($s);
?>

  • 0

#6 __ak

__ak

    CC Newcomer

  • Just Joined
  • PipPip
  • 24 posts

Posted 16 March 2011 - 11:24 PM

Uhh, but I see here that I get m\u005ege instead of mu005ege (must have missed a stripslashes somewhere) - which means I can make a preg_replace on the unicode, but doesn't like that way of making hakcs all the time.
Wanna make it work as supposed.

But I get unicode everytime I use json_encode.
  • 0

#7 __ak

__ak

    CC Newcomer

  • Just Joined
  • PipPip
  • 24 posts

Posted 16 March 2011 - 11:32 PM

@Alexander OFF-T: Sorry about the doubble posts and doubble threads I've made in this category - keeps forgetting someone needs to approve them.
  • 0

#8 Alexander

Alexander

    YOL9

  • Moderator
  • 3963 posts
  • Location:Vancouver, Eh! Cleverness: 200
  • Programming Language:C, C++, PHP, Assembly

Posted 16 March 2011 - 11:54 PM

It is fine about the postings. JSON will create Javascript compatible arrays or objects for serialization purposes, and Javascript will not be able to properly store UTF encoded characters in native strings, so it is converted to the \u0000 format. Do you require Javascript compatibility, or can you use serialization (serialize() and unserialize()) functions which PHP provides for database storage?
  • 0

All new problems require investigation, and so if errors are problems, try to learn as much as you can and report back.


#9 __ak

__ak

    CC Newcomer

  • Just Joined
  • PipPip
  • 24 posts

Posted 17 March 2011 - 01:02 AM

Yeah, I kinda need JS compatibility, 'cause the JSON is splitted up and putted into text fields frontend, through JS.
Could it be PDO that's stripping the backslashes before the 'u' ?
  • 0

#10 Alexander

Alexander

    YOL9

  • Moderator
  • 3963 posts
  • Location:Vancouver, Eh! Cleverness: 200
  • Programming Language:C, C++, PHP, Assembly

Posted 17 March 2011 - 01:08 AM

If you use PDO prepared statements, there is absolutely no escaping required so it would not automatically filter it. It must be before that.
  • 0

All new problems require investigation, and so if errors are problems, try to learn as much as you can and report back.


#11 __ak

__ak

    CC Newcomer

  • Just Joined
  • PipPip
  • 24 posts

Posted 17 March 2011 - 01:47 AM

Okay, so PDO has no role here.

I've tried to utf8 en/decode as I think is right, but i end up with "null" in DB.

            foreach($_POST['jsonArguments'] as $key=>$arg) {
    		$val = $_POST['jsonArguments'][$key];
                    $json_parse = json_decode(utf8_decode($val));
                    if($json_parse) $val = $json_parse;
                    $json[$arg] = $val;
            }
            $result = json_encode(utf8_encode($json));

  • 0

#12 Alexander

Alexander

    YOL9

  • Moderator
  • 3963 posts
  • Location:Vancouver, Eh! Cleverness: 200
  • Programming Language:C, C++, PHP, Assembly

Posted 17 March 2011 - 04:44 PM

That should not matter, somewhere something is stripping the tags, that is why you should look at the data ($_POST['jsonArguments']) and see if the tags are alright there. Also check every time you modify that data.
  • 0

All new problems require investigation, and so if errors are problems, try to learn as much as you can and report back.






Also tagged with one or more of these keywords: php, unicode, json

Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download