character encoding problem £ turn to ?

**ukultra** · 07-24-2006, 11:21 AM

Posted this before for a different reason but never had an answer that worked.

I have a php script - hostdirector, in the config file I use the sign £ for currency.
It works for any emails that are sent out but not for the web page itself.

Now, when I view and copy the souce of a problem page and upload it, it then works. It seems that it happens when the page is being processed on the server.

Changed doctype (always confuses me so help there would be welcome)

Any clues to whats happening would be very welcome.

The site isn't live just yet so you can view it, although I maybe changing it as you try it.

http://www.ukultra.co.uk/sales/client/

If you want to go through whole process use the invoice me payment. Or you can use user : ukultra pass: demodemo to try it out. Go to "order extras" and you'll see the problem - or you should do. If not then its my browsers???

I'm not great with the finer points of web development so speak slowly

Cheers,
Colin

**ukultra** · 07-24-2006, 12:08 PM

Just an update...
If I go into the browser settings and force encoding to Western (ISO-8859-1) it all works. But not sure what default settings are so if users have there's set to AUTO they'd get an error. Hope this helps you experts track the problem down for me.

Also, when using auto, it switches between western (iso-8859-1) and UTF-8 - No expert here but i guess it reads some pages different to others depending on content found.

HELP! I'm goin nuts.

**Spathiphyllum** · 07-24-2006, 01:01 PM

This is the nature of character encoding. By defining it in the header, it tells clients which character set to use to decode the response content.

If you use different OS's, editors, and parsers (servers), they each have the opportunity to (mis)represent characters which it does not recognize in its database of characters.

To solve this issue, the industry has been trying to utilize the Unicode character set which is larger and more uniform than the subsets used in, for example, Western English. You'll need to do a search for unicode representations of "currency pounds" that can be written out in a more strictly defined longform and that will survive all of the parsings that you use to create this character during development.

£ (default windows charset (ASCII subset))
£ (longform)
£ (translated numeric - this bboard actually translates it - this should read "ampersand[&]hash[#]163[163]semicolon[;]")

Do a search for "+unicode +currency" and then make sure your software subroutines either a) all recognize and interpret the symbol format properly, or b) can be edited with a regex substitution pattern replacing the "incorrect" format with the desired one. [e.g. $output = s/£/\£/g;]

**Connie** · 07-24-2006, 01:58 PM

You are using utf-8 for character encoding. I looked at the source on this page and there is no character encoding. So I wonder if the problem is the fact you are using utf-8. This page displays your special charecter.

I think that is what Spath was getting at.

I do know it's not the doctype.

**Spathiphyllum** · 07-24-2006, 03:27 PM

Yes. What Connie stated about doctype is correct. Doctype will not effect characters. It is used to define the markup specifications, i.e. HTML 4.0, XHTML 1.0, XML, etc. of your code - the specific "rules" as it were, that your content is supposed to follow.

Your issue, ukultra, is partially with this:

Code:

<head>
  <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" />
<head>

Now, as pointed out by Connie, you are using utf-8 (I used iso-8859-1), which may be fine, or not. It is one set of many character sets available and will instruct browsers to display ASCII and other symbols a certain way. Some of those character sets are larger than others and will incorporate a rule for the British pound. Others won't and rather than stop sending the source data, they'll substitute that question mark indicating the placeholder for the character it did not recognize.

The regex stuff had to do with the software you have in place outputting dynamic content either to mail or to HTML. Apparently, your email software is using a large enough character set to include the pound symbol that you can read OK, while the browser is using a smaller one. You'll need to change your meta-header to define a different character set (html header section), define more specifically which character you need for the pound symbol (inline HTML extended character definition), or write your own hack in the processing software such that one character for pound is sent to email output while another is sent to HTML output (back-end script recoding).

Does this make sense? There are several ways to approach this issue and the answer depends on several factors. I can't really tell you which one to use.

**ukultra** · 07-24-2006, 04:34 PM

Cheers! Yes it makes some sence to me. I really need to get my head in some books and get to grips with this stuff. I spotted I had this in the pages:
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />

Removed it and everything seems fine.

If you can recommend some good resources that aren't too complicated I'd appreciate it.

Thanks for your help, it now seems a little more understandable!

Colin

**Spathiphyllum** · 07-24-2006, 05:52 PM

Originally Posted by ukultra

If you can recommend some good resources that aren't too complicated I'd appreciate it.

THE authority on this stuff is the World Wide Web Consortium (W3C). It is the international web standards group that defines how internet content should be rendered. Most any resource that teaches webmastering material acquires the source info from W3C and rehashes it in some form or another.

You should learn these first:
HTML Markup
Cascading Style Sheets
Validation

Yes, it will take time, but don't get overwhelmed. A little at a time regularly is best. It will fall into place but start from a generic standard rather than coding for each browser. W3C is perfect for setting a baseline.

If you need the nitty-gritty on something, search for Request for Comments (or more specifically, "RFC"). Many sites carry these technical documents. They are the engineering guides, proposals, and debating papers for the flow of internet information... and they define EVERYTHING. But they can get pretty deep. W3C will instruct you incorporating such technical papers without overwhelming you... and you won't even know it. But the RFC's are the raw data from which W3C creates many of its tutorials.

Good luck.