Home All Groups Group Topic Archive Search About


Author
25 Apr 2006 7:50 AM
Meelis Lilbok
Hi

My ASP pages uses UTF-8 encoding.

How to convert UTF-8 text from Request.Form("text") to UNICODE for searching
frm MSSQL Database?



Best regards;
Meelis

Author
25 Apr 2006 9:38 AM
Anthony Jones
Show quote
"Meelis Lilbok" <meelis.lil***@deltmar.ee> wrote in message
news:%23yAYg0DaGHA.4036@TK2MSFTNGP04.phx.gbl...
> Hi
>
> My ASP pages uses UTF-8 encoding.
>
> How to convert UTF-8 text from Request.Form("text") to UNICODE for
searching
> frm MSSQL Database?
>
>
>
> Best regards;
> Meelis
>
>
>

x = Request.Form("text").

x now contains a Unicode string

When passing to a ADO command object parameter make sure the parameter type
is adVarWChar.

Anthony.
Author
25 Apr 2006 9:58 AM
Meelis Lilbok
>
> x = Request.Form("text").

Nope, x is in UTF-8 format! Thats the problem

I use activex dll and API calls to convert UTF-8 to UNICODE, but where use
of activex is disabled this will not work

Meelis
Author
25 Apr 2006 11:14 AM
Anthony Jones
"Meelis Lilbok" <meelis.lil***@deltmar.ee> wrote in message
news:%23G%23dI8EaGHA.4780@TK2MSFTNGP02.phx.gbl...
> >
> > x = Request.Form("text").
>
> Nope, x is in UTF-8 format! Thats the problem
>
> I use activex dll and API calls to convert UTF-8 to UNICODE, but where use
> of activex is disabled this will not work
>
> Meelis
>

VBScript supports only one string format and that is Unicode.

I suspect that the form submission is using UTF-8 but the server side script
doesn't know that and is treating it as ISO-8859-1 or the like.  Hence you
are getting a Unicode string that contains a series of UTF-8 encodings.

What is the character encoding of page that contains the text control?

Does the page actually inform the client of the character encoding used for
the page?

What method is used to submit the form GET or POST?

What is the Enctype of the form?

Is AcceptCharset specified for the Form?

What Browser are you using?

Anthony.
Author
25 Apr 2006 12:21 PM
Meelis Lilbok
Show quote
> What is the character encoding of page that contains the text control?
UTF-8


>
> Does the page actually inform the client of the character encoding used
> for
> the page?
Yes


> What method is used to submit the form GET or POST?
POST

> What is the Enctype of the form?

None, because page encoding is UTF-8

> Is AcceptCharset specified for the Form?
No

> What Browser are you using?
IE6

Meelis
Author
25 Apr 2006 12:26 PM
Meelis Lilbok
For example

If i enter into text box estonian word "väike"
and submit form to antoher pages search.asp
and read Request.Form("text")
i get väike (UTF-8)



Meelis








Show quote
"Anthony Jones" <A**@yadayadayada.com> wrote in message
news:ulUWzlFaGHA.3612@TK2MSFTNGP03.phx.gbl...
>
> "Meelis Lilbok" <meelis.lil***@deltmar.ee> wrote in message
> news:%23G%23dI8EaGHA.4780@TK2MSFTNGP02.phx.gbl...
>> >
>> > x = Request.Form("text").
>>
>> Nope, x is in UTF-8 format! Thats the problem
>>
>> I use activex dll and API calls to convert UTF-8 to UNICODE, but where
>> use
>> of activex is disabled this will not work
>>
>> Meelis
>>
>
> VBScript supports only one string format and that is Unicode.
>
> I suspect that the form submission is using UTF-8 but the server side
> script
> doesn't know that and is treating it as ISO-8859-1 or the like.  Hence you
> are getting a Unicode string that contains a series of UTF-8 encodings.
>
> What is the character encoding of page that contains the text control?
>
> Does the page actually inform the client of the character encoding used
> for
> the page?
>
> What method is used to submit the form GET or POST?
>
> What is the Enctype of the form?
>
> Is AcceptCharset specified for the Form?
>
> What Browser are you using?
>
> Anthony.
>
>
Author
25 Apr 2006 9:34 PM
Anthony Jones
"Meelis Lilbok" <meelis.lil***@deltmar.ee> wrote in message
news:emhxcOGaGHA.4752@TK2MSFTNGP02.phx.gbl...
> For example
>
> If i enter into text box estonian word "väike"
> and submit form to antoher pages search.asp
> and read Request.Form("text")
> i get väike (UTF-8)
>
>

Having looked into it a bit more it would seem that the forms approach just
isn't compatible with UTF-8 or unicode.  There doesn't seem to be a way to
inform the server of the actual charset used to encode the form values.

I'm actually quite amazed at this.

What do you actually need to do?

Do you need to support input characters beyond ISO-8859-1?  If not I would
suggest you ditch UTF-8 and use ISO-8859-1 everywhere instead.

Other wise it is possible to do the decoding in VBScript yourself but it's
really messy.  A small VB6 component would make this a lot easier.

Ditching Forms may be another option and post XML instead.  (This is what I
do, I don't use forms)

Anthony.
Author
26 Apr 2006 5:54 AM
Meelis Lilbok
Hi

cant use  ISO-8859-1, beacuse i need support cyrillic chars too.
its easier to use my activex dll with convert functions :))


Best Regadrs;
Meelis




Show quote
"Anthony Jones" <A**@yadayadayada.com> wrote in message
news:%23lHNaALaGHA.440@TK2MSFTNGP05.phx.gbl...
>
> "Meelis Lilbok" <meelis.lil***@deltmar.ee> wrote in message
> news:emhxcOGaGHA.4752@TK2MSFTNGP02.phx.gbl...
>> For example
>>
>> If i enter into text box estonian word "väike"
>> and submit form to antoher pages search.asp
>> and read Request.Form("text")
>> i get väike (UTF-8)
>>
>>
>
> Having looked into it a bit more it would seem that the forms approach
> just
> isn't compatible with UTF-8 or unicode.  There doesn't seem to be a way to
> inform the server of the actual charset used to encode the form values.
>
> I'm actually quite amazed at this.
>
> What do you actually need to do?
>
> Do you need to support input characters beyond ISO-8859-1?  If not I would
> suggest you ditch UTF-8 and use ISO-8859-1 everywhere instead.
>
> Other wise it is possible to do the decoding in VBScript yourself but it's
> really messy.  A small VB6 component would make this a lot easier.
>
> Ditching Forms may be another option and post XML instead.  (This is what
> I
> do, I don't use forms)
>
> Anthony.
>
>
Author
26 Apr 2006 2:07 PM
Egbert Nierop (MVP for IIS)
"Meelis Lilbok" <meelis.lil***@deltmar.ee> wrote in message
news:%23yAYg0DaGHA.4036@TK2MSFTNGP04.phx.gbl...
> Hi
>
> My ASP pages uses UTF-8 encoding.
>
> How to convert UTF-8 text from Request.Form("text") to UNICODE for
> searching frm MSSQL Database?

use at the first line of your ASP page
<% codepage=65001%>

--
compatible web farm Session replacement for Asp and Asp.Net
http://www.nieropwebconsult.nl/asp_session_manager.htm
Author
26 Apr 2006 3:19 PM
Anthony Jones
Show quote
"Egbert Nierop (MVP for IIS)" <egbert_nierop@nospam.invalid> wrote in
message news:udU5PrTaGHA.3992@TK2MSFTNGP05.phx.gbl...
>
> "Meelis Lilbok" <meelis.lil***@deltmar.ee> wrote in message
> news:%23yAYg0DaGHA.4036@TK2MSFTNGP04.phx.gbl...
> > Hi
> >
> > My ASP pages uses UTF-8 encoding.
> >
> > How to convert UTF-8 text from Request.Form("text") to UNICODE for
> > searching frm MSSQL Database?
>
> use at the first line of your ASP page
> <% codepage=65001%>
>

did you mean:-

<%@ codepage=65001 %>

I don't think that helps.  The value of session.codepage doesn't seem to
impact the assumptions made by server about the encoding of the request
data.



Show quote
> --
> compatible web farm Session replacement for Asp and Asp.Net
> http://www.nieropwebconsult.nl/asp_session_manager.htm
>
Author
26 Apr 2006 8:33 PM
Egbert Nierop (MVP for IIS)
Show quote
"Anthony Jones" <A**@yadayadayada.com> wrote in message
news:esPrbTUaGHA.4036@TK2MSFTNGP04.phx.gbl...
>
> "Egbert Nierop (MVP for IIS)" <egbert_nierop@nospam.invalid> wrote in
> message news:udU5PrTaGHA.3992@TK2MSFTNGP05.phx.gbl...
>>
>> "Meelis Lilbok" <meelis.lil***@deltmar.ee> wrote in message
>> news:%23yAYg0DaGHA.4036@TK2MSFTNGP04.phx.gbl...
>> > Hi
>> >
>> > My ASP pages uses UTF-8 encoding.
>> >
>> > How to convert UTF-8 text from Request.Form("text") to UNICODE for
>> > searching frm MSSQL Database?
>>
>> use at the first line of your ASP page
>> <% codepage=65001%>
>>
>
> did you mean:-
>
> <%@ codepage=65001 %>
>
> I don't think that helps.  The value of session.codepage doesn't seem to
> impact the assumptions made by server about the encoding of the request
> data.

however you are wrong :)

This really is saying that all input Request.* and output (response.write)
processes UTF-8 format.

Show quote
>
>
>> --
>> compatible web farm Session replacement for Asp and Asp.Net
>> http://www.nieropwebconsult.nl/asp_session_manager.htm
>>
>
>
Author
27 Apr 2006 6:03 AM
Meelis Lilbok
Hi Egbert


Problem is not displayng UTF-8, all pages are using UTF-8
Problem is when i wanna make a query from MSSQL server, then i must convert
UTF-8 to UNICODE.

And <% codepage=65001%> does not work on IIS4 :)

And this is only possible when i use ActiveX DLL with MultiByteToWidechar
and WideCharToMultybite API's.

Meelis






Show quote
"Egbert Nierop (MVP for IIS)" <egbert_nierop@nospam.invalid> wrote in
message news:eYaltCXaGHA.4580@TK2MSFTNGP03.phx.gbl...
>
> "Anthony Jones" <A**@yadayadayada.com> wrote in message
> news:esPrbTUaGHA.4036@TK2MSFTNGP04.phx.gbl...
>>
>> "Egbert Nierop (MVP for IIS)" <egbert_nierop@nospam.invalid> wrote in
>> message news:udU5PrTaGHA.3992@TK2MSFTNGP05.phx.gbl...
>>>
>>> "Meelis Lilbok" <meelis.lil***@deltmar.ee> wrote in message
>>> news:%23yAYg0DaGHA.4036@TK2MSFTNGP04.phx.gbl...
>>> > Hi
>>> >
>>> > My ASP pages uses UTF-8 encoding.
>>> >
>>> > How to convert UTF-8 text from Request.Form("text") to UNICODE for
>>> > searching frm MSSQL Database?
>>>
>>> use at the first line of your ASP page
>>> <% codepage=65001%>
>>>
>>
>> did you mean:-
>>
>> <%@ codepage=65001 %>
>>
>> I don't think that helps.  The value of session.codepage doesn't seem to
>> impact the assumptions made by server about the encoding of the request
>> data.
>
> however you are wrong :)
>
> This really is saying that all input Request.* and output (response.write)
> processes UTF-8 format.
>
>>
>>
>>> --
>>> compatible web farm Session replacement for Asp and Asp.Net
>>> http://www.nieropwebconsult.nl/asp_session_manager.htm
>>>
>>
>>
>
Author
27 Apr 2006 9:44 AM
Egbert Nierop (MVP for IIS)
"Meelis Lilbok" <meelis.lil***@deltmar.ee> wrote in message
news:Od67UBcaGHA.4416@TK2MSFTNGP04.phx.gbl...
> Hi Egbert
>
>
> Problem is not displayng UTF-8, all pages are using UTF-8
> Problem is when i wanna make a query from MSSQL server, then i must
> convert UTF-8 to UNICODE.
>
> And <% codepage=65001%> does not work on IIS4 :)

Why didn't you say so.
IIS4 indeed does not support that. Or better said, Oleautomation does not
support, so ADO and others do not support that either.
I'd really work on asking your boss upgrading! Because, if you need to
convert it manually, it will be a hard job, you'll end up converting all SQL
data / user-input data etc!


Show quote
>And this is only possible when i use ActiveX DLL with MultiByteToWidechar
>and WideCharToMultybite API's.

> Meelis
>
>
>
>
>
>
> "Egbert Nierop (MVP for IIS)" <egbert_nierop@nospam.invalid> wrote in
> message news:eYaltCXaGHA.4580@TK2MSFTNGP03.phx.gbl...
>>
>> "Anthony Jones" <A**@yadayadayada.com> wrote in message
>> news:esPrbTUaGHA.4036@TK2MSFTNGP04.phx.gbl...
>>>
>>> "Egbert Nierop (MVP for IIS)" <egbert_nierop@nospam.invalid> wrote in
>>> message news:udU5PrTaGHA.3992@TK2MSFTNGP05.phx.gbl...
>>>>
>>>> "Meelis Lilbok" <meelis.lil***@deltmar.ee> wrote in message
>>>> news:%23yAYg0DaGHA.4036@TK2MSFTNGP04.phx.gbl...
>>>> > Hi
>>>> >
>>>> > My ASP pages uses UTF-8 encoding.
>>>> >
>>>> > How to convert UTF-8 text from Request.Form("text") to UNICODE for
>>>> > searching frm MSSQL Database?
>>>>
>>>> use at the first line of your ASP page
>>>> <% codepage=65001%>
>>>>
>>>
>>> did you mean:-
>>>
>>> <%@ codepage=65001 %>
>>>
>>> I don't think that helps.  The value of session.codepage doesn't seem to
>>> impact the assumptions made by server about the encoding of the request
>>> data.
>>
>> however you are wrong :)
>>
>> This really is saying that all input Request.* and output
>> (response.write) processes UTF-8 format.
>>
>>>
>>>
>>>> --
>>>> compatible web farm Session replacement for Asp and Asp.Net
>>>> http://www.nieropwebconsult.nl/asp_session_manager.htm
>>>>
>>>
>>>
>>
>
>
Author
27 Apr 2006 10:11 AM
Meelis Lilbok
Yeah i know

Some our clients still!! use IIS4 and then i use again my ActiveX DLL to
convert all strings to UTF-8, works fine ;)


Meelis



Show quote
"Egbert Nierop (MVP for IIS)" <egbert_nierop@nospam.invalid> wrote in
message news:Oq$6y8daGHA.1352@TK2MSFTNGP05.phx.gbl...
>
> "Meelis Lilbok" <meelis.lil***@deltmar.ee> wrote in message
> news:Od67UBcaGHA.4416@TK2MSFTNGP04.phx.gbl...
>> Hi Egbert
>>
>>
>> Problem is not displayng UTF-8, all pages are using UTF-8
>> Problem is when i wanna make a query from MSSQL server, then i must
>> convert UTF-8 to UNICODE.
>>
>> And <% codepage=65001%> does not work on IIS4 :)
>
> Why didn't you say so.
> IIS4 indeed does not support that. Or better said, Oleautomation does not
> support, so ADO and others do not support that either.
> I'd really work on asking your boss upgrading! Because, if you need to
> convert it manually, it will be a hard job, you'll end up converting all
> SQL data / user-input data etc!
>
>
>>And this is only possible when i use ActiveX DLL with MultiByteToWidechar
>>and WideCharToMultybite API's.
>
>> Meelis
>>
>>
>>
>>
>>
>>
>> "Egbert Nierop (MVP for IIS)" <egbert_nierop@nospam.invalid> wrote in
>> message news:eYaltCXaGHA.4580@TK2MSFTNGP03.phx.gbl...
>>>
>>> "Anthony Jones" <A**@yadayadayada.com> wrote in message
>>> news:esPrbTUaGHA.4036@TK2MSFTNGP04.phx.gbl...
>>>>
>>>> "Egbert Nierop (MVP for IIS)" <egbert_nierop@nospam.invalid> wrote in
>>>> message news:udU5PrTaGHA.3992@TK2MSFTNGP05.phx.gbl...
>>>>>
>>>>> "Meelis Lilbok" <meelis.lil***@deltmar.ee> wrote in message
>>>>> news:%23yAYg0DaGHA.4036@TK2MSFTNGP04.phx.gbl...
>>>>> > Hi
>>>>> >
>>>>> > My ASP pages uses UTF-8 encoding.
>>>>> >
>>>>> > How to convert UTF-8 text from Request.Form("text") to UNICODE for
>>>>> > searching frm MSSQL Database?
>>>>>
>>>>> use at the first line of your ASP page
>>>>> <% codepage=65001%>
>>>>>
>>>>
>>>> did you mean:-
>>>>
>>>> <%@ codepage=65001 %>
>>>>
>>>> I don't think that helps.  The value of session.codepage doesn't seem
>>>> to
>>>> impact the assumptions made by server about the encoding of the request
>>>> data.
>>>
>>> however you are wrong :)
>>>
>>> This really is saying that all input Request.* and output
>>> (response.write) processes UTF-8 format.
>>>
>>>>
>>>>
>>>>> --
>>>>> compatible web farm Session replacement for Asp and Asp.Net
>>>>> http://www.nieropwebconsult.nl/asp_session_manager.htm
>>>>>
>>>>
>>>>
>>>
>>
>>
>
Author
27 Apr 2006 12:43 PM
Anthony Jones
Show quote
"Egbert Nierop (MVP for IIS)" <egbert_nierop@nospam.invalid> wrote in
message news:eYaltCXaGHA.4580@TK2MSFTNGP03.phx.gbl...
>
> "Anthony Jones" <A**@yadayadayada.com> wrote in message
> news:esPrbTUaGHA.4036@TK2MSFTNGP04.phx.gbl...
> >
> > "Egbert Nierop (MVP for IIS)" <egbert_nierop@nospam.invalid> wrote in
> > message news:udU5PrTaGHA.3992@TK2MSFTNGP05.phx.gbl...
> >>
> >> "Meelis Lilbok" <meelis.lil***@deltmar.ee> wrote in message
> >> news:%23yAYg0DaGHA.4036@TK2MSFTNGP04.phx.gbl...
> >> > Hi
> >> >
> >> > My ASP pages uses UTF-8 encoding.
> >> >
> >> > How to convert UTF-8 text from Request.Form("text") to UNICODE for
> >> > searching frm MSSQL Database?
> >>
> >> use at the first line of your ASP page
> >> <% codepage=65001%>
> >>
> >
> > did you mean:-
> >
> > <%@ codepage=65001 %>
> >
> > I don't think that helps.  The value of session.codepage doesn't seem to
> > impact the assumptions made by server about the encoding of the request
> > data.
>
> however you are wrong :)
>

I am.  Don't how I managed it in my first round of tests.  Did them again
and it works as you say.

The receiving page needs to be using a codepage that matches the character
set that the client browser thinks the source page is using.

In IIS 5.1/IIS 6 setting Response.codepage has the same effect which is a
bit counter intuative.


Show quote
> This really is saying that all input Request.* and output (response.write)
> processes UTF-8 format.
>
> >
> >
> >> --
> >> compatible web farm Session replacement for Asp and Asp.Net
> >> http://www.nieropwebconsult.nl/asp_session_manager.htm
> >>
> >
> >
>
Author
28 Apr 2006 6:47 AM
Egbert Nierop (MVP for IIS)
Show quote
"Anthony Jones" <A**@yadayadayada.com> wrote in message
news:uB7KrgfaGHA.3408@TK2MSFTNGP04.phx.gbl...
>
> "Egbert Nierop (MVP for IIS)" <egbert_nierop@nospam.invalid> wrote in
> message news:eYaltCXaGHA.4580@TK2MSFTNGP03.phx.gbl...
>>
>> "Anthony Jones" <A**@yadayadayada.com> wrote in message
>> news:esPrbTUaGHA.4036@TK2MSFTNGP04.phx.gbl...
>> >
>> > "Egbert Nierop (MVP for IIS)" <egbert_nierop@nospam.invalid> wrote in
>> > message news:udU5PrTaGHA.3992@TK2MSFTNGP05.phx.gbl...
>> >>
>> >> "Meelis Lilbok" <meelis.lil***@deltmar.ee> wrote in message
>> >> news:%23yAYg0DaGHA.4036@TK2MSFTNGP04.phx.gbl...
>> >> > Hi
>> >> >
>> >> > My ASP pages uses UTF-8 encoding.
>> >> >
>> >> > How to convert UTF-8 text from Request.Form("text") to UNICODE for
>> >> > searching frm MSSQL Database?
>> >>
>> >> use at the first line of your ASP page
>> >> <% codepage=65001%>
>> >>
>> >
>> > did you mean:-
>> >
>> > <%@ codepage=65001 %>
>> >
>> > I don't think that helps.  The value of session.codepage doesn't seem
>> > to
>> > impact the assumptions made by server about the encoding of the request
>> > data.
>>
>> however you are wrong :)
>>
>
> I am.  Don't how I managed it in my first round of tests.  Did them again
> and it works as you say.
>
> The receiving page needs to be using a codepage that matches the character
> set that the client browser thinks the source page is using.

Right, and that is set by using

Response.CharSet = "utf-8"

Show quote
> In IIS 5.1/IIS 6 setting Response.codepage has the same effect which is a
> bit counter intuative.
>
>
>> This really is saying that all input Request.* and output
>> (response.write)
>> processes UTF-8 format.

AddThis Social Bookmark Button