win32utils devel - Feb 2006 - Fwd: win32-clipboard and Unicode zero bytes

Hi all,

I''m forwarding this message from Brian Marick.  If you
run this test script and then paste the results into a
Unicode aware text editor, you''ll notice that it only
prints one character instead of three.

I tried changing the strlen to _tcslen and strcpy to
_tcscpy, but that didn''t help.  I mucked around a bit
with the MultiByteToWideChar function, too, but didn''t
have any luck.

Any ideas?

Thanks,

Dan 

# test.rb
$:.unshift Dir.pwd
require ''clipboard''

def unicopy(hex_name_array)
 
Win32::Clipboard.set_data(win32unicode(hex_name_array),
                           
Win32::Clipboard::UNICODETEXT)
  true
end


def extract_flipped_bytes(unicode_char_string)
  [unicode_char_string[2,2].to_i(16),
    unicode_char_string[0,2].to_i(16)]
end


def win32unicode(unicode_char_strings)
  flipped_bytes = unicode_char_strings.collect do |
str |
    extract_flipped_bytes(str)
  end
  flipped_bytes.flatten.pack(''C*'')
end

puts unicopy(%w{ 0401 0400 0402 })

Note: forwarded message attached.


__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 
-------------- next part --------------
An embedded message was scrubbed...
From: Brian Marick <marick at exampler.com>
Subject: win32-clipboard and Unicode zero bytes
Date: Mon, 30 Jan 2006 23:18:21 -0600
Size: 2677
Url:
http://rubyforge.org/pipermail/win32utils-devel/attachments/20060201/0d798d05/attachment.eml

Hi,
----- Original Message ----- 
From: "Daniel Berger" <djberg96 at yahoo.com>
To: <win32utils-devel at rubyforge.org>
Sent: Thursday, February 02, 2006 9:39 AM
Subject: [Win32utils-devel] Fwd: win32-clipboard and Unicode zero bytes

> Hi all,
>
> I''m forwarding this message from Brian Marick.  If you
> run this test script and then paste the results into a
> Unicode aware text editor, you''ll notice that it only
> prints one character instead of three.
>
> I tried changing the strlen to _tcslen and strcpy to
> _tcscpy, but that didn''t help.  I mucked around a bit
> with the MultiByteToWideChar function, too, but didn''t
> have any luck.
>
> Any ideas?
>
> Thanks,
>
> Dan
>
Here is some patches of clipboard.c

Line 70:
   hMem = GlobalAlloc(GHND, strlen(data) + sizeof(TCHAR*));
to
   hMem = GlobalAlloc(GHND, RSTRING(rbData)->len + sizeof(TCHAR*));

Line 75:
   strcpy((TCHAR *)GlobalLock(hMem), data);
to
   memcpy((TCHAR *)GlobalLock(hMem), data, RSTRING(rbData)->len);

HTH,

Park Heesob

> -----Original Message-----
> From: win32utils-devel-bounces at rubyforge.org 
> [mailto:win32utils-devel-bounces at rubyforge.org] On Behalf Of 
> Park Heesob
> Sent: Thursday, February 02, 2006 5:07 AM
> To: Development and ideas for win32utils projects
> Subject: Re: [Win32utils-devel] Fwd: win32-clipboard and 
> Unicode zero bytes
<snip>
> Here is some patches of clipboard.c
> 
> Line 70:
>    hMem = GlobalAlloc(GHND, strlen(data) + sizeof(TCHAR*));
> to
>    hMem = GlobalAlloc(GHND, RSTRING(rbData)->len + sizeof(TCHAR*));
> 
> Line 75:
>    strcpy((TCHAR *)GlobalLock(hMem), data);
> to
>    memcpy((TCHAR *)GlobalLock(hMem), data, RSTRING(rbData)->len);
> 
> HTH,
> 
> Park Heesob
Works great, thanks!  Please commit.

Dan

Berger, Daniel wrote:
>>-----Original Message-----
>>From: win32utils-devel-bounces at rubyforge.org 
>>[mailto:win32utils-devel-bounces at rubyforge.org] On Behalf Of 
>>Park Heesob
>>Sent: Thursday, February 02, 2006 5:07 AM
>>To: Development and ideas for win32utils projects
>>Subject: Re: [Win32utils-devel] Fwd: win32-clipboard and 
>>Unicode zero bytes
>>    
>>
>
><snip>
>
>  
>
>>Here is some patches of clipboard.c
>>
>>Line 70:
>>   hMem = GlobalAlloc(GHND, strlen(data) + sizeof(TCHAR*));
>>to
>>   hMem = GlobalAlloc(GHND, RSTRING(rbData)->len + sizeof(TCHAR*));
>>
>>Line 75:
>>   strcpy((TCHAR *)GlobalLock(hMem), data);
>>to
>>   memcpy((TCHAR *)GlobalLock(hMem), data, RSTRING(rbData)->len);
>>
>>HTH,
>>
>>Park Heesob
>>    
>>
>
>Works great, thanks!  Please commit.
>
>Dan
>
>_______________________________________________
>win32utils-devel mailing list
>win32utils-devel at rubyforge.org
>http://rubyforge.org/mailman/listinfo/win32utils-devel
>
>  
>
Nevermind, I''ll commit it now and put out a new release shortly.

Here''s a question for everyone, though - in the age of Unicode, is 
strcpy obsolete?  Should we always use memcpy?

Dan

Hi,
On Thu, 02 Feb 2006 17:28:59 -0700, Daniel Berger wrote
> Berger, Daniel wrote:
> 
> >>-----Original Message-----
> >>From: win32utils-devel-bounces at rubyforge.org 
> >>[mailto:win32utils-devel-bounces at rubyforge.org] On Behalf Of 
> >>Park Heesob
> >>Sent: Thursday, February 02, 2006 5:07 AM
> >>To: Development and ideas for win32utils projects
> >>Subject: Re: [Win32utils-devel] Fwd: win32-clipboard and 
> >>Unicode zero bytes
> >>    
> >>
> >
> ><snip>
> >
> >  
> >
> >>Here is some patches of clipboard.c
> >>
> >>Line 70:
> >>   hMem = GlobalAlloc(GHND, strlen(data) + sizeof(TCHAR*));
> >>to
> >>   hMem = GlobalAlloc(GHND, RSTRING(rbData)->len +
sizeof(TCHAR*));
> >>
> >>Line 75:
> >>   strcpy((TCHAR *)GlobalLock(hMem), data);
> >>to
> >>   memcpy((TCHAR *)GlobalLock(hMem), data,
RSTRING(rbData)->len);
> >>
> >>HTH,
> >>
> >>Park Heesob
> >>    
> >>
> >
> >Works great, thanks!  Please commit.
> >
> >Dan
> >
> Nevermind, I''ll commit it now and put out a new release shortly.
> 
> Here''s a question for everyone, though - in the age of Unicode, is
> strcpy obsolete?  Should we always use memcpy?
> 
> Dan
Refer to http://www.tbray.org/ongoing/When/200x/2003/04/26/UTF
         http://www.i18nguy.com/unicode/c-unicode.html
         http://www.marin.clara.net/tips/unicode_tips.htm

You can use strcpy and strlen in case of UTF-8, but cannot use in UTF-16, UTF-32
etc.
In Unicode environment, you should use wcscpy and wcslen instead of strcpy and
strlen.

Regards,

Park Heesob

Park Heesob wrote:
>Hi,
>On Thu, 02 Feb 2006 17:28:59 -0700, Daniel Berger wrote
>  
>
>>Berger, Daniel wrote:
>>
>>    
>>
>>>>-----Original Message-----
>>>>From: win32utils-devel-bounces at rubyforge.org 
>>>>[mailto:win32utils-devel-bounces at rubyforge.org] On Behalf Of 
>>>>Park Heesob
>>>>Sent: Thursday, February 02, 2006 5:07 AM
>>>>To: Development and ideas for win32utils projects
>>>>Subject: Re: [Win32utils-devel] Fwd: win32-clipboard and 
>>>>Unicode zero bytes
>>>>   
>>>>
>>>>        
>>>>
>>><snip>
>>>
>>> 
>>>
>>>      
>>>
>>>>Here is some patches of clipboard.c
>>>>
>>>>Line 70:
>>>>  hMem = GlobalAlloc(GHND, strlen(data) + sizeof(TCHAR*));
>>>>to
>>>>  hMem = GlobalAlloc(GHND, RSTRING(rbData)->len +
sizeof(TCHAR*));
>>>>
>>>>Line 75:
>>>>  strcpy((TCHAR *)GlobalLock(hMem), data);
>>>>to
>>>>  memcpy((TCHAR *)GlobalLock(hMem), data,
RSTRING(rbData)->len);
>>>>
>>>>HTH,
>>>>
>>>>Park Heesob
>>>>   
>>>>
>>>>        
>>>>
>>>Works great, thanks!  Please commit.
>>>
>>>Dan
>>>
>>>      
>>>
>>Nevermind, I''ll commit it now and put out a new release
shortly.
>>
>>Here''s a question for everyone, though - in the age of Unicode,
is
>>strcpy obsolete?  Should we always use memcpy?
>>
>>Dan
>>    
>>
>
>Refer to http://www.tbray.org/ongoing/When/200x/2003/04/26/UTF
>         http://www.i18nguy.com/unicode/c-unicode.html
>         http://www.marin.clara.net/tips/unicode_tips.htm
>
>You can use strcpy and strlen in case of UTF-8, but cannot use in UTF-16,
UTF-32 etc.
>In Unicode environment, you should use wcscpy and wcslen instead of strcpy
and strlen.
>
>Regards,
>
>Park Heesob
>  
>
But, with UNICODE defined, doesn''t _tcscpy and _tcslen use wcscpy and 
wcslen behind the scenes?  Based on a comment from someone else it looks 
like the issue is with multibyte character sets, in which case MBCS 
needs to be defined in order for the tchar functions to handle this 
situation correctly?

Is there any downside to just defining MBCS (instead of UNICODE) in all 
C extensions for Windows?

Thanks for the links, btw.

Regards,

Dan