similar to: [LLVMdev] [cfe-dev] Unicode path handling on Windows

Right, but maybe if you switch to using tolower_l() and pass an appropriate locale you can get it to work the same way. I'm not sure what locale that would have to be, but it needs to match whatever NTFS uses for its $upcase file. On Oct 3, 2011, at 12:19 PM, Nikola Smiljanic wrote: > CharLowerW does the right thing. But I still need Windows.h to use it :) > > On Mon, Oct 3, 2011

CharLowerW does the right thing. But I still need Windows.h to use it :) On Mon, Oct 3, 2011 at 8:43 PM, Bryce Cogswell <bryceco at gmail.com> wrote: > Locale-specific is not what we want, but I don't believe Windows exposes an > alternative API that does what we want. (Does CharLower give a different > answer than tolower?) > > However, looking over the FileManager code

towlower doesn't seem to work with my test string in Cyrillic. This function does locale-specific conversion, is this what we want? Here's the whole thing, with all the calls to ::stat replaced with llvm::sys::fs::Stat. On Fri, Sep 30, 2011 at 8:04 PM, Bryce Cogswell <bryceco at gmail.com> wrote: > You can use _iswupper and _towlower instead of CharLowerBuffW. They don't

I tried to do the conversion to lowercase inside GetFullPath by adding an additional bool parameter to this function. It's not perfect but seems much better than repeating the whole UTF8 to UTF16 and UTF16 to UTF8 conversion again. The problem I have is with access to CharLowerBuffW. I need Windows.h for this function but when I try to include it I get a bunch of errors because Language

You can use _iswupper and _towlower instead of CharLowerBuffW. They don't require windows.h and work with /Za. On Sep 30, 2011, at 12:24 AM, Nikola Smiljanic wrote: > I tried to do the conversion to lowercase inside GetFullPath by adding an additional bool parameter to this function. It's not perfect but seems much better than repeating the whole UTF8 to UTF16 and UTF16 to UTF8

Locale-specific is not what we want, but I don't believe Windows exposes an alternative API that does what we want. (Does CharLower give a different answer than tolower?) However, looking over the FileManager code a little more I'm not even sure using the path is the best solution, it seems it would be better to use inode like the unix code does. Windows doesn't support inode (the

I agree they are broken on all platforms. However, FileManager.cpp already contains an #if WIN32 conditional around the code calling LowercaseString, so you can use MultiByteToWideChar and CharLowerBuffW directly there, and not call the LowercaseString function. I don't think there are any other places where LowercaseString is called with non-ascii data, so you can punt on fixing it for now.

I have a problem with Lowercase and Uppercase functions. These are broken on all platforms, not only Window, so I can't just #ifdef and use CharLowerBuffW. I need a portable way to convert from UTF8 to UTF16. There is set of functions inside clang/Basic/ConvertUTF, but LLVM can't depend on this. What do I do? On Tue, Sep 27, 2011 at 5:09 AM, Bryce Cogswell <bryceco at yahoo.com>

I think the assert you have for _stat64i32 is fine. It is a constant expression so should compile to nothing, and the chance of the definition changing is pretty much zero. LowercaseString appears to be used by ASM printers where the output is expected to be ASCII, and then some WIN32-conditioned code in FileManager.cpp. I hate to say it but you'll probably need to convert the UTF-8 paths to

Here's a new patch that fixes all the issues mentioned before. Note that this isn't final, I didn't want to replace all calls to ::stat so that it's easier to review. I have only one more questions: 1. _wopen accepts _stat64i32 instead of stat structure. These two are exactly the same, the only difference is that stat uses time_t and _stat64i32 uses __time64_t (time_t is a

On Tue, Sep 20, 2011 at 4:15 PM, Nikola Smiljanic <popizdeh at gmail.com> wrote: > OK since this approach makes sense I'll shoot with my questions :) > 1. Where should get_utf8_argv go and is the name of this function OK? Right > now the function is inside llvm::sys::fs namespace because I need access to > Windows.h, should I leave it there. I don't think it belongs

That should be fine. I don't believe the concern about performing a char-by-char conversion is valid; for example the NTFS-3G driver uses a simplistic upcase table and seems to work fine. I suspect Windows does the same. On Oct 3, 2011, at 1:12 PM, Nikola Smiljanic wrote: > How about this: > > for (int i = 0; i != NumWChars; ++i) > absPath[i] = std::tolower(absPath[i],

The problem is not in the functions that return multibyte strings (the multibyte string is coming from argv) but in the functions that can't handle utf8 input on windows, such as ::open and ::stat. llvm::sys::fs module assumes utf8 input and I don't think this is true for windows. One solution would be to make the module work with multibyte strings as I've done, and the other one

On Tue, Oct 4, 2011 at 4:19 AM, Nikola Smiljanic <popizdeh at gmail.com> wrote: > In that case I think that this is it :) > > On Tue, Oct 4, 2011 at 8:24 AM, Bryce Cogswell <bryceco at gmail.com> wrote: >> >> That should be fine. I don't believe the concern about performing a >> char-by-char conversion is valid; for example the NTFS-3G driver uses a

In that case I think that this is it :) On Tue, Oct 4, 2011 at 8:24 AM, Bryce Cogswell <bryceco at gmail.com> wrote: > That should be fine. I don't believe the concern about performing a > char-by-char conversion is valid; for example the NTFS-3G driver uses a > simplistic upcase table and seems to work fine. I suspect Windows does the > same. > > -------------- next

How about this: for (int i = 0; i != NumWChars; ++i) absPath[i] = std::tolower(absPath[i], std::locale()); seems to be working just fine? On Mon, Oct 3, 2011 at 9:27 PM, Bryce Cogswell <bryceco at gmail.com> wrote: > Right, but maybe if you switch to using tolower_l() and pass an appropriate > locale you can get it to work the same way. I'm not sure what locale that

On Oct 3, 2011, at 11:53 AM, Aaron Ballman wrote: > On Mon, Oct 3, 2011 at 1:43 PM, Bryce Cogswell <bryceco at gmail.com> wrote: >> However, looking over the FileManager code a little more I'm not even sure >> using the path is the best solution, it seems it would be better to use >> inode like the unix code does. Windows doesn't support inode (the s_ino >>

On Mon, Oct 3, 2011 at 1:43 PM, Bryce Cogswell <bryceco at gmail.com> wrote: > However, looking over the FileManager code a little more I'm not even sure > using the path is the best solution, it seems it would be better to use > inode like the unix code does. Windows doesn't support inode (the s_ino > field in stat), but it does have nFileIndexHigh/nFileIndexLow which

As was mentioned once before, the correct solution is to never use multibyte anywhere. Any Windows functions that currently return multibyte strings should be converted to their wide-string (unicode) equivalent, with the result converted to UTF-8. > From: Nikola Smiljanic <popizdeh at gmail.com> > > I think I got it this time. I realized that ::open and ::stat work just fine with

On Tue, Sep 20, 2011 at 11:52 AM, Nikola Smiljanic <popizdeh at gmail.com> wrote: > I spent some more time on this. My idea was to use functionality from > llvm::sys::fs like file_status instead of stat struct, but as it turns out > this is not really possible. file_status structure is not a replacement for > stat, nor are there functions inside llvm::sys::fs that can replace