Take a look at how the Opus Tools package handles it. So far as I remember, it requires minimal changes to the file(s) containing references to the main entry point and to fopen. It imports the command line arguments as wchar_t and converts them to UTF-8, and has replacement functions for fopen and some text output that accept UTF-8 instead of ANSI. I think it automatically kicks in on definition of WIN32, or something like that. Worth a look, at least as a base implementation. On Tue, 5 Mar 2013 11:39:19 -0500 "Ben Allison" <benski at winamp.com> wrote:> I would be willing to add Windows Unicode support (and large file > mode) to the flac binary tool, but frankly I'm not sure that it could > be done as a single cross-platform source file. There would be a > significant amount of either Windows API function calls or > MSVC-specific extensions to the C library (like _wfopen). It would > quickly devolve into an #ifdef _MSC_VER mess. > > >> ktf said: > >> No unicode support for Windows either. > > > > Somebody that knows about windows unicode needs to work on this and > > supply a patch. I'm happy to help out with guidance and testing. > > _______________________________________________ > flac-dev mailing list > flac-dev at xiph.org > http://lists.xiph.org/mailman/listinfo/flac-dev
My worry is more around metadata handling and parsing commandline input. You can't pass UTF-8 in setlocale on windows (from the docs: "If you provide a code page like UTF-7 or UTF-8, setlocale will fail, returning NULL."). There is a good chance that the behavior will still be correct with UTF-8 strings even when in the "C" locale, but I'm not entirely sure. If so, you might be correct, we could just add _wmain(), have it convert the commandline parameters to UTF-8 and call the existing main() function. We also have to be sure to open the FLAC objects with either callback-functions for I/O or with the version where you pass a FILE * object. Having proper display from the commandline is another issue entirely (and a painful one at that). I have some helper code for console printf that converts as necessary on Windows - but it's glitchy when combined with printf calls from elsewhere in the code due to buffering. -Ben> Take a look at how the Opus Tools package handles it. So far as I > remember, it requires minimal changes to the file(s) containing > references to the main entry point and to fopen. It imports the command > line arguments as wchar_t and converts them to UTF-8, and has > replacement functions for fopen and some text output that accept UTF-8 > instead of ANSI. I think it automatically kicks in on definition of > WIN32, or something like that. Worth a look, at least as a base > implementation. > > On Tue, 5 Mar 2013 11:39:19 -0500 > "Ben Allison" <benski at winamp.com> wrote: > >> I would be willing to add Windows Unicode support (and large file >> mode) to the flac binary tool, but frankly I'm not sure that it could >> be done as a single cross-platform source file. There would be a >> significant amount of either Windows API function calls or >> MSVC-specific extensions to the C library (like _wfopen). It would >> quickly devolve into an #ifdef _MSC_VER mess. >> >> >> ktf said: >> >> No unicode support for Windows either. >> > >> > Somebody that knows about windows unicode needs to work on this and >> > supply a patch. I'm happy to help out with guidance and testing. >> >> _______________________________________________ >> flac-dev mailing list >> flac-dev at xiph.org >> http://lists.xiph.org/mailman/listinfo/flac-dev > > _______________________________________________ > flac-dev mailing list > flac-dev at xiph.org > http://lists.xiph.org/mailman/listinfo/flac-dev >
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 06.03.2013 01:20, Ben Allison wrote:> On 06.03.2013 01:02, Chris Moeller wrote: >> On Tue, 5 Mar 2013 11:39:19 -0500, Ben Allison wrote: >>> On 05.03.2013 12:30, Erik de Castro Lopo wrote: >>>> ktf said: >>>>> No unicode support for Windows either. >>>> >>>> Somebody that knows about windows unicode needs to work on >>>> this and supply a patch. I'm happy to help out with guidance >>>> and testing. >>> I would be willing to add Windows Unicode support (and large >>> file mode) to the flac binary tool, but frankly I'm not sure >>> that it could be done as a single cross-platform source file. >>> There would be a significant amount of either Windows API >>> function calls or MSVC-specific extensions to the C library >>> (like _wfopen). It would quickly devolve into an #ifdef >>> _MSC_VER mess. >>> >> Take a look at how the Opus Tools package handles it. So far as >> I remember, it requires minimal changes to the file(s) >> containing references to the main entry point and to fopen. It >> imports the command line arguments as wchar_t and converts them >> to UTF-8, and has replacement functions for fopen and some text >> output that accept UTF-8 instead of ANSI. I think it >> automatically kicks in on definition of WIN32, or something like >> that. Worth a look, at least as a base implementation. >> > My worry is more around metadata handling and parsing commandline > input. You can't pass UTF-8 in setlocale on windows (from the docs: > "If you provide a code page like UTF-7 or UTF-8, setlocale will > fail, returning NULL.").You will have to hunt down the places where this happens, and do something else. Linking to a good internationalization library would help a lot.> If so, you might be correct, we could just add _wmain(), have it > convert the commandline parameters to UTF-8 and call the existing > main() function._wmain() requires support from the toolchain. AFAIK, only mingw-w64 supports wmain.> Having proper display from the commandline is another issue > entirely (and a painful one at that). I have some helper code for > console printf that converts as necessary on Windows - but it's > glitchy when combined with printf calls from elsewhere in the code > due to buffering.How is it glitchy? -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (MingW32) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQEcBAEBAgAGBQJRNpe7AAoJEOs4Jb6SI2Cw/OAH/RMeFht0NN8CXXA1QZTz976N 4smLqobphozThnJP2zQPH+KJIDTZdY0WryROCO8s0t/j4CGBlTRZPlz2y2yYg49U 3ZPBcZTDSR/cN4toOBcknb9awa8XrGJy7xJcwOPzMbzowPtBpU72ablTG7D8mFow PXhqXma99yLdQSC7LOcB/MAR1LgZxGmUUMsIig4wDL9rhZ5rzmvzLd/WWJow3WPT 4EKChs8ylxjhDrxTkCRNJ5N8WHMY1M39wx5dqkpyP0X61fIaCb0v8JbR10YPzpQG ZJ3nX91N0aiJ3J8YLw7TML1x5xCf7hSr3BYmX2uk5A+/BtsoyeeJJsLhNe7JroY=Qt0d -----END PGP SIGNATURE-----
(2013/03/06 6:20), Ben Allison wrote:> My worry is more around metadata handling and parsing commandline input. > You can't pass UTF-8 in setlocale on windows (from the docs: "If you > provide a code page like UTF-7 or UTF-8, setlocale will fail, returning > NULL."). There is a good chance that the behavior will still be correct > with UTF-8 strings even when in the "C" locale, but I'm not entirely sure.MSVC locale implementation doesn't support UTF-8. However, it's just that you cannot use standard C wchar conversion function such as mbcrtowc(), fgetwc(), fwprintf() or something. You could still use WideCharToMultiByte() or something directly and it's enough.> If so, you might be correct, we could just add _wmain(), have it convert > the commandline parameters to UTF-8 and call the existing main() function._wmain() is poorly (or not) supported by MinGW. Instead, you can use CommandLineToArgvW(), or __wgetmainargs().> Having proper display from the commandline is another issue entirely (and > a painful one at that). I have some helper code for console printf that > converts as necessary on Windows - but it's glitchy when combined with > printf calls from elsewhere in the code due to buffering.I think stdio buffering is not an issue as long as single thread is being used. You could just fflush() before WriteConsoleW() in your printf() wrapper, and it would be enough if synchronization for multithreading is not required. Actually I also have written such kind of wrapper for fdkaac frontend: https://github.com/nu774/fdkaac/blob/master/src/compat_win32.c IIRC opustool just sets UTF-8 codepage(65001) via SetConsoleCP() and directly printf'ing to console with UTF8 string. It works, but it's also glitchy. Since console fonts are bound to each console codepage, console window instantly gets resized when I execute opusinfo command. This behavior might not be visible for users using Unicode font by default (Lucida Console or something).