Xen devel - Apr 2007 - Correct management of .po files for translation

In updating Fedora rawhide to work against xen-unstable.hg (3.0.5) I noticed
that there is now gettext support for translating the error messages returned
from XenAPI in xm. Unfortauntely the way the .po files have been created is
completely wrong & useless for translators to work kwith.

First there is no master .pot file - merely an english
''translation''. Since
the convention is for the untranslated strings to be in english to start with,
providing an english .po  file is redundant unless you specialize it to a
variant like  en_GB.po

The core problem though is that the catalog file ..

tools/python/xen/xm/messages/en/xen-xm.po

..does not contain untranslated strings at all - it is instead full of symbolic
constants.

  msgid "INTERNAL_ERROR"
  msgstr "Internal error: %(1)s."

  msgid "MAP_DUPLICATE_KEY"
  msgstr "This map already contains %(1)s -> %(2)s."

  msgid "MESSAGE_METHOD_UNKNOWN"
  msgstr "The method %(1)s is unsupported."

  msgid "MESSAGE_PARAMETER_COUNT_MISMATCH"
  msgstr "The method %(1)s takes %(2)s argument(s) (%(3)s given)."

There are two critical reasons why msgid should always contain the master
english untranslated string:

  - Translators need to be able to compare the original english alongside
    the translated text to ensure accurate translations. Referring to external
    files to find the english is not practical

  - The msgmerge tool used to periodically update the catelogs looks at
    the msgid entries to identify added / removed / editted strings. If
    the msgid is a symbolic constant it''ll never be able to identify
strings
    which have changed & thus mark them as needing re-translation

Basically the code is abusing the gettext translation service to do both the
constant -> string conversion & the translation in one go. The normal way
to
do the constant -> string conversion is to have a statically declared hash
table in the application itself. This contains the master english strings
annotated with  N_(...string..).  The xgettext program will then generate
the .pot files automatically in the correct format.

I''m attaching an example to show how this ought to work in the Xen
case.
The makefiles could be hooked up to automatically run

  xgettext --keyword=N_ -o xen-xm.pot tools/python/xen/xm/XenAPI.py

Which generates a sensible catalog file looking like

  #: tools/python/xen/xm/XenAPI.py:58
  #, python-format
  msgid "Internal error: %(1)s."
  msgstr ""

  #: tools/python/xen/xm/XenAPI.py:59
  #, python-format
  msgid "This map already contains %(1)s -> %(2)s."
  msgstr ""

  #: tools/python/xen/xm/XenAPI.py:60
  #, python-format
  msgid "The method %(1)s is unsupported."
  msgstr ""

For each new language the translators simply do xen-xm.pot to  fr_FR.po, and 
fill in the msgstr field. When xen-xm.pot is later updated, the translations
can in turn be updated using  msgmerge, etc, etc


Regards,
Dan.
-- 
|=- Red Hat, Engineering, Emerging Technologies, Boston.  +1 978 392 2496 -=|
|=-           Perl modules: http://search.cpan.org/~danberr/              -=|
|=-               Projects: http://freshmeat.net/~danielpb/               -=|
|=-  GnuPG: 7D3B9505   F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505  -=| 


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Wed, Apr 04, 2007 at 05:02:23PM +0100, Daniel P. Berrange wrote:
> In updating Fedora rawhide to work against xen-unstable.hg (3.0.5) I
noticed
> that there is now gettext support for translating the error messages
returned
> from XenAPI in xm. Unfortauntely the way the .po files have been created is
> completely wrong & useless for translators to work kwith.
> 
> First there is no master .pot file - merely an english
''translation''. Since
> the convention is for the untranslated strings to be in english to start
with,
> providing an english .po  file is redundant unless you specialize it to a
> variant like  en_GB.po
> 
> The core problem though is that the catalog file ..
> 
> tools/python/xen/xm/messages/en/xen-xm.po
> 
> ..does not contain untranslated strings at all - it is instead full of
symbolic
> constants.
> 
>   msgid "INTERNAL_ERROR"
>   msgstr "Internal error: %(1)s."
> 
>   msgid "MAP_DUPLICATE_KEY"
>   msgstr "This map already contains %(1)s -> %(2)s."
> 
>   msgid "MESSAGE_METHOD_UNKNOWN"
>   msgstr "The method %(1)s is unsupported."
> 
>   msgid "MESSAGE_PARAMETER_COUNT_MISMATCH"
>   msgstr "The method %(1)s takes %(2)s argument(s) (%(3)s
given)."
> 
> There are two critical reasons why msgid should always contain the master
> english untranslated string:
> 
>   - Translators need to be able to compare the original english alongside
>     the translated text to ensure accurate translations. Referring to
external
>     files to find the english is not practical
> 
>   - The msgmerge tool used to periodically update the catelogs looks at
>     the msgid entries to identify added / removed / editted strings. If
>     the msgid is a symbolic constant it''ll never be able to
identify strings
>     which have changed & thus mark them as needing re-translation
> 
> Basically the code is abusing the gettext translation service to do both
the
> constant -> string conversion & the translation in one go. The
normal way to
> do the constant -> string conversion is to have a statically declared
hash
> table in the application itself. This contains the master english strings
> annotated with  N_(...string..).  The xgettext program will then generate
> the .pot files automatically in the correct format.
> 
> I''m attaching an example to show how this ought to work in the Xen
case.
> The makefiles could be hooked up to automatically run
> 
>   xgettext --keyword=N_ -o xen-xm.pot tools/python/xen/xm/XenAPI.py
Sounds good to me, Daniel, thanks.  Could you write a patch to hook up the
Makefiles as well?  I reckon we''ll be able to sneak this one in for
3.0.5 if
you did.

Ewan.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Wed, Apr 04, 2007 at 05:31:16PM +0100, Ewan Mellor
wrote:
> On Wed, Apr 04, 2007 at 05:02:23PM +0100, Daniel P. Berrange wrote:
> > I''m attaching an example to show how this ought to work in
the Xen case.
> > The makefiles could be hooked up to automatically run
> > 
> >   xgettext --keyword=N_ -o xen-xm.pot tools/python/xen/xm/XenAPI.py
> 
> Sounds good to me, Daniel, thanks.  Could you write a patch to hook up the
> Makefiles as well?  I reckon we''ll be able to sneak this one in
for 3.0.5 if
> you did.
Haha - I walked right into that trap, didn''t I :-) I''ll knock
up a suitable
patch for the Makefile & try and post it tomorrow.

Regards,
Dan.
-- 
|=- Red Hat, Engineering, Emerging Technologies, Boston.  +1 978 392 2496 -=|
|=-           Perl modules: http://search.cpan.org/~danberr/              -=|
|=-               Projects: http://freshmeat.net/~danielpb/               -=|
|=-  GnuPG: 7D3B9505   F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505  -=| 

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Wed, Apr 04, 2007 at 05:31:16PM +0100, Ewan Mellor
wrote:
> On Wed, Apr 04, 2007 at 05:02:23PM +0100, Daniel P. Berrange wrote:
> 
> > Basically the code is abusing the gettext translation service to do
both the
> > constant -> string conversion & the translation in one go. The
normal way to
> > do the constant -> string conversion is to have a statically
declared hash
> > table in the application itself. This contains the master english
strings
> > annotated with  N_(...string..).  The xgettext program will then
generate
> > the .pot files automatically in the correct format.
> > 
> > I''m attaching an example to show how this ought to work in
the Xen case.
> > The makefiles could be hooked up to automatically run
> > 
> >   xgettext --keyword=N_ -o xen-xm.pot tools/python/xen/xm/XenAPI.py
> 
> Sounds good to me, Daniel, thanks.  Could you write a patch to hook up the
> Makefiles as well?  I reckon we''ll be able to sneak this one in
for 3.0.5 if
> you did.
I''m attaching a patch which hooks up the tools/python/Makefile to
update
the master xen-xm.pot file with new translatable text on every biuld, and
to merge these into any translations. The install rule was tweaked to deal
with the different build process.

There is no actual language translation in the patch - when adding new 
translations simply 

   cd xen/xm/messages
   cp xen-xm.pot fr_FR.po
   vi fr_FR.po
    ....

Also edit tools/python/Makefile to add the language code
''fr_FR'' to the
LINGUAS makefile variable.

NB, since I don''t have a current xen-unstable.hg kernel build
operational
on my dev hardware currently I''ve not yet been able to test the correct
operational of the XenAPI.py changes in this patch. Hopefully I''ll be
able
to get a working kernel on this machine soon

 a/tools/python/xen/xm/messages/en/xen-xm.po |   69 ----------------------------
 b/tools/python/remove-potcdate.sed          |   19 +++++++
 b/tools/python/xen/xm/messages/xen-xm.pot   |   63 +++++++++++++++++++++++++
 config/StdGNU.mk                            |    1 
 tools/python/Makefile                       |   68 ++++++++++++++++++++++-----
 tools/python/xen/xm/XenAPI.py               |   18 ++++++-
 7 files changed, 218 insertions(+), 83 deletions(-)


   Signed-off-by: Daniel P. Berrange <berrange@redhat.com>

Regards,
Dan.
-- 
|=- Red Hat, Engineering, Emerging Technologies, Boston.  +1 978 392 2496 -=|
|=-           Perl modules: http://search.cpan.org/~danberr/              -=|
|=-               Projects: http://freshmeat.net/~danielpb/               -=|
|=-  GnuPG: 7D3B9505   F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505  -=| 


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel