Fontconfig - Dec 2005 - Strange behaviour of fc-match of fontconfig current cvs (2.3.93).

Hi,
  After upgrading to fontconfig 2.3.93, the behaviour of  fc-match
becomes very strange. First, fonts can''t be matched by their localized
names anymore. It''s ok when using fontconfig 2.3.91 or earlier.
Second, fc-match gives a different default font than the one I set in
config file.
  I think these issues may be caused by the updating of fcmatch.c
between 2.3.91 and 2.3.93. Do you have any idea about it?

  I tested it on SUSE Linux 10.0 with latest fontconfig installed. I
have four Simplified Chinese fonts installed:

FZHeiTi,????:style=Regular
FZFangSong,????:style=Regular
FZKaiTi,????:style=Regular
FZSongTi,????:style=Regular

  Run fc-match without any parameter will return:
FZFangSong.ttf: "FZFangSong" "Regular"

  However, it should return FZSongTi according to my config file.

  And I can only match these fonts with their latin name, like:

$ fc-match FZSongTi
FZSongTi.ttf: "FZSongTi" "Regular"

  Matching with localized name will give wrong result:

$ fc-match ????
FZFangSong.ttf: "FZFangSong" "Regular"

Regards
James Su

Hi,
  Reverting FcFontSetMatch()  to the one in 2.3.91 does fix this
issue. So there is something wrong in the new algorithm.
  I think at least we should reuse FcCompare() in FcFontSetMatch()
rather than using another piece of code. FcFontSetSort should have the
same result with FcFontSetMatch () at this point.

Regards
James Su

On 12/25/05, Zhe Su <james.su@gmail.com> wrote:
> Hi,
>   After upgrading to fontconfig 2.3.93, the behaviour of  fc-match
> becomes very strange. First, fonts can''t be matched by their
localized
> names anymore. It''s ok when using fontconfig 2.3.91 or earlier.
> Second, fc-match gives a different default font than the one I set in
> config file.
>   I think these issues may be caused by the updating of fcmatch.c
> between 2.3.91 and 2.3.93. Do you have any idea about it?
>
>   I tested it on SUSE Linux 10.0 with latest fontconfig installed. I
> have four Simplified Chinese fonts installed:
>
> FZHeiTi,????:style=Regular
> FZFangSong,????:style=Regular
> FZKaiTi,????:style=Regular
> FZSongTi,????:style=Regular
>
>   Run fc-match without any parameter will return:
> FZFangSong.ttf: "FZFangSong" "Regular"
>
>   However, it should return FZSongTi according to my config file.
>
>   And I can only match these fonts with their latin name, like:
>
> $ fc-match FZSongTi
> FZSongTi.ttf: "FZSongTi" "Regular"
>
>   Matching with localized name will give wrong result:
>
> $ fc-match ????
> FZFangSong.ttf: "FZFangSong" "Regular"
>
> Regards
> James Su
>

Quoting Zhe Su <james.su@gmail.com>:
> Hi,
>  Reverting FcFontSetMatch()  to the one in 2.3.91 does fix this
> issue. So there is something wrong in the new algorithm.
>  I think at least we should reuse FcCompare() in FcFontSetMatch()
> rather than using another piece of code. FcFontSetSort should have the
> same result with FcFontSetMatch () at this point.
Inlining and doing optimizations on FcCompare results in a huge perf gain
whenever this codepath is used.  Of course it should work correctly, but
I''d
rather fix the bug than reuse FcCompare.

pat

Quoting Zhe Su <james.su@gmail.com>:
> Hi,
>  Yes. I suggest to revert to the old FcFontSetMatch(), before fixing
> the new code.
I''d rather give it some time to have someone (preferably the SuSE guys)
fix it
first, since we''re in a devel series right now.  If the problem
persists for
too long then we can revert it.

pat

Zhe Su <james.su@gmail.com> ????????:
> Hi,
>   After upgrading to fontconfig 2.3.93, the behaviour of  fc-match
> becomes very strange. First, fonts can''t be matched by their
localized
> names anymore. It''s ok when using fontconfig 2.3.91 or earlier.
> Second, fc-match gives a different default font than the one I set in
> config file.
>   I think these issues may be caused by the updating of fcmatch.c
> between 2.3.91 and 2.3.93. Do you have any idea about it?
>
>   I tested it on SUSE Linux 10.0 with latest fontconfig installed. I
> have four Simplified Chinese fonts installed:
>
> FZHeiTi,????:style=Regular
> FZFangSong,????:style=Regular
> FZKaiTi,????:style=Regular
> FZSongTi,????:style=Regular
>
>   Run fc-match without any parameter will return:
> FZFangSong.ttf: "FZFangSong" "Regular"
>
>   However, it should return FZSongTi according to my config file.
>
>   And I can only match these fonts with their latin name, like:
>
> $ fc-match FZSongTi
> FZSongTi.ttf: "FZSongTi" "Regular"
>
>   Matching with localized name will give wrong result:
>
> $ fc-match ????
> FZFangSong.ttf: "FZFangSong" "Regular"
I cannot reproduce this with the fontconfig-2.3.93.20051222 (CVS
checkout of fontconfig 2.4 branch from December 22th 2005) packages I
built for SuSE 10.0 (at ftp://ftp.suse.com/pub/projects/m17n/10.0):

mfabian@magellan:~$ LANG=zh_CN.UTF-8 fc-match
FZSongTi.ttf: "FZSongTi" "Regular"
mfabian@magellan:~$ LC_ALL=zh_CN.UTF-8 fc-match
FZSongTi.ttf: "FZSongTi" "Regular"
mfabian@magellan:~$ LC_ALL=zh_CN.UTF-8 fc-match FZSongTi
FZSongTi.ttf: "FZSongTi" "Regular"
mfabian@magellan:~$ LC_ALL=zh_CN.UTF-8 fc-match ????
FZSongTi.ttf: "????" "Regular"
mfabian@magellan:~$ rpm -q fontconfig
fontconfig-2.3.93.20051222-0.1
mfabian@magellan:~$

-- 
Mike FABIAN   <mfabian@suse.de>   http://www.suse.de/~mfabian
?????????????

Hi,
  Try match ·½Õý¿¬Ìå :-)

Regards
James Su

On 12/28/05, Mike FABIAN <mfabian@suse.de> wrote:
> Zhe Su <james.su@gmail.com> ¤µ¤ó¤Ï•ø¤­¤Þ¤·¤¿:
>
> > Hi,
> >   After upgrading to fontconfig 2.3.93, the behaviour of  fc-match
> > becomes very strange. First, fonts can't be matched by their
localized
> > names anymore. It's ok when using fontconfig 2.3.91 or earlier.
> > Second, fc-match gives a different default font than the one I set in
> > config file.
> >   I think these issues may be caused by the updating of fcmatch.c
> > between 2.3.91 and 2.3.93. Do you have any idea about it?
> >
> >   I tested it on SUSE Linux 10.0 with latest fontconfig installed. I
> > have four Simplified Chinese fonts installed:
> >
> > FZHeiTi,·½ÕýºÚÌå:style=Regular
> > FZFangSong,·½Õý·ÂËÎ:style=Regular
> > FZKaiTi,·½Õý¿¬Ìå:style=Regular
> > FZSongTi,·½ÕýËÎÌå:style=Regular
> >
> >   Run fc-match without any parameter will return:
> > FZFangSong.ttf: "FZFangSong" "Regular"
> >
> >   However, it should return FZSongTi according to my config file.
> >
> >   And I can only match these fonts with their latin name, like:
> >
> > $ fc-match FZSongTi
> > FZSongTi.ttf: "FZSongTi" "Regular"
> >
> >   Matching with localized name will give wrong result:
> >
> > $ fc-match ·½ÕýËÎÌå
> > FZFangSong.ttf: "FZFangSong" "Regular"
>
> I cannot reproduce this with the fontconfig-2.3.93.20051222 (CVS
> checkout of fontconfig 2.4 branch from December 22th 2005) packages I
> built for SuSE 10.0 (at ftp://ftp.suse.com/pub/projects/m17n/10.0):
>
> mfabian@magellan:~$ LANG=zh_CN.UTF-8 fc-match
> FZSongTi.ttf: "FZSongTi" "Regular"
> mfabian@magellan:~$ LC_ALL=zh_CN.UTF-8 fc-match
> FZSongTi.ttf: "FZSongTi" "Regular"
> mfabian@magellan:~$ LC_ALL=zh_CN.UTF-8 fc-match FZSongTi
> FZSongTi.ttf: "FZSongTi" "Regular"
> mfabian@magellan:~$ LC_ALL=zh_CN.UTF-8 fc-match ·½ÕýËÎÌå
> FZSongTi.ttf: "·½ÕýËÎÌå" "Regular"
> mfabian@magellan:~$ rpm -q fontconfig
> fontconfig-2.3.93.20051222-0.1
> mfabian@magellan:~$
>
> --
> Mike FABIAN   <mfabian@suse.de>   http://www.suse.de/~mfabian
> ˯Ãß²»×ã¤Ï¤¤¤¤ÊËÊ¤Δ³¤À¡£
>
>

Zhe Su <james.su@gmail.com> :
> Hi,
>   Try match  :-)
Yes, this doesn''t work for me either:

mfabian@magellan:~$ LC_ALL=zh_CN.UTF-8 fc-match 
FZSongTi.ttf: "FZSongTi" "Regular"
mfabian@magellan:~$ LC_ALL=zh_TW.UTF-8 fc-match 
FZMingTiB.ttf: "FZMingTiB" "Regular"
mfabian@magellan:~$ fc-list | grep 
FZKaiTi,:style=Regular
mfabian@magellan:~$
> Regards
> James Su
>
> On 12/28/05, Mike FABIAN <mfabian@suse.de> wrote:
>> Zhe Su <james.su@gmail.com> :
>>
>> > Hi,
>> >   After upgrading to fontconfig 2.3.93, the behaviour of  fc-match
>> > becomes very strange. First, fonts can''t be matched by
their localized
>> > names anymore. It''s ok when using fontconfig 2.3.91 or
earlier.
>> > Second, fc-match gives a different default font than the one I set
in
>> > config file.
>> >   I think these issues may be caused by the updating of fcmatch.c
>> > between 2.3.91 and 2.3.93. Do you have any idea about it?
>> >
>> >   I tested it on SUSE Linux 10.0 with latest fontconfig installed.
I
>> > have four Simplified Chinese fonts installed:
>> >
>> > FZHeiTi,:style=Regular
>> > FZFangSong,:style=Regular
>> > FZKaiTi,:style=Regular
>> > FZSongTi,:style=Regular
>> >
>> >   Run fc-match without any parameter will return:
>> > FZFangSong.ttf: "FZFangSong" "Regular"
>> >
>> >   However, it should return FZSongTi according to my config file.
>> >
>> >   And I can only match these fonts with their latin name, like:
>> >
>> > $ fc-match FZSongTi
>> > FZSongTi.ttf: "FZSongTi" "Regular"
>> >
>> >   Matching with localized name will give wrong result:
>> >
>> > $ fc-match 
>> > FZFangSong.ttf: "FZFangSong" "Regular"
>>
>> I cannot reproduce this with the fontconfig-2.3.93.20051222 (CVS
>> checkout of fontconfig 2.4 branch from December 22th 2005) packages I
>> built for SuSE 10.0 (at ftp://ftp.suse.com/pub/projects/m17n/10.0):
>>
>> mfabian@magellan:~$ LANG=zh_CN.UTF-8 fc-match
>> FZSongTi.ttf: "FZSongTi" "Regular"
>> mfabian@magellan:~$ LC_ALL=zh_CN.UTF-8 fc-match
>> FZSongTi.ttf: "FZSongTi" "Regular"
>> mfabian@magellan:~$ LC_ALL=zh_CN.UTF-8 fc-match FZSongTi
>> FZSongTi.ttf: "FZSongTi" "Regular"
>> mfabian@magellan:~$ LC_ALL=zh_CN.UTF-8 fc-match 
>> FZSongTi.ttf: "" "Regular"
>> mfabian@magellan:~$ rpm -q fontconfig
>> fontconfig-2.3.93.20051222-0.1
>> mfabian@magellan:~$
>>
>> --
>> Mike FABIAN   <mfabian@suse.de>   http://www.suse.de/~mfabian
>> 
>>
>>
-- 
Mike FABIAN   <mfabian@suse.de>   http://www.suse.de/~mfabian

Hi,
  The attachment is an updated patch to fix this issue and enhance the
performance furthermore. And a potential bug of multiple FontSets
handling should be fixed.
  Please review and commit.

Regards
James Su

On 12/28/05, Mike FABIAN <mfabian@suse.de> wrote:
> Zhe Su <james.su@gmail.com> ????????:
>
> > Hi,
> >   Try match ???? :-)
>
> Yes, this doesn''t work for me either:
>
> mfabian@magellan:~$ LC_ALL=zh_CN.UTF-8 fc-match ????
> FZSongTi.ttf: "FZSongTi" "Regular"
> mfabian@magellan:~$ LC_ALL=zh_TW.UTF-8 fc-match ????
> FZMingTiB.ttf: "FZMingTiB" "Regular"
> mfabian@magellan:~$ fc-list | grep ????
> FZKaiTi,????:style=Regular
> mfabian@magellan:~$
>
> > Regards
> > James Su
> >
> > On 12/28/05, Mike FABIAN <mfabian@suse.de> wrote:
> >> Zhe Su <james.su@gmail.com> ?????????:
> >>
> >> > Hi,
> >> >   After upgrading to fontconfig 2.3.93, the behaviour of 
fc-match
> >> > becomes very strange. First, fonts can''t be matched
by their localized
> >> > names anymore. It''s ok when using fontconfig 2.3.91
or earlier.
> >> > Second, fc-match gives a different default font than the one
I set in
> >> > config file.
> >> >   I think these issues may be caused by the updating of
fcmatch.c
> >> > between 2.3.91 and 2.3.93. Do you have any idea about it?
> >> >
> >> >   I tested it on SUSE Linux 10.0 with latest fontconfig
installed. I
> >> > have four Simplified Chinese fonts installed:
> >> >
> >> > FZHeiTi,????:style=Regular
> >> > FZFangSong,????:style=Regular
> >> > FZKaiTi,????:style=Regular
> >> > FZSongTi,????:style=Regular
> >> >
> >> >   Run fc-match without any parameter will return:
> >> > FZFangSong.ttf: "FZFangSong" "Regular"
> >> >
> >> >   However, it should return FZSongTi according to my config
file.
> >> >
> >> >   And I can only match these fonts with their latin name,
like:
> >> >
> >> > $ fc-match FZSongTi
> >> > FZSongTi.ttf: "FZSongTi" "Regular"
> >> >
> >> >   Matching with localized name will give wrong result:
> >> >
> >> > $ fc-match ????
> >> > FZFangSong.ttf: "FZFangSong" "Regular"
> >>
> >> I cannot reproduce this with the fontconfig-2.3.93.20051222 (CVS
> >> checkout of fontconfig 2.4 branch from December 22th 2005)
packages I
> >> built for SuSE 10.0 (at
ftp://ftp.suse.com/pub/projects/m17n/10.0):
> >>
> >> mfabian@magellan:~$ LANG=zh_CN.UTF-8 fc-match
> >> FZSongTi.ttf: "FZSongTi" "Regular"
> >> mfabian@magellan:~$ LC_ALL=zh_CN.UTF-8 fc-match
> >> FZSongTi.ttf: "FZSongTi" "Regular"
> >> mfabian@magellan:~$ LC_ALL=zh_CN.UTF-8 fc-match FZSongTi
> >> FZSongTi.ttf: "FZSongTi" "Regular"
> >> mfabian@magellan:~$ LC_ALL=zh_CN.UTF-8 fc-match ????
> >> FZSongTi.ttf: "????" "Regular"
> >> mfabian@magellan:~$ rpm -q fontconfig
> >> fontconfig-2.3.93.20051222-0.1
> >> mfabian@magellan:~$
> >>
> >> --
> >> Mike FABIAN   <mfabian@suse.de>  
http://www.suse.de/~mfabian
> >> ??????????"???
> >>
> >>
>
> --
> Mike FABIAN   <mfabian@suse.de>   http://www.suse.de/~mfabian
> ?????????????
>
>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: fcmatch-2.3.93.patch
Type: text/x-patch
Size: 9385 bytes
Desc: not available
Url :
http://lists.freedesktop.org/archives/fontconfig/attachments/20051228/547fb7b8/fcmatch-2.3.93.bin

Quoting Zhe Su <james.su@gmail.com>:
> Hi,
>  The attachment is an updated patch to fix this issue and enhance the
> performance furthermore. And a potential bug of multiple FontSets
> handling should be fixed.
>  Please review and commit.
Hi James,

I won''t be able to commit the patch until at least the 31st, since I
don''t
currently have access to a suitable computer.  I''ll try to do it then.

pat

Hi,
  That''s alright. And please excuse me that there is a little bug in
that patch. A new patch is attached.

Regards
James Su

On 12/28/05, Patrick Lam <plam@mit.edu> wrote:
> Quoting Zhe Su <james.su@gmail.com>:
>
> > Hi,
> >  The attachment is an updated patch to fix this issue and enhance the
> > performance furthermore. And a potential bug of multiple FontSets
> > handling should be fixed.
> >  Please review and commit.
>
> Hi James,
>
> I won''t be able to commit the patch until at least the 31st, since
I don''t
> currently have access to a suitable computer.  I''ll try to do it
then.
>
> pat
>
> _______________________________________________
> Fontconfig mailing list
> Fontconfig@lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/fontconfig
>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: fcmatch-2.3.93-take2.patch
Type: text/x-patch
Size: 9440 bytes
Desc: not available
Url :
http://lists.freedesktop.org/archives/fontconfig/attachments/20051228/74b404e8/fcmatch-2.3.93-take2-0001.bin

Zhe Su <james.su@gmail.com> ????????:
> Hi,
>   That''s alright. And please excuse me that there is a little bug
in
> that patch. A new patch is attached.
I have tried the patch and verified that it solves the problem
reported by James. RPM packages for SuSE 10.0 are here:

    ftp://ftp.suse.com/pub/projetcs/m17n/10.0

I have not yet done any benchmarks, therefore I don''t know what the
patch does to the performance.
> Regards
> James Su
>
> On 12/28/05, Patrick Lam <plam@mit.edu> wrote:
>> Quoting Zhe Su <james.su@gmail.com>:
>>
>> > Hi,
>> >  The attachment is an updated patch to fix this issue and enhance
the
>> > performance furthermore. And a potential bug of multiple FontSets
>> > handling should be fixed.
>> >  Please review and commit.
>>
>> Hi James,
>>
>> I won''t be able to commit the patch until at least the 31st,
since I don''t
>> currently have access to a suitable computer.  I''ll try to do
it then.
>>
>> pat
>>
>> _______________________________________________
>> Fontconfig mailing list
>> Fontconfig@lists.freedesktop.org
>> http://lists.freedesktop.org/mailman/listinfo/fontconfig
>>
>
> _______________________________________________
> Fontconfig mailing list
> Fontconfig@lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/fontconfig
-- 
Mike FABIAN   <mfabian@suse.de>   http://www.suse.de/~mfabian
?????????????

Zhe Su wrote:
> Hi,
>   That''s alright. And please excuse me that there is a little bug
in
> that patch. A new patch is attached.
I''ve managed to download this patch onto my computer and will look at
it
soon.

pat

Zhe Su wrote:
> Hi,
>   That''s alright. And please excuse me that there is a little bug
in
> that patch. A new patch is attached.
I''ve committed this patch.  I hope it''s alright; I
couldn''t see anything
wrong with it.  If I understand correctly, the crux of the correctness
issue is the change of

	score += v2_best_score;

to

	v = score;

(which would be score = v2_best_score in the old code).

and the rest is a performance optimization.  Is that right?  Could
someone verify that this change alone fixes the bug?

pat

Patrick Lam <plam@MIT.EDU> ????????:
> I''ve committed this patch.  I hope it''s alright; I
couldn''t see anything
> wrong with it.  If I understand correctly, the crux of the correctness
> issue is the change of
>
> 	score += v2_best_score;
>
> to
>
> 	v = score;
>
> (which would be score = v2_best_score in the old code).
>
> and the rest is a performance optimization.  Is that right?  Could
> someone verify that this change alone fixes the bug?
I cannot find exactly the two lines you mention above.

There is a similar part in Zhe Su''s patch though which I have attached
here.

Applying *only* this part does *not* solve the problem, actually
it makes it worse:

mfabian@magellan:/var/tmp/abuild/x86_64$ LC_ALL=zh_CN.UTF-8 fc-match ????
ethio24f-uni.bdf: "ethio24f-uni" "Regular"
mfabian@magellan:/var/tmp/abuild/x86_64$ LC_ALL=zh_CN.UTF-8 fc-match FZSongTi
ethio24f-uni.bdf: "ethio24f-uni" "Regular"
mfabian@magellan:/var/tmp/abuild/x86_64$ LANG=ja_JP.UTF-8 fc-match sans
ethio24f-uni.bdf: "ethio24f-uni" "Regular"

Zhe Su''s complete patch does really solve the problem though.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: ttt.patch
Type: text/x-patch
Size: 560 bytes
Desc: not available
Url :
http://lists.freedesktop.org/archives/fontconfig/attachments/20060109/19cda094/ttt.bin
-------------- next part --------------

-- 
Mike FABIAN   <mfabian@suse.de>   http://www.suse.de/~mfabian
?????????????

Mike FABIAN wrote:
> I cannot find exactly the two lines you mention above.
> 
> There is a similar part in Zhe Su''s patch though which I have
attached
> here.
> 
> Applying *only* this part does *not* solve the problem, actually
> it makes it worse:
> 
> mfabian@magellan:/var/tmp/abuild/x86_64$ LC_ALL=zh_CN.UTF-8 fc-match ????
> ethio24f-uni.bdf: "ethio24f-uni" "Regular"
> mfabian@magellan:/var/tmp/abuild/x86_64$ LC_ALL=zh_CN.UTF-8 fc-match
FZSongTi
> ethio24f-uni.bdf: "ethio24f-uni" "Regular"
> mfabian@magellan:/var/tmp/abuild/x86_64$ LANG=ja_JP.UTF-8 fc-match sans
> ethio24f-uni.bdf: "ethio24f-uni" "Regular"
> 
> Zhe Su''s complete patch does really solve the problem though.
Hmm, weird.

I guess I''ll leave it in unless someone complains... I
couldn''t spot
anything wrong with the patch.

pat