Mike FABIAN
2005-Nov-21 08:51 UTC
[Fontconfig] severe performance regression when global fonts.cache-2 files are missing
Tested with a CVS snapshot of the fontconfig 2.4 branch from yesterday. I noticed that if the global fonts.cache-2 files are out of date, applications using fontconfig take a long time to start. I understand that this happens once because the user cache file ~/.fonts.cache-2 has to be generated. But if this file has been generated, subsequently called applications should start fast. With the current fontconfig, the slowness doesn''t go away. To reduce the problem to a simple test case, I deleted all global fonts.cache-2 files and then tried the following: mfabian@magellan:~$ time fc-list "Luxi Sans" ; ls -l --time-style=full-iso .fonts.cache-2 Luxi Sans:style=Bold Luxi Sans:style=Regular Luxi Sans:style=Oblique Luxi Sans:style=Bold Oblique real 2m18.368s user 0m52.559s sys 0m2.780s -rw------- 1 mfabian suse 4515026 2005-10-20 18:23:33.000000000 +0200 .fonts.cache-2 mfabian@magellan:~$ time fc-list "Luxi Sans" ; ls -l --time-style=full-iso .fonts.cache-2 Luxi Sans:style=Bold Luxi Sans:style=Regular Luxi Sans:style=Oblique Luxi Sans:style=Bold Oblique real 0m5.154s user 0m1.200s sys 0m0.208s -rw------- 1 mfabian suse 45890 2005-10-20 18:25:00.000000000 +0200 .fonts.cache-2 mfabian@magellan:~$ time fc-list "Luxi Sans" ; ls -l --time-style=full-iso .fonts.cache-2 Luxi Sans:style=Bold Luxi Sans:style=Regular Luxi Sans:style=Oblique Luxi Sans:style=Bold Oblique real 2m13.954s user 0m53.459s sys 0m3.244s -rw------- 1 mfabian suse 4515026 2005-10-20 18:27:38.000000000 +0200 .fonts.cache-2 mfabian@magellan:~$ time fc-list "Luxi Sans" ; ls -l --time-style=full-iso .fonts.cache-2 Luxi Sans:style=Bold Luxi Sans:style=Regular Luxi Sans:style=Oblique Luxi Sans:style=Bold Oblique real 0m4.074s user 0m1.220s sys 0m0.196s -rw------- 1 mfabian suse 45890 2005-10-20 18:28:30.000000000 +0200 .fonts.cache-2 mfabian@magellan:~$ You see that the first invocation of "fc-list" is slow and generates a 4.5 MB ~/.fonts.cache-2 file. The next invocation of "fc-list" is fast but surprisingly changes the ~/.fonts.cache-2 again and reduces it to a size of only 45 KB. This cannot be correct. The next invocation of "fc-list" is slow again and regenerates the 4.5 MB ~/.fonts.cache-2 file again. I.e. every second invocation of "fc-list" is fast but apparently destroys the ~/.fonts.cache-2 file. -- Mike FABIAN <mfabian@suse.de> http://www.suse.de/~mfabian ?????????????
Patrick Lam
2005-Nov-21 08:51 UTC
[Fontconfig] Re: severe performance regression when global fonts.cache-2 files are missing
Mike FABIAN wrote:> Tested with a CVS snapshot of the fontconfig 2.4 branch from > yesterday. > > I noticed that if the global fonts.cache-2 files are out of date, > applications using fontconfig take a long time to start. > > I understand that this happens once because the user cache file > ~/.fonts.cache-2 has to be generated. But if this file has been > generated, subsequently called applications should start fast.I''ve experienced problems like this before but I can''t reliably reproduce it on my system right now. Can you see if it happens if you erase your ~/.fonts.cache-2 and start again from scratch? pat
Mike FABIAN
2005-Nov-21 08:51 UTC
[Fontconfig] Re: severe performance regression when global fonts.cache-2 files are missing
Patrick Lam <plam@MIT.EDU> ????????:> Mike FABIAN wrote: >> Tested with a CVS snapshot of the fontconfig 2.4 branch from >> yesterday. >> >> I noticed that if the global fonts.cache-2 files are out of date, >> applications using fontconfig take a long time to start. >> >> I understand that this happens once because the user cache file >> ~/.fonts.cache-2 has to be generated. But if this file has been >> generated, subsequently called applications should start fast. > > CVS HEAD will now remove ~/.fonts.cache-2 if it was previously invalid.Thank you, it seems to work. When I write junk into ~/.fonts.cache-2 on purpose now, the file seems to be recreated correctly. -- Mike FABIAN <mfabian@suse.de> http://www.suse.de/~mfabian ?????????????
Mike FABIAN
2005-Nov-21 08:51 UTC
[Fontconfig] Re: fontconfig often crashes when there is something wrong with the cache files
Patrick Lam <plam@MIT.EDU> ????????:> I notice that this is occuring on x86_64, which may be somewhat relevant.It happens on i386 as well. Well, I''m not sure whether exactly this crash happens on i386, but we have a fair amount of crashes here on i386 as well. -- Mike FABIAN <mfabian@suse.de> http://www.suse.de/~mfabian ?????????????
Mike FABIAN
2005-Nov-21 08:51 UTC
[Fontconfig] Re: fontconfig often crashes when there is something wrong with the cache files
Patrick Lam <plam@MIT.EDU> ????????:> Mike FABIAN wrote: >> Patrick Lam <plam@MIT.EDU> ????????: >> >> >>>Hmm, let''s put an assert there too. Try this patch (to see if the cache >>>files themselves are corrupted in a simple way) and tell me if you can >>>make it abort. I don''t think that''ll happen; I think the corruption is >>>elsewhere. >> >> >> Yes, with that patch I can make it abort: > > That''s quite interesting. I''ll have to add more general consistency > checks on the cache files, then. > > Are these cross-platform cache files, by any chance? That''ll help for > debugging purposes.How to check whether these are cross platforms cache files? Would it help to give you access to such a cache file? -- Mike FABIAN <mfabian@suse.de> http://www.suse.de/~mfabian ?????????????
Mike FABIAN
2005-Nov-21 08:51 UTC
[Fontconfig] Re: severe performance regression when global fonts.cache-2 files are missing
Patrick Lam <plam@MIT.EDU> ????????:> Mike FABIAN wrote: >> Tested with a CVS snapshot of the fontconfig 2.4 branch from >> yesterday. >> >> I noticed that if the global fonts.cache-2 files are out of date, >> applications using fontconfig take a long time to start. >> >> I understand that this happens once because the user cache file >> ~/.fonts.cache-2 has to be generated. But if this file has been >> generated, subsequently called applications should start fast. > > I''ve experienced problems like this before but I can''t reliably > reproduce it on my system right now. Can you see if it happens if you > erase your ~/.fonts.cache-2 and start again from scratch?After erasing ~/.fonts.cache-2, the problem was gone: mfabian@magellan:~$ time fc-list "Luxi Sans" ; ls -l --time-style=full-iso .fonts.cache-2 Luxi Sans:style=Bold Luxi Sans:style=Regular Luxi Sans:style=Oblique Luxi Sans:style=Bold Oblique real 2m17.041s user 0m50.395s sys 0m2.880s -rw------- 1 mfabian suse 4555986 2005-10-20 23:43:55.000000000 +0200 .fonts.cache-2 mfabian@magellan:~$ time fc-list "Luxi Sans" ; ls -l --time-style=full-iso .fonts.cache-2 Luxi Sans:style=Bold Luxi Sans:style=Regular Luxi Sans:style=Oblique Luxi Sans:style=Bold Oblique real 0m4.089s user 0m1.096s sys 0m0.168s -rw------- 1 mfabian suse 4555986 2005-10-20 23:43:55.000000000 +0200 .fonts.cache-2 mfabian@magellan:~$ time fc-list "Luxi Sans" ; ls -l --time-style=full-iso .fonts.cache-2 Luxi Sans:style=Bold Luxi Sans:style=Regular Luxi Sans:style=Oblique Luxi Sans:style=Bold Oblique real 0m1.519s user 0m1.108s sys 0m0.128s -rw------- 1 mfabian suse 4555986 2005-10-20 23:43:55.000000000 +0200 .fonts.cache-2 mfabian@magellan:~$ time fc-list "Luxi Sans" ; ls -l --time-style=full-iso .fonts.cache-2 Luxi Sans:style=Bold Luxi Sans:style=Regular Luxi Sans:style=Oblique Luxi Sans:style=Bold Oblique real 0m1.284s user 0m1.140s sys 0m0.108s -rw------- 1 mfabian suse 4555986 2005-10-20 23:43:55.000000000 +0200 .fonts.cache-2 mfabian@magellan:~$ This is very strange, certainly there is a bug but it appears to be not easy to reproduce. -- Mike FABIAN <mfabian@suse.de> http://www.suse.de/~mfabian ?????????????
Patrick Lam
2005-Nov-21 08:51 UTC
[Fontconfig] Re: fontconfig often crashes when there is something wrong with the cache files
Mike FABIAN wrote:> For details see: > > http://bugzilla.novell.com/show_bug.cgi?id=128080 > > This bugreport also has a backtrace attached (comment #10). > > Until now I don''t know a easy way to reproduce the problem, it occurs > "sometimes" and goes away by calling "fc-cache -f" as root. > I.e. I guess there is something wrong with the cache files. > > I have started debugging this but until now I only found that > it crashes in fccharset.c in > > FcCharSet * > FcCharSetCopy (FcCharSet *src) > { > if (src->ref != FC_REF_CONSTANT) > src->ref++; > return src; > } > > > when src->ref happens to be 0 (which is != FC_REF_CONSTANT) > and then tries to execute > > src->ref++; > > which won''t work because "src" is v.u.c where v is of type FcValue and > FcCharSet is "const" in that union (fontconfig.h), therefore it cannot > be incremented:The constness is not a problem, since that''s cast away. However, the problem might be that the charset is in mmapped space and therefore ref can''t be incremented. But that''s strange, because mmapped charsets should always have ref set to -1. Also, the backtrace seems to point to line 359, which is the statement just after the increment. However, v.u.c just seems to be not an address (''address 0x2aaaad90c4ef out of bounds'') I notice that this is occuring on x86_64, which may be somewhat relevant. If you can reproduce this problem, there are two pieces of information which would be useful. 1) step up to fclist.c:431 and print v there, before canonicalization. 2) also print charsets[0]. Hmm, let''s put an assert there too. Try this patch (to see if the cache files themselves are corrupted in a simple way) and tell me if you can make it abort. I don''t think that''ll happen; I think the corruption is elsewhere. diff -u -r1.25.4.4 fccharset.c --- src/fccharset.c 22 Sep 2005 23:45:53 -0000 1.25.4.4 +++ src/fccharset.c 25 Oct 2005 14:46:42 -0000 @@ -1406,6 +1406,7 @@ FcCharSetUnserialize (FcCache metadata, void *block_ptr) { int bi = FcCacheBankToIndex(metadata.bank); + int i; if (!FcCharSetEnsureBank(bi)) return 0; @@ -1422,6 +1423,10 @@ block_ptr = (void *)((char *)block_ptr + (sizeof(int) * metadata.charset_leaf_idx_count)); + for (i = 0; i < metadata.charset_count; i++) + if (charsets[bi][i].ref != FC_REF_CONSTANT) + abort(); + return block_ptr; } pat
Patrick Lam
2005-Nov-21 08:51 UTC
[Fontconfig] Re: severe performance regression when global fonts.cache-2 files are missing
Mike FABIAN wrote:> Tested with a CVS snapshot of the fontconfig 2.4 branch from > yesterday. > > I noticed that if the global fonts.cache-2 files are out of date, > applications using fontconfig take a long time to start. > > I understand that this happens once because the user cache file > ~/.fonts.cache-2 has to be generated. But if this file has been > generated, subsequently called applications should start fast.CVS HEAD will now remove ~/.fonts.cache-2 if it was previously invalid. pat
Patrick Lam
2005-Nov-21 08:51 UTC
[Fontconfig] Re: fontconfig often crashes when there is something wrong with the cache files
Mike FABIAN wrote:> Patrick Lam <plam@MIT.EDU> ????????: > > >>Hmm, let''s put an assert there too. Try this patch (to see if the cache >>files themselves are corrupted in a simple way) and tell me if you can >>make it abort. I don''t think that''ll happen; I think the corruption is >>elsewhere. > > > Yes, with that patch I can make it abort:That''s quite interesting. I''ll have to add more general consistency checks on the cache files, then. Are these cross-platform cache files, by any chance? That''ll help for debugging purposes. pat
Mike FABIAN
2005-Nov-21 08:51 UTC
[Fontconfig] fontconfig often crashes when there is something wrong with the cache files
For details see:
http://bugzilla.novell.com/show_bug.cgi?id=128080
This bugreport also has a backtrace attached (comment #10).
Until now I don''t know a easy way to reproduce the problem, it occurs
"sometimes" and goes away by calling "fc-cache -f" as root.
I.e. I guess there is something wrong with the cache files.
I have started debugging this but until now I only found that
it crashes in fccharset.c in
FcCharSet *
FcCharSetCopy (FcCharSet *src)
{
if (src->ref != FC_REF_CONSTANT)
src->ref++;
return src;
}
when src->ref happens to be 0 (which is != FC_REF_CONSTANT)
and then tries to execute
src->ref++;
which won''t work because "src" is v.u.c where v is of type
FcValue and
FcCharSet is "const" in that union (fontconfig.h), therefore it cannot
be incremented:
typedef struct _FcValue {
FcType type;
union {
const FcChar8 *s;
int s_off;
int i;
FcBool b;
double d;
const FcMatrix *m;
const FcCharSet *c;
int c_off;
void *f;
const FcPattern *p;
const FcLangSet *l;
int l_off; // this is a difference of char *s
} u;
} FcValue;
--
Mike FABIAN <mfabian@suse.de> http://www.suse.de/~mfabian
?????????????
Mike FABIAN
2005-Nov-21 08:51 UTC
[Fontconfig] Re: fontconfig often crashes when there is something wrong with the cache files
Patrick Lam <plam@MIT.EDU> ????????:> Hmm, let''s put an assert there too. Try this patch (to see if the cache > files themselves are corrupted in a simple way) and tell me if you can > make it abort. I don''t think that''ll happen; I think the corruption is > elsewhere.Yes, with that patch I can make it abort: mfabian@magellan:~$ xfd -fa sans calling abort() ???????? (core dumped) mfabian@magellan:~$ /usr/lib/qt3/bin/qtconfig calling abort() ???????? (core dumped) mfabian@magellan:~$> diff -u -r1.25.4.4 fccharset.c > --- src/fccharset.c 22 Sep 2005 23:45:53 -0000 1.25.4.4 > +++ src/fccharset.c 25 Oct 2005 14:46:42 -0000 > @@ -1406,6 +1406,7 @@ > FcCharSetUnserialize (FcCache metadata, void *block_ptr) > { > int bi = FcCacheBankToIndex(metadata.bank); > + int i; > if (!FcCharSetEnsureBank(bi)) > return 0; > > @@ -1422,6 +1423,10 @@ > block_ptr = (void *)((char *)block_ptr + > (sizeof(int) * metadata.charset_leaf_idx_count)); > > + for (i = 0; i < metadata.charset_count; i++) > + if (charsets[bi][i].ref != FC_REF_CONSTANT) > + abort(); > + > return block_ptr; > } > > pat >-- Mike FABIAN <mfabian@suse.de> http://www.suse.de/~mfabian ?????????????
Mike FABIAN
2005-Nov-21 08:51 UTC
[Fontconfig] Re: fontconfig often crashes when there is something wrong with the cache files
Patrick Lam <plam@MIT.EDU> ????????:> Mike FABIAN wrote: >> For details see: >> >> http://bugzilla.novell.com/show_bug.cgi?id=128080 >> >> This bugreport also has a backtrace attached (comment #10). >> >> Until now I don''t know a easy way to reproduce the problem, it occurs >> "sometimes" and goes away by calling "fc-cache -f" as root. >> I.e. I guess there is something wrong with the cache files. >> >> I have started debugging this but until now I only found that >> it crashes in fccharset.c in >> >> FcCharSet * >> FcCharSetCopy (FcCharSet *src) >> { >> if (src->ref != FC_REF_CONSTANT) >> src->ref++; >> return src; >> } >> >> >> when src->ref happens to be 0 (which is != FC_REF_CONSTANT) >> and then tries to execute >> >> src->ref++; >> >> which won''t work because "src" is v.u.c where v is of type FcValue and >> FcCharSet is "const" in that union (fontconfig.h), therefore it cannot >> be incremented: > > The constness is not a problem, since that''s cast away. However, the > problem might be that the charset is in mmapped space and therefore ref > can''t be incremented. But that''s strange, because mmapped charsets > should always have ref set to -1. Also, the backtrace seems to point to > line 359, which is the statement just after the increment.The shift of one line is probably because I had added a printf for debugging somewhere. When single stepping with gdb, it crashed exactly on the src->ref++; line. And when that happened, ref was equal to 0. -- Mike FABIAN <mfabian@suse.de> http://www.suse.de/~mfabian ?????????????
Mike FABIAN
2005-Nov-21 08:51 UTC
[Fontconfig] Re: severe performance regression when global fonts.cache-2 files are missing
Mike FABIAN <mfabian@suse.de> ????????:> This is very strange, certainly there is a bug but it appears to be > not easy to reproduce.I found that I can easily by writing some random junk into ~/.fonts.cache-2. Like this: mfabian@magellan:~$ echo "hello" > .fonts.cache-2 mfabian@magellan:~$ time fc-list "Luxi Sans" ; ls -l --time-style=full-iso .fonts.cache-2 Luxi Sans:style=Bold Luxi Sans:style=Regular Luxi Sans:style=Oblique Luxi Sans:style=Bold Oblique real 2m21.643s user 0m50.751s sys 0m3.060s -rw------- 1 mfabian suse 4564178 2005-10-20 23:51:02.000000000 +0200 .fonts.cache-2 mfabian@magellan:~$ time fc-list "Luxi Sans" ; ls -l --time-style=full-iso .fonts.cache-2 Luxi Sans:style=Bold Luxi Sans:style=Regular Luxi Sans:style=Oblique Luxi Sans:style=Bold Oblique real 2m16.382s user 0m50.819s sys 0m2.968s -rw------- 1 mfabian suse 9127122 2005-10-20 23:53:52.000000000 +0200 .fonts.cache-2 mfabian@magellan:~$ time fc-list "Luxi Sans" ; ls -l --time-style=full-iso .fonts.cache-2 Luxi Sans:style=Bold Luxi Sans:style=Regular Luxi Sans:style=Oblique Luxi Sans:style=Bold Oblique real 2m19.644s user 0m50.855s sys 0m3.008s -rw------- 1 mfabian suse 13690066 2005-10-21 00:29:06.000000000 +0200 .fonts.cache-2 mfabian@magellan:~$ time fc-list "Luxi Sans" ; ls -l --time-style=full-iso .fonts.cache-2 Luxi Sans:style=Bold Luxi Sans:style=Regular Luxi Sans:style=Oblique Luxi Sans:style=Bold Oblique real 2m23.269s user 0m50.883s sys 0m3.060s -rw------- 1 mfabian suse 18253010 2005-10-21 00:57:03.000000000 +0200 .fonts.cache-2 mfabian@magellan:~$ The first call of "fc-list" should have fixed the broken ~/.fonts.cache-2 file, but apparently this didn''t work, subsequent calls of "fc-list" continue to take a lot of time and the ~/.fonts.cache-2 file grows bigger and bigger with each call. -- Mike FABIAN <mfabian@suse.de> http://www.suse.de/~mfabian ?????????????