thr3ads.net - Fontconfig - [Fontconfig] Patch: fix for fc-cache with relative path [Jan 2006]

If this information is useful, please help other people find it:
Share via:

Frederic Crozat

2006-Jan-06 13:14 UTC

[Fontconfig] Patch: fix for fc-cache with relative path

Hi,

the new cache location code has introduced new bugs in fontconfig :
-when using fc-cache . in a directory, the string used to generate
md5sum key is always "./fonts.cache-2" which will collide if the same
command is run in several directories.
-a similar problem happen when using : fc-cache /foo/bar and
fc-cache /foo/bar/ : strings used to generate md5sum are not the same,
resulting in different cache.

The attached patch fixes the issue.

I''ve also found other problems but I haven''t fixed them (help
welcome) :
-if you have a directory containing fonts.cache-2 file which are not
up-to-date but cache in /var/cache is up to date, running fc-cache will
not remove those fonts.cache-2, even with fc-cache -f. I tried to force
removal of old cache when using -f but doing so would break multiarch
cache (for instance, we run fc-cache -f when upgrading fontconfig
package, to be sure cache is coherent after a fontconfig upgrade).
-in some directories containing only ttf fonts, running fc-cache -v
again and again always regenerate the cache (which is strange, since the
directory was not modified at all).
-this last bug is causing huge delays when starting applications for the
first time after running fc-cache as root, since cache is detected as
dirty and fontconfig regenerates it for the user (and on my test system,
I have two big chinese fonts of 20MB and 16MB ...).

--
Frederic Crozat <fcrozat@mandriva.com>
Mandriva
-------------- next part --------------
A non-text attachment was scrubbed...
Name: fontconfig-2.3.93-realpath.patch
Type: text/x-patch
Size: 920 bytes
Desc: not available
Url :
http://lists.freedesktop.org/archives/fontconfig/attachments/20060104/f8ee0240/fontconfig-2.3.93-realpath.bin

Patrick Lam

2006-Jan-06 22:52 UTC

head link

[Fontconfig] Patch: fix for fc-cache with relative path

On Wed, 4 Jan 2006, Frederic Crozat wrote:
> the new cache location code has introduced new bugs in fontconfig :
> -when using fc-cache . in a directory, the string used to generate
> md5sum key is always "./fonts.cache-2" which will collide if the
same
> command is run in several directories.
> -a similar problem happen when using : fc-cache /foo/bar and
> fc-cache /foo/bar/ : strings used to generate md5sum are not the same,
> resulting in different cache.
I can fix the second problem but not the first.
> The attached patch fixes the issue.
I was looking for something like realpath.  Unfortunately, realpath 
doesn''t quite seem to be what we want, since its manpage says:

BUGS

Avoid using this function. It is broken by design since (unless using the 
non-standard resolved_path == NULL feature)  it is impossible to determine 
a suitable size for the output buffer, resolved_path. According to POSIX a 
buffer of size PATH_MAX suffices, but PATH_MAX need not be a defined 
constant, and may have to be obtained using pathconf().  And asking 
pathconf() does not really help, since on the one hand POSIX warns that 
the result of pathconf() may be huge and unsuitable for mallocing memory. 
And on the other hand pathconf() may return -1 to signify that PATH_MAX is 
not bounded.

pat

Patrick Lam

2006-Jan-08 02:59 UTC

head link

[Fontconfig] Patch: fix for fc-cache with relative path

Frederic Crozat wrote:> Hi,
> 
> the new cache location code has introduced new bugs in fontconfig :
> -when using fc-cache . in a directory, the string used to generate
> md5sum key is always "./fonts.cache-2" which will collide if the
same
> command is run in several directories.
> -a similar problem happen when using : fc-cache /foo/bar and
> fc-cache /foo/bar/ : strings used to generate md5sum are not the same,
> resulting in different cache.
I have a better patch for this issue but can''t commit it because my
laptop is about to run out of battery.  Maybe tomorrow.  It uses the
directory paths in the FcConfig data structure to normalize paths.

pat

Patrick Lam

2006-Jan-09 06:04 UTC

head link

[Fontconfig] Patch: fix for fc-cache with relative path

Frederic Crozat wrote:> Hi,
> 
> the new cache location code has introduced new bugs in fontconfig :
> -when using fc-cache . in a directory, the string used to generate
> md5sum key is always "./fonts.cache-2" which will collide if the
same
> command is run in several directories.
> -a similar problem happen when using : fc-cache /foo/bar and
> fc-cache /foo/bar/ : strings used to generate md5sum are not the same,
> resulting in different cache.
> 
> The attached patch fixes the issue.
I''ve committed my patch.
> I''ve also found other problems but I haven''t fixed them
(help welcome) :
> -if you have a directory containing fonts.cache-2 file which are not
> up-to-date but cache in /var/cache is up to date, running fc-cache will
> not remove those fonts.cache-2, even with fc-cache -f. I tried to force
> removal of old cache when using -f but doing so would break multiarch
> cache (for instance, we run fc-cache -f when upgrading fontconfig
> package, to be sure cache is coherent after a fontconfig upgrade).
Yeah, I''m not sure what the correct solution to that problem is. 
I''ll
have to think about it.
> -in some directories containing only ttf fonts, running fc-cache -v
> again and again always regenerate the cache (which is strange, since the
> directory was not modified at all).
> -this last bug is causing huge delays when starting applications for the
> first time after running fc-cache as root, since cache is detected as
> dirty and fontconfig regenerates it for the user (and on my test system,
> I have two big chinese fonts of 20MB  and 16MB ...).
Do you have any more hints on when these problems occur?

pat

Frederic Crozat

2006-Jan-09 06:37 UTC

head link

[Fontconfig] Patch: fix for fc-cache with relative path

Le lundi 09 janvier 2006 ? 21:06 +0700, Patrick Lam a ?crit
:> Frederic Crozat wrote:
> > Hi,
> > 
> > the new cache location code has introduced new bugs in fontconfig :
> > -when using fc-cache . in a directory, the string used to generate
> > md5sum key is always "./fonts.cache-2" which will collide if
the same
> > command is run in several directories.
> > -a similar problem happen when using : fc-cache /foo/bar and
> > fc-cache /foo/bar/ : strings used to generate md5sum are not the same,
> > resulting in different cache.
> > 
> > The attached patch fixes the issue.
> 
> I''ve committed my patch.
Good, I''ll test it later this week. I knew about realpath ugliness but
I
couldn''t figure a good way to replace it (too bad fontconfig
doesn''t use
glib ;)

> > -in some directories containing only ttf fonts, running fc-cache -v
> > again and again always regenerate the cache (which is strange, since
the
> > directory was not modified at all).
> > -this last bug is causing huge delays when starting applications for
the
> > first time after running fc-cache as root, since cache is detected as
> > dirty and fontconfig regenerates it for the user (and on my test
system,
> > I have two big chinese fonts of 20MB  and 16MB ...).
> 
> Do you have any more hints on when these problems occur?
Those two patches are no longer reproducible after removing all files
in /var/cache/fontconfig and re-running fc-cache with my "realpath"
patch. I guess old broken cache where causing this. People which were
reporting similar issues on cooker confirmed it fixed it.

I guess your similar fix will have a similar effect.

-- 
Frederic Crozat <fcrozat@mandriva.com>
Mandriva

Patrick Lam

2006-Jan-10 05:19 UTC

head link

[Fontconfig] Patch: fix for fc-cache with relative path

Frederic Crozat wrote:> Good, I''ll test it later this week. I knew about realpath ugliness
but I
> couldn''t figure a good way to replace it (too bad fontconfig
doesn''t use
> glib ;)
The patch I committed earlier was slightly broken, but I think I''ve
fixed it now.
>>>-in some directories containing only ttf fonts, running fc-cache -v
>>>again and again always regenerate the cache (which is strange, since
the
>>>directory was not modified at all).
>>>-this last bug is causing huge delays when starting applications for
the
>>>first time after running fc-cache as root, since cache is detected
as
>>>dirty and fontconfig regenerates it for the user (and on my test
system,
>>>I have two big chinese fonts of 20MB  and 16MB ...).
>>
>>Do you have any more hints on when these problems occur?
> 
> Those two patches are no longer reproducible after removing all files
> in /var/cache/fontconfig and re-running fc-cache with my
"realpath"
> patch. I guess old broken cache where causing this. People which were
> reporting similar issues on cooker confirmed it fixed it.
> 
> I guess your similar fix will have a similar effect.
Actually, I figured out what caused this: the directory cache contains
the absolute path, but fc-cache . will try to open a cache for path
''.'',
which fails.  This works now.

pat

Mike FABIAN

2006-Jan-12 08:42 UTC

head link

[Fontconfig] Re: Patch: fix for fc-cache with relative path

Patrick Lam <plam@MIT.EDU> ????????:
> Frederic Crozat wrote:
>> Hi,
>> 
>> the new cache location code has introduced new bugs in fontconfig :
>> -when using fc-cache . in a directory, the string used to generate
>> md5sum key is always "./fonts.cache-2" which will collide if
the same
>> command is run in several directories.
Looks like this problem is *not* fixed yet (CVS checkout from today):

    mfabian@magellan:/usr/X11R6/lib/X11/fonts/truetype$ sudo time fc-cache -v .
    fc-cache: ".": caching, 919 fonts, 0 dirs
    fc-cache: succeeded
    35.78user 1.12system 2:06.67elapsed 29%CPU (0avgtext+0avgdata 0maxresident)k
    0inputs+0outputs (18668major+233607minor)pagefaults 0swaps
    mfabian@magellan:/usr/X11R6/lib/X11/fonts/truetype$ ls
/var/cache/fontconfig/
    d6867d8c7d183335b46263581b96fdc1.cache-2  stamp
    mfabian@magellan:/usr/X11R6/lib/X11/fonts/truetype$

one cache file has been created. Try again:

    mfabian@magellan:/usr/X11R6/lib/X11/fonts/truetype$ sudo time fc-cache -v .
    fc-cache: ".": skipping, 919 fonts, 0 dirs
    fc-cache: succeeded
    0.00user 0.00system 0:02.21elapsed 0%CPU (0avgtext+0avgdata 0maxresident)k
    0inputs+0outputs (6major+341minor)pagefaults 0swaps
    mfabian@magellan:/usr/X11R6/lib/X11/fonts/truetype$

OK, skipped now, cache is up to date.

Try to generate a cache for a different directory:

    mfabian@magellan:/usr/X11R6/lib/X11/fonts$ cd ../Type1/
    mfabian@magellan:/usr/X11R6/lib/X11/fonts/Type1$ sudo time fc-cache -v .
    fc-cache: ".": skipping, 919 fonts, 0 dirs
    fc-cache: succeeded
    0.00user 0.00system 0:02.02elapsed 0%CPU (0avgtext+0avgdata
    0maxresident)k
    0inputs+0outputs (0major+348minor)pagefaults 0swaps
    mfabian@magellan:/usr/X11R6/lib/X11/fonts/Type1$

Surprisingly skipped, although no cache for this directory
exists (neither in /var/cache/fontconfig nor in
/usr/X11R6/lib/X11/fonts/Type1).

No new cache file has been generated:

    mfabian@magellan:/usr/X11R6/lib/X11/fonts/Type1$ ls /var/cache/fontconfig/
    d6867d8c7d183335b46263581b96fdc1.cache-2  stamp
    mfabian@magellan:/usr/X11R6/lib/X11/fonts/Type1$ 
>> -a similar problem happen when using : fc-cache /foo/bar and
>> fc-cache /foo/bar/ : strings used to generate md5sum are not the same,
>> resulting in different cache.
>> 
>> The attached patch fixes the issue.
>
> I''ve committed my patch.
Hmm, that problem seems to persist as well:

    mfabian@magellan:/usr/X11R6/lib/X11/fonts/Type1$ sudo time fc-cache -v
/usr/X11R6/lib/X11/fonts/Type1
    fc-cache: "/usr/X11R6/lib/X11/fonts/Type1": caching, 59 fonts, 0
dirs
    fc-cache: succeeded
    0.30user 0.02system 0:02.81elapsed 11%CPU (0avgtext+0avgdata 0maxresident)k
    0inputs+0outputs (198major+1357minor)pagefaults 0swaps
    mfabian@magellan:/usr/X11R6/lib/X11/fonts/Type1$

One new cache file has been generated:

    mfabian@magellan:/usr/X11R6/lib/X11/fonts/Type1$ ls /var/cache/fontconfig/
    57fddf4aeff95a0b98650bf3f13c26fe.cache-2  stamp
    d6867d8c7d183335b46263581b96fdc1.cache-2
    mfabian@magellan:/usr/X11R6/lib/X11/fonts/Type1$

Try again adding a ''/'' at the end of the path:

    mfabian@magellan:/usr/X11R6/lib/X11/fonts/Type1$ sudo time fc-cache -v
/usr/X11R6/lib/X11/fonts/Type1/
    fc-cache: "/usr/X11R6/lib/X11/fonts/Type1/": caching, 59 fonts, 0
dirs
    fc-cache: succeeded
    0.28user 0.01system 0:02.31elapsed 12%CPU (0avgtext+0avgdata 0maxresident)k
    0inputs+0outputs (0major+1550minor)pagefaults 0swaps
    mfabian@magellan:/usr/X11R6/lib/X11/fonts/Type1$

A new cache has been generated:

    mfabian@magellan:/usr/X11R6/lib/X11/fonts/Type1$ ls /var/cache/fontconfig/
    57fddf4aeff95a0b98650bf3f13c26fe.cache-2
    d6867d8c7d183335b46263581b96fdc1.cache-2
    b2b474fcfbd349e17fee496b6c581dbd.cache-2  stamp
    mfabian@magellan:/usr/X11R6/lib/X11/fonts/Type1$

-- 
Mike FABIAN   <mfabian@suse.de>   http://www.suse.de/~mfabian
?????????????

Frederic Crozat

2006-Jan-12 08:45 UTC

head link

[Fontconfig] Re: Patch: fix for fc-cache with relative path

Le jeudi 12 janvier 2006 ? 13:11 +0100, Mike FABIAN a ?crit
:> Mike FABIAN <mfabian@suse.de> ????????:
> 
> > No new cache file has been generated:
> >
> >     mfabian@magellan:/usr/X11R6/lib/X11/fonts/Type1$ ls
/var/cache/fontconfig/
> >     d6867d8c7d183335b46263581b96fdc1.cache-2  stamp
> >     mfabian@magellan:/usr/X11R6/lib/X11/fonts/Type1$ 
> >
> >>> -a similar problem happen when using : fc-cache /foo/bar and
> >>> fc-cache /foo/bar/ : strings used to generate md5sum are not
the same,
> >>> resulting in different cache.
> >>> 
> >>> The attached patch fixes the issue.
> >>
> >> I''ve committed my patch.
> >
> > Hmm, that problem seems to persist as well:
> > [ ... description how to reproduce ...]
> 
> I believe I can fix this with the attached patch:
I can confirm it fixes both issues : fc-cache . and fc-cache /foo/bar
vs /foo/bar/

-- 
Frederic Crozat <fcrozat@mandriva.com>
Mandriva

Mike FABIAN

2006-Jan-12 08:45 UTC

head link

[Fontconfig] Re: Patch: fix for fc-cache with relative path

Mike FABIAN <mfabian@suse.de> ????????:
> No new cache file has been generated:
>
>     mfabian@magellan:/usr/X11R6/lib/X11/fonts/Type1$ ls
/var/cache/fontconfig/
>     d6867d8c7d183335b46263581b96fdc1.cache-2  stamp
>     mfabian@magellan:/usr/X11R6/lib/X11/fonts/Type1$ 
>
>>> -a similar problem happen when using : fc-cache /foo/bar and
>>> fc-cache /foo/bar/ : strings used to generate md5sum are not the
same,
>>> resulting in different cache.
>>> 
>>> The attached patch fixes the issue.
>>
>> I''ve committed my patch.
>
> Hmm, that problem seems to persist as well:
> [ ... description how to reproduce ...]
I believe I can fix this with the attached patch:

-------------- next part --------------
A non-text attachment was scrubbed...
Name: normalize-path-in-fc-cache.patch
Type: text/x-patch
Size: 698 bytes
Desc: not available
Url :
http://lists.freedesktop.org/archives/fontconfig/attachments/20060112/c4c16f88/normalize-path-in-fc-cache.bin
-------------- next part --------------

-- 
Mike FABIAN   <mfabian@suse.de>   http://www.suse.de/~mfabian
?????????????

Mike FABIAN

2006-Jan-12 09:11 UTC

head link

[Fontconfig] Re: Patch: fix for fc-cache with relative path

Mike FABIAN <mfabian@suse.de> ????????:
> Mike FABIAN <mfabian@suse.de> ????????:
>
>> No new cache file has been generated:
>>
>>     mfabian@magellan:/usr/X11R6/lib/X11/fonts/Type1$ ls
/var/cache/fontconfig/
>>     d6867d8c7d183335b46263581b96fdc1.cache-2  stamp
>>     mfabian@magellan:/usr/X11R6/lib/X11/fonts/Type1$ 
>>
>>>> -a similar problem happen when using : fc-cache /foo/bar and
>>>> fc-cache /foo/bar/ : strings used to generate md5sum are not
the same,
>>>> resulting in different cache.
>>>> 
>>>> The attached patch fixes the issue.
>>>
>>> I''ve committed my patch.
>>
>> Hmm, that problem seems to persist as well:
>> [ ... description how to reproduce ...]
>
> I believe I can fix this with the attached patch:
The last patch was buggy because it used FcConfigNormalizeFontDir()
but failed to include fcint.h.

I have fixed this (patch attached).

*But*, I found that my patch has the following side effect: 

Because I replaced FcInitLoadConfig () by FcInitLoadConfigAndFonts (),
"fc-cache /foo/bar" (called as root) may generate /root/.fonts.cache-2
first before creating the cache in /var/cache/fontconfig for the
requested directory:

    root@magellan:~# rm .fonts.cache-2
    root@magellan:~# rm /var/cache/fontconfig/*cache-2
    root@magellan:~# time fc-cache -v /usr/X11R6/lib/X11/fonts/Type1
    fc-cache: "/usr/X11R6/lib/X11/fonts/Type1": caching, 59 fonts, 0
dirs
    fc-cache: succeeded

    real    2m33.714s
    user    0m48.471s
    sys     0m2.532s
    root@magellan:~# ls -l --time-style=full-iso /var/cache/fontconfig/*cache-2
    -rw-r--r-- 1 root root 81920 2006-01-12 17:28:35.000000000 +0100
/var/cache/fontconfig/57fddf4aeff95a0b98650bf3f13c26fe.cache-2
    root@magellan:~# ls -l --time-style=full-iso .fonts.cache-2
    -rw------- 1 root root 4698113 2006-01-12 17:28:33.000000000 +0100
.fonts.cache-2
    root@magellan:~# 

This may cause "fc-cache" to be very slow, if neither
/root/.fonts.cache-2 nor any cache in /var/cache/fontconfig exist,
"fc-cache" will first cache *all* fonts in /root/.fonts.cache-2 (which
takes a long time) and then generate the requested cache in
/var/cache/fontconfig.

I think this problem is caused because I used FcInitLoadConfigAndFonts ().

But without calling this function, FcConfigNormalizeFontDir() cannot
work right because after calling FcInitLoadConfig () the list of font
directories in the "config" structure is only the list of directories
configured in the *.conf files. Their subdirectories are still missing
in the list, they are only added when calling FcInitLoadConfigAndFonts
().

-------------- next part --------------
A non-text attachment was scrubbed...
Name: normalize-path-in-fc-cache.patch
Type: text/x-patch
Size: 1414 bytes
Desc: not available
Url :
http://lists.freedesktop.org/archives/fontconfig/attachments/20060112/7d31a1c9/normalize-path-in-fc-cache.bin
-------------- next part --------------

-- 
Mike FABIAN   <mfabian@suse.de>   http://www.suse.de/~mfabian
?????????????

Patrick Lam

2006-Jan-13 05:52 UTC

head link

[Fontconfig] Re: Patch: fix for fc-cache with relative path

That looks like it''s probably correct.  I''ll look at the
various patches
floating around tomorrow, I think...

pat


On Thu, 12 Jan 2006, Mike FABIAN wrote:
> Mike FABIAN <mfabian@suse.de> ¤µ¤ó¤Ï½ñ¤¤Þ¤·¤¿:
>
>> No new cache file has been generated:
>>
>>     mfabian@magellan:/usr/X11R6/lib/X11/fonts/Type1$ ls
/var/cache/fontconfig/
>>     d6867d8c7d183335b46263581b96fdc1.cache-2  stamp
>>     mfabian@magellan:/usr/X11R6/lib/X11/fonts/Type1$
>>
>>>> -a similar problem happen when using : fc-cache /foo/bar and
>>>> fc-cache /foo/bar/ : strings used to generate md5sum are not
the same,
>>>> resulting in different cache.
>>>>
>>>> The attached patch fixes the issue.
>>>
>>> I''ve committed my patch.
>>
>> Hmm, that problem seems to persist as well:
>> [ ... description how to reproduce ...]
>
> I believe I can fix this with the attached patch:
>
>

Mike FABIAN

2006-Jan-13 06:01 UTC

head link

[Fontconfig] Re: Patch: fix for fc-cache with relative path

Patrick Lam <plam@MIT.EDU> ????????:
> That looks like it''s probably correct.  I''ll look at the
various
> patches floating around tomorrow, I think...
Probably correct. But don''t forget my second mail where I noticed
that this patch has the flaw of making the first run of fc-cache
*very* slow because /root/.fonts.cache-2 is generated first
and then the cache files in /var/cache/fontconfig.

My only idea at the moment is to add a parameter to the
FcInitLoadConfigAndFonts() function call which can be true or false
and if it is false no caches are generated, only the list of font
directories is updated.

Or something like this.

Unfortunately I am very busy with other stuff now and cannot think
about this problem now.
> On Thu, 12 Jan 2006, Mike FABIAN wrote:
>
>> Mike FABIAN <mfabian@suse.de> ????????:
>>
>>> No new cache file has been generated:
>>>
>>>     mfabian@magellan:/usr/X11R6/lib/X11/fonts/Type1$ ls
/var/cache/fontconfig/
>>>     d6867d8c7d183335b46263581b96fdc1.cache-2  stamp
>>>     mfabian@magellan:/usr/X11R6/lib/X11/fonts/Type1$
>>>
>>>>> -a similar problem happen when using : fc-cache /foo/bar
and
>>>>> fc-cache /foo/bar/ : strings used to generate md5sum are
not the same,
>>>>> resulting in different cache.
>>>>>
>>>>> The attached patch fixes the issue.
>>>>
>>>> I''ve committed my patch.
>>>
>>> Hmm, that problem seems to persist as well:
>>> [ ... description how to reproduce ...]
>>
>> I believe I can fix this with the attached patch:
>>
>>
-- 
Mike FABIAN   <mfabian@suse.de>   http://www.suse.de/~mfabian
?????????????

Fontconfig - Jan 2006 - Patch: fix for fc-cache with relative path

[Fontconfig] Patch: fix for fc-cache with relative path

[Fontconfig] Patch: fix for fc-cache with relative path

[Fontconfig] Patch: fix for fc-cache with relative path

[Fontconfig] Patch: fix for fc-cache with relative path

[Fontconfig] Patch: fix for fc-cache with relative path

[Fontconfig] Patch: fix for fc-cache with relative path

[Fontconfig] Re: Patch: fix for fc-cache with relative path

[Fontconfig] Re: Patch: fix for fc-cache with relative path

[Fontconfig] Re: Patch: fix for fc-cache with relative path

[Fontconfig] Re: Patch: fix for fc-cache with relative path

[Fontconfig] Re: Patch: fix for fc-cache with relative path

[Fontconfig] Re: Patch: fix for fc-cache with relative path