Lubos Lunak wrote:> However, when I tested the 2.3.90 release yesterday, I was originally > disappointed to see it performed noticeably worse than the last stable > version. Short version: Please apply the attached patch. Longer version: The > problem turned out to be directories that contain files that fontconfig > doesn''t seem to use yet it analyses them (.pcf.gz files). If a directory > contains only .pcf.gz files and has no subdirectories, then the code that the > patch removes will not write a cache file, yet the return FcTrue means the > directory has been successfully cached. However if you look in > FcCacheReadDirs() you can see that if it can''t use the cache file it will > call FcDirScanConfig(). So I suggest the attached patch which will write a > more or less empty cache file even in such directories.I''m not sure that this patch will not cause problems. I think that I want to generate a zero-length file in such a case, without a header at all. But I''m a bit confused: when I run fc-cache in a directory with only pcf.gz files, fc-cache *does* generate a fonts.cache-2 file for me. Why is this different for you?> And, actually, it seems I''ve just found another problem :(. If I run > Konqueror, run ''fc-cache -f'' and go back to Konqueror, it crashes quite soon. > Guessing from strace of fc-cache, where I cannot see any call to unlink(), > and from the calls to ftruncate() I assume that the new cache file contents > are written directly in the same file instead of unlinking the old file > first, and fontconfig doesn''t survive that. As I don''t understand the code > that much I cannot confirm this theory for sure, which also means no patch > for this, sorry :).Which cache file are you talking about? $HOME/.fonts.cache-2, or a directory cache? (Are you running fc-cache as root?) pat
On Tuesday 27 September 2005 17:47, Patrick Lam wrote:> Lubos Lunak wrote: > > However, when I tested the 2.3.90 release yesterday, I was originally > > disappointed to see it performed noticeably worse than the last stable > > version. Short version: Please apply the attached patch. Longer version: > > The problem turned out to be directories that contain files that > > fontconfig doesn''t seem to use yet it analyses them (.pcf.gz files). If a > > directory contains only .pcf.gz files and has no subdirectories, then the > > code that the patch removes will not write a cache file, yet the return > > FcTrue means the directory has been successfully cached. However if you > > look in > > FcCacheReadDirs() you can see that if it can''t use the cache file it will > > call FcDirScanConfig(). So I suggest the attached patch which will write > > a more or less empty cache file even in such directories. > > I''m not sure that this patch will not cause problems. I think that I > want to generate a zero-length file in such a case, without a header at > all. But I''m a bit confused: when I run fc-cache in a directory with > only pcf.gz files, fc-cache *does* generate a fonts.cache-2 file for me. > Why is this different for you?Maybe that directory you have has also a subdirectory? In the attached file there''s output of ''ls'' in /usr/X11R6/lib/X11/fonts/latin2/misc and also the output of ''fc-cache -f -v''. You can see that it says 0 fonts, 0 dirs, which AFAICS exactly matches the condition my patches removes.> > > And, actually, it seems I''ve just found another problem :(. If I run > > Konqueror, run ''fc-cache -f'' and go back to Konqueror, it crashes quite > > soon. Guessing from strace of fc-cache, where I cannot see any call to > > unlink(), and from the calls to ftruncate() I assume that the new cache > > file contents are written directly in the same file instead of unlinking > > the old file first, and fontconfig doesn''t survive that. As I don''t > > understand the code that much I cannot confirm this theory for sure, > > which also means no patch for this, sorry :). > > Which cache file are you talking about? $HOME/.fonts.cache-2, or a > directory cache? (Are you running fc-cache as root?)Any cache file I''d say (and yes, I ran fc-cache as root). AFAIK if you truncate a file that''s mmap-ed, any access to the no longer existing part of the file results in bad things. I don''t know the structure of the cache file, it seems to me it can be built in several steps for different architectures, but I''d expect that the code can''t cope with the data simply changing under its hands, and unless the format specifically prevents that I''d expect that to happen. Would there be any problem if you simply unlinked the old file before writing the new one? -- Lubos Lunak KDE Developer l.lunak@suse.cz l.lunak@kde.org -------------- next part -------------- . .. .fonts-config-timestamp 10x20.pcf.gz 14x26.pcf.gz 5x7.pcf.gz 5x8.pcf.gz 6x13.pcf.gz 6x13B.pcf.gz 6x14.pcf.gz 6x14B.pcf.gz 6x9.pcf.gz 7x13.pcf.gz 7x13B.pcf.gz 7x14.pcf.gz 7x14B.pcf.gz 8x10.pcf.gz 8x12.pcf.gz 8x12B.pcf.gz 8x14.pcf.gz 8x14B.pcf.gz 8x16.pcf.gz 8x16B.pcf.gz 8x16BS.pcf.gz 8x8.pcf.gz 9x15.pcf.gz 9x15B.pcf.gz 9x17.pcf.gz 9x17B.pcf.gz encodings.dir fonts.alias fonts.cache-1 fonts.cache-2 fonts.dir fonts.scale fc-cache: "/usr/share/fonts": skipping, 0 fonts, 2 dirs fc-cache: "/usr/share/fonts/bdf": skipping, 0 fonts, 0 dirs fc-cache: "/usr/share/fonts/wine": skipping, 15 fonts, 0 dirs fc-cache: "/usr/X11R6/lib/X11/fonts": skipping, 0 fonts, 13 dirs fc-cache: "/usr/X11R6/lib/X11/fonts/75dpi": skipping, 398 fonts, 0 dirs fc-cache: "/usr/X11R6/lib/X11/fonts/truetype": skipping, 25 fonts, 0 dirs fc-cache: "/usr/X11R6/lib/X11/fonts/Type1": skipping, 21 fonts, 0 dirs fc-cache: "/usr/X11R6/lib/X11/fonts/misc": skipping, 65 fonts, 0 dirs fc-cache: "/usr/X11R6/lib/X11/fonts/japanese": skipping, 0 fonts, 0 dirs fc-cache: "/usr/X11R6/lib/X11/fonts/URW": skipping, 52 fonts, 0 dirs fc-cache: "/usr/X11R6/lib/X11/fonts/latin2": skipping, 0 fonts, 1 dirs fc-cache: "/usr/X11R6/lib/X11/fonts/latin2/misc": skipping, 0 fonts, 0 dirs fc-cache: "/usr/X11R6/lib/X11/fonts/CID": skipping, 0 fonts, 0 dirs fc-cache: "/usr/X11R6/lib/X11/fonts/encodings": skipping, 0 fonts, 1 dirs fc-cache: "/usr/X11R6/lib/X11/fonts/encodings/large": skipping, 0 fonts, 0 dirs fc-cache: "/usr/X11R6/lib/X11/fonts/local": skipping, 0 fonts, 0 dirs fc-cache: "/usr/X11R6/lib/X11/fonts/util": skipping, 0 fonts, 0 dirs fc-cache: "/usr/X11R6/lib/X11/fonts/baekmuk": skipping, 0 fonts, 0 dirs fc-cache: "/usr/X11R6/lib/X11/fonts/uni": skipping, 2 fonts, 0 dirs fc-cache: "/opt/kde3/share/fonts": skipping, 3 fonts, 1 dirs fc-cache: "/opt/kde3/share/fonts/override": skipping, 0 fonts, 0 dirs fc-cache: "/usr/lib/ooo-2.0/share/fonts": skipping, 0 fonts, 1 dirs fc-cache: "/usr/lib/ooo-2.0/share/fonts/truetype": skipping, 1 fonts, 0 dirs fc-cache: "/usr/lib/ooo-1.1/share/fonts": skipping, no such directory fc-cache: "/opt/OpenOffice.org/share/fonts": skipping, no such directory fc-cache: "/opt/staroffice6.0/share/fonts": skipping, no such directory fc-cache: "/usr/lib64/jvm/java-1.5.0-sun-1.5.0_01/jre/lib/fonts": skipping, no such directory fc-cache: "/usr/lib/jvm/java-1.5.0-sun-1.5.0_01/jre/lib/fonts": skipping, no such directory fc-cache: "/usr/lib/jvm/java-1.4.2-sun-1.4.2.06/jre/lib/fonts": skipping, 12 fonts, 0 dirs fc-cache: "/usr/X11R6/lib/Acrobat7/Resource/Font": skipping, 16 fonts, 1 dirs fc-cache: "/usr/X11R6/lib/Acrobat7/Resource/Font/PFM": skipping, 0 fonts, 0 dirs fc-cache: "/usr/local/share/fonts/": skipping, no such directory fc-cache: "/root/.fonts": skipping, no such directory fc-cache: succeeded
On Tue, 2005-09-27 at 18:12 +0200, Lubos Lunak wrote:> On Tuesday 27 September 2005 17:47, Patrick Lam wrote: > > Lubos Lunak wrote: > > > However, when I tested the 2.3.90 release yesterday, I was originally > > > disappointed to see it performed noticeably worse than the last stable > > > version. Short version: Please apply the attached patch. Longer version: > > > The problem turned out to be directories that contain files that > > > fontconfig doesn''t seem to use yet it analyses them (.pcf.gz files). If a > > > directory contains only .pcf.gz files and has no subdirectories, then the > > > code that the patch removes will not write a cache file, yet the return > > > FcTrue means the directory has been successfully cached. However if you > > > look in > > > FcCacheReadDirs() you can see that if it can''t use the cache file it will > > > call FcDirScanConfig(). So I suggest the attached patch which will write > > > a more or less empty cache file even in such directories. > > > > I''m not sure that this patch will not cause problems. I think that I > > want to generate a zero-length file in such a case, without a header at > > all. But I''m a bit confused: when I run fc-cache in a directory with > > only pcf.gz files, fc-cache *does* generate a fonts.cache-2 file for me. > > Why is this different for you? > > Maybe that directory you have has also a subdirectory? In the attached file > there''s output of ''ls'' in /usr/X11R6/lib/X11/fonts/latin2/misc and also the > output of ''fc-cache -f -v''. You can see that it says 0 fonts, 0 dirs, which > AFAICS exactly matches the condition my patches removes. > > > > > > And, actually, it seems I''ve just found another problem :(. If I run > > > Konqueror, run ''fc-cache -f'' and go back to Konqueror, it crashes quite > > > soon. Guessing from strace of fc-cache, where I cannot see any call to > > > unlink(), and from the calls to ftruncate() I assume that the new cache > > > file contents are written directly in the same file instead of unlinking > > > the old file first, and fontconfig doesn''t survive that. As I don''t > > > understand the code that much I cannot confirm this theory for sure, > > > which also means no patch for this, sorry :). > > > > Which cache file are you talking about? $HOME/.fonts.cache-2, or a > > directory cache? (Are you running fc-cache as root?) > > Any cache file I''d say (and yes, I ran fc-cache as root). AFAIK if you > truncate a file that''s mmap-ed, any access to the no longer existing part of > the file results in bad things. I don''t know the structure of the cache file, > it seems to me it can be built in several steps for different architectures, > but I''d expect that the code can''t cope with the data simply changing under > its hands, and unless the format specifically prevents that I''d expect that > to happen. Would there be any problem if you simply unlinked the old file > before writing the new one? >Thats what we do with other mmapped caches in GTK+, atomically replace them by writing the new cache to a tmp file, and then moving it in place. Matthias
Lubos Lunak wrote:> Any cache file I''d say (and yes, I ran fc-cache as root). AFAIK if you > truncate a file that''s mmap-ed, any access to the no longer existing part of > the file results in bad things. I don''t know the structure of the cache file, > it seems to me it can be built in several steps for different architectures, > but I''d expect that the code can''t cope with the data simply changing under > its hands, and unless the format specifically prevents that I''d expect that > to happen. Would there be any problem if you simply unlinked the old file > before writing the new one?I now use FcAtomic to write to these cache files. Try the latest CVS version. pat
Lubos Lunak wrote:> However, when I tested the 2.3.90 release yesterday, I was originally > disappointed to see it performed noticeably worse than the last stable > version. Short version: Please apply the attached patch. Longer version: The > problem turned out to be directories that contain files that fontconfig > doesn''t seem to use yet it analyses them (.pcf.gz files). If a directory > contains only .pcf.gz files and has no subdirectories, then the code that the > patch removes will not write a cache file, yet the return FcTrue means the > directory has been successfully cached. However if you look in > FcCacheReadDirs() you can see that if it can''t use the cache file it will > call FcDirScanConfig(). So I suggest the attached patch which will write a > more or less empty cache file even in such directories.Yes, I''ve applied your patch. The current implementation of FcDirCacheWrite does allow it to successfully write zero-length cache files. (In the past, it would have crashed, but I seem to have fixed those cases.) pat
On Wednesday 28 September 2005 02:24, Patrick Lam wrote:> Lubos Lunak wrote: > > Any cache file I''d say (and yes, I ran fc-cache as root). AFAIK if you > > truncate a file that''s mmap-ed, any access to the no longer existing part > > of the file results in bad things. I don''t know the structure of the > > cache file, it seems to me it can be built in several steps for different > > architectures, but I''d expect that the code can''t cope with the data > > simply changing under its hands, and unless the format specifically > > prevents that I''d expect that to happen. Would there be any problem if > > you simply unlinked the old file before writing the new one? > > I now use FcAtomic to write to these cache files. Try the latest CVS > version.It''s ok now, thanks. -- Lubos Lunak KDE Developer l.lunak@suse.cz l.lunak@kde.org
Hello, first of all I''d like to thank for the recent work on improving fontconfig''s performance. It''s good to see this problem will be finally solved. However, when I tested the 2.3.90 release yesterday, I was originally disappointed to see it performed noticeably worse than the last stable version. Short version: Please apply the attached patch. Longer version: The problem turned out to be directories that contain files that fontconfig doesn''t seem to use yet it analyses them (.pcf.gz files). If a directory contains only .pcf.gz files and has no subdirectories, then the code that the patch removes will not write a cache file, yet the return FcTrue means the directory has been successfully cached. However if you look in FcCacheReadDirs() you can see that if it can''t use the cache file it will call FcDirScanConfig(). So I suggest the attached patch which will write a more or less empty cache file even in such directories. With this patch both the memory and startup time problems of fontconfig seem to be gone. Great :). And, actually, it seems I''ve just found another problem :(. If I run Konqueror, run ''fc-cache -f'' and go back to Konqueror, it crashes quite soon. Guessing from strace of fc-cache, where I cannot see any call to unlink(), and from the calls to ftruncate() I assume that the new cache file contents are written directly in the same file instead of unlinking the old file first, and fontconfig doesn''t survive that. As I don''t understand the code that much I cannot confirm this theory for sure, which also means no patch for this, sorry :). Thanks -- Lubos Lunak KDE Developer l.lunak@suse.cz l.lunak@kde.org -------------- next part -------------- A non-text attachment was scrubbed... Name: fccache.c.patch Type: text/x-diff Size: 386 bytes Desc: not available Url : http://lists.freedesktop.org/archives/fontconfig/attachments/20050927/fe1e9042/fccache.c.bin