Matthias Clasen wrote:> It would probably be worth pointing this out in the fc-cache man page.I''ll shortly commit this to the branch: Index: fc-cache/fc-cache.sgml ==================================================================RCS file: /cvs/fontconfig/fontconfig/fc-cache/fc-cache.sgml,v retrieving revision 1.1 diff -u -r1.1 fc-cache.sgml --- fc-cache/fc-cache.sgml 9 Oct 2003 18:19:07 -0000 1.1 +++ fc-cache/fc-cache.sgml 23 Sep 2005 03:57:53 -0000 @@ -88,6 +88,13 @@ This cache is used to speed up application startup when using the fontconfig library.</para> + <para>Note that <command>&dhpackage;</command> must be executed + once per architecture to generate font information customized + for that architecture. On a subsequent run, + <command>&dhpackage;</command> will augment the cache + information files with the information for the new + architecture. </para> + </refsect1> <refsect1> <title>OPTIONS</title>
On Thu, 2005-09-22 at 23:28 -0400, Patrick Lam wrote:> Matthias Clasen wrote: > >>It can''t possibly be platform-independent if it''s to be mmapped... > >>otherwise you end up making another copy of the data before you can use > >>it, which eliminates the whole space savings benefit. > > > > Well, you can''t use it as C structures, thats true. But if you look at > > e.g. the GTK+ icon cache or the xdgmime cache formats, it is possible to > > design e.g. a hashtable in a way which allows it to be directly used > > out of a platform-neutral mmapped file. > > I don''t think that works as well here, since there are more complicated > data structures, especially ones that get returned to the user (so they > definitely need to be duplicated).Thats true.> >>I''ve tested it on multiple platforms before, but please test it again. > >>It was designed to work on multiple platforms, by keeping one copy per > >>platform (all the copies are stored in the same file, but in different > >>sections). > > > > Ah, how does that work ? Do I have to run fc-cache on all platforms I > > want to have caches for ? > > Yes, you do. >It would probably be worth pointing this out in the fc-cache man page.
Matthias Clasen wrote:>>It can''t possibly be platform-independent if it''s to be mmapped... >>otherwise you end up making another copy of the data before you can use >>it, which eliminates the whole space savings benefit. > > Well, you can''t use it as C structures, thats true. But if you look at > e.g. the GTK+ icon cache or the xdgmime cache formats, it is possible to > design e.g. a hashtable in a way which allows it to be directly used > out of a platform-neutral mmapped file.I don''t think that works as well here, since there are more complicated data structures, especially ones that get returned to the user (so they definitely need to be duplicated).>>I''ve tested it on multiple platforms before, but please test it again. >>It was designed to work on multiple platforms, by keeping one copy per >>platform (all the copies are stored in the same file, but in different >>sections). > > Ah, how does that work ? Do I have to run fc-cache on all platforms I > want to have caches for ?Yes, you do. pat
On Thu, 2005-09-22 at 16:50 -0400, Patrick Lam wrote:> Matthias Clasen wrote: > > Ok, I poked some more at this, and it seems to be a problem with > > FcNameRegisterObjectTypes() and/ or FcNameGetObjectType(). What is > > happening is that Xft calls FcNameRegisterObjectTypes() to register some > > additional names, "render" being the third one. There seems to be some > > confusion wrt to the assignment of ids to the registered strings, at > > least we end up with "render" from the Xft strings getting id 2, which > > is the same id as "style" from the FC registered strings got. This has > > then the effect that setting "render" to TRUE sets "style" instead, > > which is bad, since "style" is not a boolean... > > Thanks a lot! I''ve committed a fix to this problem now, and > gnome-font-properties works for me. I suppose I should have guessed > that people might actually use FcNameRegisterObjectTypes. > > Now that gnome-terminal and gnome-font-properties work, I think it > should be safe to make a development release, hopefully tonight, after > looking at Behdad''s patch.Yay, gnome-font-properties works now. The one issue I still see is firefox refusing to start if FONTCONFIG_PATH is set in /usr/bin/firefox, but that doesn''t have to block a release, I think. You left a debugging printf in fcpat.c, btw. Matthias
On Sun, 2005-09-11 at 08:46 -0400, Patrick Lam wrote:> Matthias Clasen wrote: > > On Sun, 2005-09-11 at 01:17 -0400, Patrick Lam wrote: > > > >>Matthias Clasen wrote: > >> > >>>Here is another problem I met after installing the mmap branch > >>>fontconfig system-wide on my system. gnome-terminal doesn''t > >>>seem to find fonts anymore. I haven''t yet tried to follow all the > >>>fontconfig pattern manipulation done in vte, but it seems to > >>>trigger an incompatibility. > >>> > >>>A similar problem seems to affect the previews in the font capplet, > >>>which also seem to pick up a very small font size. > >> > >>I have fixed this problem. Let me know if that clears up the problem > >>for you. > > > > No, I still see the same problems. gnome-terminal and > > gnome-font-properties come up with fonts of very small size, and > > firefox segfaults. > > I''m definitely using the fontconfig branch on this computer with > gnome-terminal and firefox, which both work. (Also thunderbird). > However, gnome-font-properties currently doesn''t work for me -- it > reports no fonts found -- so I''ll look into that. (I thought that my > patch got it working last night).I looked a bit more into my problems, and found that firefox crashes with an assertion inside pango, if the FONTCONFIG_PATH is set (even if it is set to the default value /etc/fonts). The assertion I run into is that a FcPatternGetString (pattern, FC_FAMILY, 0, (FcChar8 **) &s) returns FcResultTypeMismatch for one pattern. And in fact, if I print the pattern, it looks as if the family name is mistaken as a boolean: Pattern 26 of 32 family: FcFalse(w) familylang: "xx"(s) style: "Gothic-Regular"(s) stylelang: "xx"(s) slant: 0(i)(s) weight: 100(i)(s) width: 100(i)(s) size: 9.75(f)(s) pixelsize: 13(f)(s) foundry: "unknown"(s) antialias: FcTrue(s) hintstyle: 2(i)(s) hinting: FcFalse(w) verticallayout: FcFalse(s) autohint: FcFalse(s) globaladvance: FcTrue(s) file: "/usr/share/fonts/japanese/TrueType/sazanami-gothic.ttf"(s) index: 0(i)(s) outline: FcTrue(s) scalable: FcTrue(s) rgba: 5(i)(s) charset: set(s) lang: aa|ast|ay|bg|bi|br|ch|da|de|en|es|eu|fj|fo|fur|fy|gd|gl| gv|ho|ia|id|ie|io|is|it|ja|kum|lb|mg|nb|nds|nl|nn|no|oc|om|os|pt|rm|ru| sel|sma|smj|so|sq|sv|sw|tn|ts|vo|wa|xh|yap|zu(s) fontversion: 65536(i)(s) capability: "otlayout:kana"(s) fontformat: "TrueType"(s) Starting firefox without a FONTCONFIG_PATH seems to work fine. Regards, Matthias
On Sun, 2005-09-11 at 01:17 -0400, Patrick Lam wrote:> Matthias Clasen wrote: > > Here is another problem I met after installing the mmap branch > > fontconfig system-wide on my system. gnome-terminal doesn''t > > seem to find fonts anymore. I haven''t yet tried to follow all the > > fontconfig pattern manipulation done in vte, but it seems to > > trigger an incompatibility. > > > > A similar problem seems to affect the previews in the font capplet, > > which also seem to pick up a very small font size. > > I have fixed this problem. Let me know if that clears up the problem > for you.No, I still see the same problems. gnome-terminal and gnome-font-properties come up with fonts of very small size, and firefox segfaults. Matthias
On Wed, 2005-09-07 at 11:42 -0400, Patrick Lam wrote:> Matthias Clasen wrote: > > I think FcGlobalCacheLoad() needs some robustness love. Just > > copy any text file to ~/.fonts.cache-2 and watch FcGlobalCacheLoad() > > go in an infinite loop: > > Actually, FcCacheSkipToArch was looping. I''ve fixed this by ensuring > that FcCacheSkipToArch always makes progress: > > if (FcCacheReadString (fd, candidate_arch_machine_name_count, > sizeof (candidate_arch_machine_name_count)) == 0) > return -1; > if (!strlen(candidate_arch_machine_name_count)) > return -1; > bs = strtol(candidate_arch_machine_name_count, &candidate_arch, 16); > > + if (!bs || bs < strlen (candidate_arch_machine_name_count)) > + return -1; > > As you suggested, I also check the return values on read and lseek; > although they weren''t the problem in this particular case, it''s probably > still better to fix them. > > Garbage .fonts.cache-2 files no longer trash fontconfig; it now > correctly regenerates the global cache file. > > patI wonder if it would be a good idea to do a development release or at least create a snapshot tarball off the 2-4 branch, to get some wider testing, or are there currently any outstanding issues with the mmap cache that make a test release impractical ? Matthias
On Thu, 2005-09-08 at 12:56 -0400, Patrick Lam wrote:> Matthias Clasen wrote: > > I wonder if it would be a good idea to do a development release or > > at least create a snapshot tarball off the 2-4 branch, to get some wider > > testing, or are there currently any outstanding issues with the mmap > > cache that make a test release impractical ? > > I think this is a good idea. I''m kind of busy until Saturday, but on > Saturday I''ll create at least a snapshot tarball. >Btw, do you have any docs on the mmap cache file format ? I doesn''t have to be anything grandiose, but a few notes like http://mail.gnome.org/archives/gtk-devel-list/2004-April/msg00065.html might be great to include in CVS. Matthias
On Fri, 2005-09-09 at 04:11 -0400, James Cloos wrote:> Incidently, it would be useful to have a program that can read in a > specified fonts.cache-2 (ui a la cat(1)''s ui) and print the equivalent > fonts.cache-1. > > The easiest way, currently, to determine the name needed to get a font > known by filename is to grep for the filename in fonts.cache-1; it''d > be a shame to loose that.fc-list : family style file | grep <filename> works reasonably well...> I''ll look into how hard that''ll be to write, but if you can whip it up > w/o thinking please do so. :-)It would be easy to do, and might be useful to test for regressions. Use FcNameUnparse to generate a suitable string for each font pattern. -keith -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part Url : http://lists.freedesktop.org/archives/fontconfig/attachments/20050910/d2bc9bee/attachment.pgp
Matthias Clasen wrote:> Here is another problem I met after installing the mmap branch > fontconfig system-wide on my system. gnome-terminal doesn''t > seem to find fonts anymore. I haven''t yet tried to follow all the > fontconfig pattern manipulation done in vte, but it seems to > trigger an incompatibility. > > A similar problem seems to affect the previews in the font capplet, > which also seem to pick up a very small font size.I have fixed this problem. Let me know if that clears up the problem for you. Tomorrow (after I get back from sailing), I intend to write up some quick docs of the cache format and see if I can whip together an fc-dump binary; then I''ll package the development snapshot 2.3.90. keithp: I''ve added the FC_RENDER member back to the FcBaseObjectTypes array; it was previously commented out. We do put FC_RENDERs into FcPatterns, although I guess that we don''t write them out to disk. So fontconfig was trying to write an FC_RENDER element into an FcPattern, failing, and bailing. Does this look like the correct solution to you? 2005-09-11 Patrick Lam <plam@mit.edu> * fontconfig/fontconfig.h: * src/fcname.c: Add a global binding for the ''render'' pattern element used by Xft; the lack of said binding prevented programs from using FcPatterns in Xft. pat
On Sun, 2005-09-11 at 01:17 -0400, Patrick Lam wrote:> keithp: I''ve added the FC_RENDER member back to the FcBaseObjectTypes > array; it was previously commented out. We do put FC_RENDERs into > FcPatterns, although I guess that we don''t write them out to disk. So > fontconfig was trying to write an FC_RENDER element into an FcPattern, > failing, and bailing. Does this look like the correct solution to you?I thought the FcBaseObjectTypes were only those which were written to disk and that patterns could contain other object types as long as they were only in memory. Fontconfig applications (and even config files) can add arbitrary object types to patterns, even those which are unknown to any existing library or application... Or am I just confused here? -keith -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part Url : http://lists.freedesktop.org/archives/fontconfig/attachments/20050911/01840c00/attachment.pgp
Matthias Clasen wrote:> I think FcGlobalCacheLoad() needs some robustness love. Just > copy any text file to ~/.fonts.cache-2 and watch FcGlobalCacheLoad() > go in an infinite loop:Actually, FcCacheSkipToArch was looping. I''ve fixed this by ensuring that FcCacheSkipToArch always makes progress: if (FcCacheReadString (fd, candidate_arch_machine_name_count, sizeof (candidate_arch_machine_name_count)) == 0) return -1; if (!strlen(candidate_arch_machine_name_count)) return -1; bs = strtol(candidate_arch_machine_name_count, &candidate_arch, 16); + if (!bs || bs < strlen (candidate_arch_machine_name_count)) + return -1; As you suggested, I also check the return values on read and lseek; although they weren''t the problem in this particular case, it''s probably still better to fix them. Garbage .fonts.cache-2 files no longer trash fontconfig; it now correctly regenerates the global cache file. pat
Matthias Clasen wrote:> I wonder if it would be a good idea to do a development release or > at least create a snapshot tarball off the 2-4 branch, to get some wider > testing, or are there currently any outstanding issues with the mmap > cache that make a test release impractical ?I think this is a good idea. I''m kind of busy until Saturday, but on Saturday I''ll create at least a snapshot tarball. pat
I wrote a freeform document that describes the structure of the cache
files; it''s included below. I''m not sure where I could put it
in CVS...
---
Fontconfig maintains two cache files: a global cache (on a per-user
basis) and directory caches (created by the superuser). The global
cache consists of a number of concatenated directory caches.
Here''s the structure of a directory cache:
* a next-offset/machine signature line in plain ASCII:
+ (as a %8x string) the offset of the next architecture''s block
+ (produced by FcCacheMachineSignature): an architecture signature
- endian-testing signature 0x12345678
- sizeof (char)
- sizeof (char *)
- sizeof (int)
- sizeof (FcPattern)
- sizeof (FcPatternEltPtr)
- sizeof (struct _FcPatternElt *)
- sizeof (FcPatternElt)
- sizeof (FcObjectPtr)
- sizeof (FcValueListPtr)
- sizeof (FcValue)
- sizeof (FcValueBinding)
- sizeof (struct _FcValueList *)
- sizeof (FcCharSet)
- sizeof (FcCharLeaf **)
- sizeof (FcChar16 *)
- sizeof (FcChar16)
- sizeof (FcCharLeaf)
- sizeof (FcChar32)
- sizeof (FcCache)
* a header block:
typedef struct _FcCache {
int magic; /* 0xFC02FC02 */
int count; /* number of bytes of data in block */
int bank; /* bank ID */
int pattern_count; /* number of FcPatterns */
int patternelt_count; /* number of FcPatternElts */
int valuelist_count; /* number of FcValueLists */
int str_count; /* size of strings appearing as FcValues */
int langset_count; /* number of FcLangSets */
int charset_count; /* number of FcCharSets */
int charset_numbers_count;
int charset_leaf_count;
int charset_leaf_idx_count;
}
* pattern data:
Stored as a character array containing FcPatterns, FcPatternElts,
FcValueLists, strings, FcLangSets and FcCharSets. To permit mmaping,
this data is aligned on an 8k boundary.
About banks: when caching a directory, fontconfig assigns a random
bank number to that directory. This bank number combines with the
static IDs to canonically identify resources; for instance, a
FcPattern might refer to its first component FcPatternElt as
(92848929, 2). The bank ids have to be part of the data structures so
that when given, for instance, an FcValueList in isolation, it is
possible to find where the FcValue is pointing to. Since the bank ids
are chosen with rand(), the probability of collision is quite low (if
the random number generator isn''t stupid).
The structure of a global cache is similar. It also starts with a
machine identification line in plain ASCII. Next, it contains a string
containing the directory name, followed by that directory''s cache (as
above); this repeats for all of the directories in the cache.
Matthias Clasen wrote:> Btw, do you have any docs on the mmap cache file format ? I doesn''t > have to be anything grandiose, but a few notes like > http://mail.gnome.org/archives/gtk-devel-list/2004-April/msg00065.html > might be great to include in CVS.I''ll put together something better this weekend and include it in the source, as well as posting it to the list. Briefly, there''s always a line of signature identifying the machine and pointing to the next block. Then you have an FcCache data structure written in binary format, followed by a chunk of memory which gets mmapped. On a global cache, you have a number of these dir chunks, prefaced by directory name. pat
James Cloos wrote:> Incidently, it would be useful to have a program that can read in a > specified fonts.cache-2 (ui a la cat(1)''s ui) and print the equivalent > fonts.cache-1. > > The easiest way, currently, to determine the name needed to get a font > known by filename is to grep for the filename in fonts.cache-1; it''d > be a shame to loose that. > > I''ll look into how hard that''ll be to write, but if you can whip it up > w/o thinking please do so. :-)Doesn''t `fc-list : file family'' do what you want? The fonts.cache-2 file doesn''t quite contain all of the information in the fonts.cache-1 file (I don''t think I have file times), but I could generate most of the information there, if you think it would be useful. pat
Here is another problem I met after installing the mmap branch fontconfig system-wide on my system. gnome-terminal doesn''t seem to find fonts anymore. I haven''t yet tried to follow all the fontconfig pattern manipulation done in vte, but it seems to trigger an incompatibility. A similar problem seems to affect the previews in the font capplet, which also seem to pick up a very small font size. The rest of the desktop seems to work fine. Matthias
Matthias Clasen wrote:> On Sun, 2005-09-11 at 01:17 -0400, Patrick Lam wrote: > >>Matthias Clasen wrote: >> >>>Here is another problem I met after installing the mmap branch >>>fontconfig system-wide on my system. gnome-terminal doesn''t >>>seem to find fonts anymore. I haven''t yet tried to follow all the >>>fontconfig pattern manipulation done in vte, but it seems to >>>trigger an incompatibility. >>> >>>A similar problem seems to affect the previews in the font capplet, >>>which also seem to pick up a very small font size. >> >>I have fixed this problem. Let me know if that clears up the problem >>for you. > > No, I still see the same problems. gnome-terminal and > gnome-font-properties come up with fonts of very small size, and > firefox segfaults.I''m definitely using the fontconfig branch on this computer with gnome-terminal and firefox, which both work. (Also thunderbird). However, gnome-font-properties currently doesn''t work for me -- it reports no fonts found -- so I''ll look into that. (I thought that my patch got it working last night). pat
On Thu, 2005-09-15 at 16:38 -0400, Patrick Lam wrote:> Matthias Clasen wrote: > > Looking at the gnome-font-properties problem some more, I see > > XftFontMatch() returning NULL. Inside XftFontMatch(), all seems > > well until the call to XftDefaultSubstitute() which returns a > > pattern where the "style" value is set to FcTrue, which later upsets > > FcCompareValueList. > > I''ve fixed the gnome-terminal problem the right way now, at least for my > system (you''ll have to rebuild all your caches). gnome-font-properties > still doesn''t work. Thanks for this tip, I''ll look into it presently > and hopefully will have a fix soon. > > pat >Yes, gnome-terminal works now, thanks.
Matthias Clasen wrote:> So this means that the file format is not endianness- and > platform-independent, but contains enough information to load it on > other platforms ? Does the current loader support loading cache files > from other platforms ? > > I guess I should just test that...It can''t possibly be platform-independent if it''s to be mmapped... otherwise you end up making another copy of the data before you can use it, which eliminates the whole space savings benefit. I''ve tested it on multiple platforms before, but please test it again. It was designed to work on multiple platforms, by keeping one copy per platform (all the copies are stored in the same file, but in different sections). pat
I think FcGlobalCacheLoad() needs some robustness love. Just
copy any text file to ~/.fonts.cache-2 and watch FcGlobalCacheLoad()
go in an infinite loop:
while (1)
{
FcCacheReadString (cache->fd, name_buf, sizeof (name_buf));
if (!strlen(name_buf))
break;
d = malloc (sizeof (FcGlobalCacheDir));
if (!d)
goto bail1;
d->next = cache->dirs;
cache->dirs = d;
d->name = FcStrCopy (name_buf);
d->ent = 0;
d->offset = lseek (cache->fd, 0, SEEK_CUR);
read (cache->fd, &d->metadata, sizeof (FcCache));
lseek (cache->fd, d->metadata.count, SEEK_CUR);
}
I have no concrete proposal for how to rework this, but at the very
least, the return values of read() and lseek() should be checked to
handle errors.
Regards, Matthias
>>>>> "Patrick" == Patrick Lam <plam@MIT.EDU> writes:Patrick> Doesn''t `fc-list : file family'' do what you want? Hmm. Missed that. But that doesn''t work in the case where the directory in question isn''t currently in the list of <dir/>s. I have several that are not listed in the fonts.conf files except when I need said family for something specific. (I especially find that type1 fonts -- as opposed to otf (sfnt/cff) and ttf (sfnt/glyf) fonts -- are more bothersome than helpful when left in full time.) Patrick> The fonts.cache-2 file doesn''t quite contain all of the Patrick> information in the fonts.cache-1 file (I don''t think I have Patrick> file times), but I could generate most of the information Patrick> there, if you think it would be useful. I don''t see a filetime in the cache-1 files. Unless the 2nd column is atime? I have this in mine: "filename" 0 "fontname" where fontname is: basename:style:slant:weight:width:foundry:index:outline:scalable:charset:lang:fontversion:capability:fontformat and am currently running w/ noatime. -JimC -- James H. Cloos, Jr. <cloos@jhcloos.com>
Incidently, it would be useful to have a program that can read in a specified fonts.cache-2 (ui a la cat(1)''s ui) and print the equivalent fonts.cache-1. The easiest way, currently, to determine the name needed to get a font known by filename is to grep for the filename in fonts.cache-1; it''d be a shame to loose that. I''ll look into how hard that''ll be to write, but if you can whip it up w/o thinking please do so. :-) -JimC -- James H. Cloos, Jr. <cloos@jhcloos.com>
On Thu, 2005-09-22 at 23:05 -0400, Patrick Lam wrote:> Matthias Clasen wrote: > > So this means that the file format is not endianness- and > > platform-independent, but contains enough information to load it on > > other platforms ? Does the current loader support loading cache files > > from other platforms ? > > > > I guess I should just test that... > > It can''t possibly be platform-independent if it''s to be mmapped... > otherwise you end up making another copy of the data before you can use > it, which eliminates the whole space savings benefit.Well, you can''t use it as C structures, thats true. But if you look at e.g. the GTK+ icon cache or the xdgmime cache formats, it is possible to design e.g. a hashtable in a way which allows it to be directly used out of a platform-neutral mmapped file.> I''ve tested it on multiple platforms before, but please test it again. > It was designed to work on multiple platforms, by keeping one copy per > platform (all the copies are stored in the same file, but in different > sections).Ah, how does that work ? Do I have to run fc-cache on all platforms I want to have caches for ? Matthias
On Thu, 2005-09-22 at 22:52 -0400, Patrick Lam wrote:> I wrote a freeform document that describes the structure of the cache > files; it''s included below. I''m not sure where I could put it in CVS... > > --- > > Fontconfig maintains two cache files: a global cache (on a per-user > basis) and directory caches (created by the superuser). The global > cache consists of a number of concatenated directory caches. > > Here''s the structure of a directory cache: > > * a next-offset/machine signature line in plain ASCII: > > + (as a %8x string) the offset of the next architecture''s block > + (produced by FcCacheMachineSignature): an architecture signature > - endian-testing signature 0x12345678 > - sizeof (char) > - sizeof (char *) > - sizeof (int) > - sizeof (FcPattern) > - sizeof (FcPatternEltPtr) > - sizeof (struct _FcPatternElt *) > - sizeof (FcPatternElt) > - sizeof (FcObjectPtr) > - sizeof (FcValueListPtr) > - sizeof (FcValue) > - sizeof (FcValueBinding) > - sizeof (struct _FcValueList *) > - sizeof (FcCharSet) > - sizeof (FcCharLeaf **) > - sizeof (FcChar16 *) > - sizeof (FcChar16) > - sizeof (FcCharLeaf) > - sizeof (FcChar32) > - sizeof (FcCache) > > * a header block: >So this means that the file format is not endianness- and platform-independent, but contains enough information to load it on other platforms ? Does the current loader support loading cache files from other platforms ? I guess I should just test that... Matthias
Matthias Clasen wrote:> Ok, I poked some more at this, and it seems to be a problem with > FcNameRegisterObjectTypes() and/ or FcNameGetObjectType(). What is > happening is that Xft calls FcNameRegisterObjectTypes() to register some > additional names, "render" being the third one. There seems to be some > confusion wrt to the assignment of ids to the registered strings, at > least we end up with "render" from the Xft strings getting id 2, which > is the same id as "style" from the FC registered strings got. This has > then the effect that setting "render" to TRUE sets "style" instead, > which is bad, since "style" is not a boolean...Thanks a lot! I''ve committed a fix to this problem now, and gnome-font-properties works for me. I suppose I should have guessed that people might actually use FcNameRegisterObjectTypes. Now that gnome-terminal and gnome-font-properties work, I think it should be safe to make a development release, hopefully tonight, after looking at Behdad''s patch. pat
On Thu, 2005-09-15 at 16:38 -0400, Patrick Lam wrote:> Matthias Clasen wrote: > > Looking at the gnome-font-properties problem some more, I see > > XftFontMatch() returning NULL. Inside XftFontMatch(), all seems > > well until the call to XftDefaultSubstitute() which returns a > > pattern where the "style" value is set to FcTrue, which later upsets > > FcCompareValueList. > > I''ve fixed the gnome-terminal problem the right way now, at least for my > system (you''ll have to rebuild all your caches). gnome-font-properties > still doesn''t work. Thanks for this tip, I''ll look into it presently > and hopefully will have a fix soon. > > pat >Ok, I poked some more at this, and it seems to be a problem with FcNameRegisterObjectTypes() and/ or FcNameGetObjectType(). What is happening is that Xft calls FcNameRegisterObjectTypes() to register some additional names, "render" being the third one. There seems to be some confusion wrt to the assignment of ids to the registered strings, at least we end up with "render" from the Xft strings getting id 2, which is the same id as "style" from the FC registered strings got. This has then the effect that setting "render" to TRUE sets "style" instead, which is bad, since "style" is not a boolean... Matthias
Matthias Clasen wrote:> Looking at the gnome-font-properties problem some more, I see > XftFontMatch() returning NULL. Inside XftFontMatch(), all seems > well until the call to XftDefaultSubstitute() which returns a > pattern where the "style" value is set to FcTrue, which later upsets > FcCompareValueList.I''ve fixed the gnome-terminal problem the right way now, at least for my system (you''ll have to rebuild all your caches). gnome-font-properties still doesn''t work. Thanks for this tip, I''ll look into it presently and hopefully will have a fix soon. pat
On Sun, 2005-09-11 at 08:46 -0400, Patrick Lam wrote:> Matthias Clasen wrote: > > On Sun, 2005-09-11 at 01:17 -0400, Patrick Lam wrote: > > > >>Matthias Clasen wrote: > >> > >>>Here is another problem I met after installing the mmap branch > >>>fontconfig system-wide on my system. gnome-terminal doesn''t > >>>seem to find fonts anymore. I haven''t yet tried to follow all the > >>>fontconfig pattern manipulation done in vte, but it seems to > >>>trigger an incompatibility. > >>> > >>>A similar problem seems to affect the previews in the font capplet, > >>>which also seem to pick up a very small font size. > >> > >>I have fixed this problem. Let me know if that clears up the problem > >>for you. > > > > No, I still see the same problems. gnome-terminal and > > gnome-font-properties come up with fonts of very small size, and > > firefox segfaults. > > I''m definitely using the fontconfig branch on this computer with > gnome-terminal and firefox, which both work. (Also thunderbird). > However, gnome-font-properties currently doesn''t work for me -- it > reports no fonts found -- so I''ll look into that. (I thought that my > patch got it working last night). > > pat >Looking at the gnome-font-properties problem some more, I see XftFontMatch() returning NULL. Inside XftFontMatch(), all seems well until the call to XftDefaultSubstitute() which returns a pattern where the "style" value is set to FcTrue, which later upsets FcCompareValueList. Matthias
On Mon, 2005-10-03 at 22:17 -0400, Patrick Lam wrote:> Matthias Clasen wrote: > > > > Actually, thinking about FcDirCacheValid some more, I don''t see how the > > single per-file timestamp can work at all with multi-platform cache > > files. Imagine that a new font gets installed in a directory with a > > multi-platform cache. If you then run fc-cache on i386, it''ll update > > only the i386 section of the cache, but update the timestamp, marking > > the whole cache file as uptodate, so a later run on x86_64 will not > > update the x86_64 segment. I think the only solution to this is to put > > per-segment timestamps into the file, and use them to determine > > cache-outdatedness on a per-platform basis. Or simply use one cache file > > per platform... > > Oops, you''re right. How about this: when we find that the cache is > invalidated on any platform, we remove all stale sections from other > platforms. That ought to be sufficient to avoid staleness (safety). Of > course, an update to a file which is not stale preserves existing > sections (liveness). With this solution, in order to fully update cache > files, you''ll have to run fc-cache on all relevant platforms once, which > is the best we can do (without cross-platform cache creation) anyway.Sounds like it should work well enough (since fonts are not installed with high frequency...) Matthias
Matthias Clasen wrote:> Just trying this again, it worked when copying an x86_64 cache to a i386 > machine. Let me try the other way around again. I notice though, that > FcDirCacheValid() needs to become a bit smarter. It should not consider > a cache valid if it doesn''t contain a section for the current platform. > Otherwise, fc-cache may fail to add a section for the current plaform, > since the cache file appears uptodate, timestamp-wise.Great. It ought to work the other way, if it works one way. But let me know what you find out. Good catch on FcDirCacheValid. I''ve just committed a fix for that which will be in 2.3.92. (Your mail arrived just a minute after I posted 2.3.91. Oh well.) Could you test that out too? pat
On Tue, 2005-10-04 at 20:42 -0400, Patrick Lam wrote:> Matthias Clasen wrote: > >>Oops, you''re right. How about this: when we find that the cache is > >>invalidated on any platform, we remove all stale sections from other > >>platforms. That ought to be sufficient to avoid staleness (safety). Of > >>course, an update to a file which is not stale preserves existing > >>sections (liveness). With this solution, in order to fully update cache > >>files, you''ll have to run fc-cache on all relevant platforms once, which > >>is the best we can do (without cross-platform cache creation) anyway. > > > > Sounds like it should work well enough (since fonts are not installed > > with high frequency...) > > I just committed a patch which unlinks the file if it''s stale; if it''s > not stale but just lacking a section, then it adds the section to the > cache. So I reverted the patch to FcDirCacheValid and added > FcDirCacheHasCurrentArch. > > So, if FcDirCacheValid && FcDirCacheHasCurrentArch, then skip. > > Otherwise, !FcDirCacheValid || !FcDirCacheHasCurrentArch. If > !FcDirCacheValid (ie stale), then unlink. In all cases, continue by > saving the cache.I think you need to copy the other architectures into the new file and unlink the old one; there''s no other way to avoid destroying existing running applications images of the cache file. Oh, would resetting the mod time on the cache file be useful in avoiding thrashing among multiple architectures? By checking whether the existing timestamp was "new enough", and re-using that value for the new file, we would then just always use the cache file timestamp itself. -keith -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part Url : http://lists.freedesktop.org/archives/fontconfig/attachments/20051004/9c934bea/attachment.pgp
Le mercredi 05 octobre 2005 ? 09:01 -0400, Matthias Clasen a ?crit :> On Wed, 2005-10-05 at 09:14 +0200, Josselin Mouette wrote: > > In this case, this is a serious violation of the Filesystem Hierarchy > > Standard. The cache files lie in /usr/share, in which case they should > > be strictly platform-independent. > > > > Would it be possible to generate a system-wide cache and to put it > > somewhere in /var/cache instead? > > The important thing is that the same cache file can be used across > platforms. Whether that is done by storing the data in a > platform-independent way, or by storing multiple platform-dependent > copies of the same data in different sections of the file shouldn''t make > any difference.Sorry, but I strongly disagree here. You cannot call such a file "platform-independent". This goes completely against the spirit of the FHS, and against the Unix way of doing things. Will you tell me how, in a real-world configuration, you''re going to automate cache generation on a /usr/share directory shared between several platforms? This approach is perfectly suitable for the user-generated file in $HOME, but it isn''t acceptable for the system directories. Even the fonts.cache-1 approach is quite questionable, as you always end up with stale cache files not registered by the packaging system and never removed. -- .''''`. Josselin Mouette /\./\ : :'' : josselin.mouette@ens-lyon.org `. `'' joss@debian.org `- Debian GNU/Linux -- The power of freedom
Le jeudi 22 septembre 2005 ? 23:00 -0400, Matthias Clasen a ?crit :> So this means that the file format is not endianness- and > platform-independent, but contains enough information to load it on > other platforms ?In this case, this is a serious violation of the Filesystem Hierarchy Standard. The cache files lie in /usr/share, in which case they should be strictly platform-independent. Would it be possible to generate a system-wide cache and to put it somewhere in /var/cache instead? -- .''''`. Josselin Mouette /\./\ : :'' : josselin.mouette@ens-lyon.org `. `'' joss@debian.org `- Debian GNU/Linux -- The power of freedom
On Wed, 2005-10-05 at 15:59 +0200, Josselin Mouette wrote:> This approach is perfectly suitable for the user-generated file in > $HOME, but it isn''t acceptable for the system directories. Even the > fonts.cache-1 approach is quite questionable, as you always end up with > stale cache files not registered by the packaging system and never > removed.Yeah, the fact that you end up with files that don''t get automatically removed at appropriate times is clearly an issue -- remove a bunch of fonts and the directories never go away by themselves. I think the architectural neutrality isn''t the real issue here, but rather the lack of formal package management for these files. I suggest that allow each configured font directory to be mirrored into the /var/cache hierarchy and store the cache files there; essentially doing a directory prefix mapping (/usr/share/fonts -> /var/cache/fontconfig) and appending the rest of the path normally. This will allow the existing naming mechanisms to be used as-is. Directories which do not have such a mapping (~/.fonts) will be handled as they are today, with cache files stored in the directories themselves. Should I give this a try? Does it seem like a good solution? If we do this, what would you think about removing the whole /var/cache/fontconfig heirarchy when fontconfig was purged? -keith -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part Url : http://lists.freedesktop.org/archives/fontconfig/attachments/20051005/2afcc056/attachment.pgp
Matthias Clasen wrote:> When compiling the mmap branch on x86_64, I get warnings about > the %4x format used for sizeof() in fccache.c, since sizeof() > apparently returns unsigned long int here. One way to fix it > would be to use %4zx (but I don''t know how widespread support > for the z modifier is), or just cast the sizeof() values to > unsigned int.Thanks. I''ve committed a bunch of casts to unsigned int, which shouldn''t really be a problem for sizeof() results for the forseeable future. pat
Matthias Clasen wrote:> Ok, I have finally gotten around to compile the mmap branch on x86_64, > and tried to see how caches work across platforms. I copied > /usr/share/fonts/bitstream-vera/fonts.cache-2 from i386 over, and ran > fc-cache -f /usr/share/fonts/bitstream-vera/fonts.cache-2 > > It did in fact update the fonts.cache-2 file, but the result did only > have one section for x86_64, the i386 section was gone. Am I missing > something ? How am I supposed to create a multi-platform fonts.cache-2 > file ?That''s the proper way to create a multi-platform fonts.cache-2 file, but it must be a bug; I''ve changed the code a lot and haven''t necessarily tested it. I''ll look into it later today. pat
Josselin Mouette wrote:> Sorry, but I strongly disagree here. You cannot call such a file > "platform-independent". This goes completely against the spirit of the > FHS, and against the Unix way of doing things. Will you tell me how, in > a real-world configuration, you''re going to automate cache generation on > a /usr/share directory shared between several platforms?Once the cache file is properly initialized on all platforms, then it sure looks like a platform-independent file to me. For cache generation, it''s just a matter of running fc-cache a number of times, rather than just once. It''s also not like fontconfig will die if a cache file is not properly initialized for a particular architecture; it will then just cache the fonts in the local cache file. pat
On Mon, 2005-10-03 at 15:51 -0400, Patrick Lam wrote:> Matthias Clasen wrote: > > Just trying this again, it worked when copying an x86_64 cache to a i386 > > machine. Let me try the other way around again. I notice though, that > > FcDirCacheValid() needs to become a bit smarter. It should not consider > > a cache valid if it doesn''t contain a section for the current platform. > > Otherwise, fc-cache may fail to add a section for the current plaform, > > since the cache file appears uptodate, timestamp-wise. > > Great. It ought to work the other way, if it works one way. But let me > know what you find out. > > Good catch on FcDirCacheValid. I''ve just committed a fix for that which > will be in 2.3.92. (Your mail arrived just a minute after I posted > 2.3.91. Oh well.) Could you test that out too? >Seems to work both ways now. Matthias
On Mon, 2005-10-03 at 16:17 -0400, Matthias Clasen wrote:> On Mon, 2005-10-03 at 15:51 -0400, Patrick Lam wrote: > > Matthias Clasen wrote: > > > Just trying this again, it worked when copying an x86_64 cache to a i386 > > > machine. Let me try the other way around again. I notice though, that > > > FcDirCacheValid() needs to become a bit smarter. It should not consider > > > a cache valid if it doesn''t contain a section for the current platform. > > > Otherwise, fc-cache may fail to add a section for the current plaform, > > > since the cache file appears uptodate, timestamp-wise. > > > > Great. It ought to work the other way, if it works one way. But let me > > know what you find out. > > > > Good catch on FcDirCacheValid. I''ve just committed a fix for that which > > will be in 2.3.92. (Your mail arrived just a minute after I posted > > 2.3.91. Oh well.) Could you test that out too? > >Actually, thinking about FcDirCacheValid some more, I don''t see how the single per-file timestamp can work at all with multi-platform cache files. Imagine that a new font gets installed in a directory with a multi-platform cache. If you then run fc-cache on i386, it''ll update only the i386 section of the cache, but update the timestamp, marking the whole cache file as uptodate, so a later run on x86_64 will not update the x86_64 segment. I think the only solution to this is to put per-segment timestamps into the file, and use them to determine cache-outdatedness on a per-platform basis. Or simply use one cache file per platform... Matthias
Matthias Clasen wrote:> > Actually, thinking about FcDirCacheValid some more, I don''t see how the > single per-file timestamp can work at all with multi-platform cache > files. Imagine that a new font gets installed in a directory with a > multi-platform cache. If you then run fc-cache on i386, it''ll update > only the i386 section of the cache, but update the timestamp, marking > the whole cache file as uptodate, so a later run on x86_64 will not > update the x86_64 segment. I think the only solution to this is to put > per-segment timestamps into the file, and use them to determine > cache-outdatedness on a per-platform basis. Or simply use one cache file > per platform...Oops, you''re right. How about this: when we find that the cache is invalidated on any platform, we remove all stale sections from other platforms. That ought to be sufficient to avoid staleness (safety). Of course, an update to a file which is not stale preserves existing sections (liveness). With this solution, in order to fully update cache files, you''ll have to run fc-cache on all relevant platforms once, which is the best we can do (without cross-platform cache creation) anyway. pat
Matthias Clasen wrote:>>Oops, you''re right. How about this: when we find that the cache is >>invalidated on any platform, we remove all stale sections from other >>platforms. That ought to be sufficient to avoid staleness (safety). Of >>course, an update to a file which is not stale preserves existing >>sections (liveness). With this solution, in order to fully update cache >>files, you''ll have to run fc-cache on all relevant platforms once, which >>is the best we can do (without cross-platform cache creation) anyway. > > Sounds like it should work well enough (since fonts are not installed > with high frequency...)I just committed a patch which unlinks the file if it''s stale; if it''s not stale but just lacking a section, then it adds the section to the cache. So I reverted the patch to FcDirCacheValid and added FcDirCacheHasCurrentArch. So, if FcDirCacheValid && FcDirCacheHasCurrentArch, then skip. Otherwise, !FcDirCacheValid || !FcDirCacheHasCurrentArch. If !FcDirCacheValid (ie stale), then unlink. In all cases, continue by saving the cache. pat
Keith Packard wrote:> I think you need to copy the other architectures into the new file and > unlink the old one; there''s no other way to avoid destroying existing > running applications images of the cache file. > > Oh, would resetting the mod time on the cache file be useful in avoiding > thrashing among multiple architectures? By checking whether the existing > timestamp was "new enough", and re-using that value for the new file, we > would then just always use the cache file timestamp itself.What about if we used FcAtomic to write the directory cache files? Then any apps using the old cache files will just have a descriptor pointing to the old file and won''t be affected by writes to a new cache file and the unlinking of the old cache file. Does that work properly over networked file systems? I think that the proper thing to do is to always create a new file. If the rest of the data is not stale, then copy other segments from old file to new file. Then unlink the old file. I don''t think resetting the mod time would suffice: that doesn''t tell you that you even updated the current section, so it seems to me that you just have no clue. pat
Patrick Lam wrote:> What about if we used FcAtomic to write the directory cache files? Then > any apps using the old cache files will just have a descriptor pointing > to the old file and won''t be affected by writes to a new cache file and > the unlinking of the old cache file. Does that work properly over > networked file systems?Indeed, we already do; I was looking at an old version when I wrote this email. I think that ought to work. pat
On Wed, 2005-10-05 at 09:14 +0200, Josselin Mouette wrote:> Le jeudi 22 septembre 2005 ? 23:00 -0400, Matthias Clasen a ?crit : > > So this means that the file format is not endianness- and > > platform-independent, but contains enough information to load it on > > other platforms ? > > In this case, this is a serious violation of the Filesystem Hierarchy > Standard. The cache files lie in /usr/share, in which case they should > be strictly platform-independent. > > Would it be possible to generate a system-wide cache and to put it > somewhere in /var/cache instead?The important thing is that the same cache file can be used across platforms. Whether that is done by storing the data in a platform-independent way, or by storing multiple platform-dependent copies of the same data in different sections of the file shouldn''t make any difference.
On Wed, 2005-10-05 at 09:14 +0200, Josselin Mouette wrote:> Le jeudi 22 septembre 2005 ? 23:00 -0400, Matthias Clasen a ?crit : > > So this means that the file format is not endianness- and > > platform-independent, but contains enough information to load it on > > other platforms ? > > In this case, this is a serious violation of the Filesystem Hierarchy > Standard. The cache files lie in /usr/share, in which case they should > be strictly platform-independent.The files are poly-platform as they can contain per-platform sections, so they can live outside of /var without trouble.> Would it be possible to generate a system-wide cache and to put it > somewhere in /var/cache instead?The set of directories referenced by each user is different (each user has a per-user ~/.fonts directory in the default configuration), so a single large cache file cannot be used. And, because each user has one (or more) private directories, we can''t insist that all cache files live in a non user-writable area. On the other hand, there have been a few requests to make it possible to use cache files that live outside the font directories. I''ve been reluctant because of the obvious naming issues (given multiple cache files, and multiple paths to each directory, coming up with appropriate names is tricky). -keith -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part Url : http://lists.freedesktop.org/archives/fontconfig/attachments/20051005/364f9df7/attachment.pgp
Matthias Clasen wrote:> I looked a bit more into my problems, and found that firefox crashes > with an assertion inside pango, if the FONTCONFIG_PATH is set (even if > it is set to the default value /etc/fonts). The assertion I run into is > that a FcPatternGetString (pattern, FC_FAMILY, 0, (FcChar8 **) &s) > returns FcResultTypeMismatch for one pattern. And in fact, if I print > the pattern, it looks as if the family name is mistaken as a boolean:Does this problem still occur? I haven''t been able to reproduce it. pat
On Wed, 2005-09-28 at 01:14 -0400, Patrick Lam wrote:> Matthias Clasen wrote: > > I looked a bit more into my problems, and found that firefox crashes > > with an assertion inside pango, if the FONTCONFIG_PATH is set (even if > > it is set to the default value /etc/fonts). The assertion I run into is > > that a FcPatternGetString (pattern, FC_FAMILY, 0, (FcChar8 **) &s) > > returns FcResultTypeMismatch for one pattern. And in fact, if I print > > the pattern, it looks as if the family name is mistaken as a boolean: > > Does this problem still occur? I haven''t been able to reproduce it. > > patNo, I haven''t seen it recently. I would assume its fixed until somebody sees it again with >= 2.3.90. Matthias
On Thu, 2005-09-22 at 23:05 -0400, Patrick Lam wrote:> Matthias Clasen wrote: > > So this means that the file format is not endianness- and > > platform-independent, but contains enough information to load it on > > other platforms ? Does the current loader support loading cache files > > from other platforms ? > > > > I guess I should just test that... > > It can''t possibly be platform-independent if it''s to be mmapped... > otherwise you end up making another copy of the data before you can use > it, which eliminates the whole space savings benefit. > > I''ve tested it on multiple platforms before, but please test it again. > It was designed to work on multiple platforms, by keeping one copy per > platform (all the copies are stored in the same file, but in different > sections). > > patOk, I have finally gotten around to compile the mmap branch on x86_64, and tried to see how caches work across platforms. I copied /usr/share/fonts/bitstream-vera/fonts.cache-2 from i386 over, and ran fc-cache -f /usr/share/fonts/bitstream-vera/fonts.cache-2 It did in fact update the fonts.cache-2 file, but the result did only have one section for x86_64, the i386 section was gone. Am I missing something ? How am I supposed to create a multi-platform fonts.cache-2 file ? Regards, Matthias
When compiling the mmap branch on x86_64, I get warnings about the %4x format used for sizeof() in fccache.c, since sizeof() apparently returns unsigned long int here. One way to fix it would be to use %4zx (but I don''t know how widespread support for the z modifier is), or just cast the sizeof() values to unsigned int. Matthias
On Wed, 2005-09-28 at 13:26 -0400, Patrick Lam wrote:> Matthias Clasen wrote: > > Ok, I have finally gotten around to compile the mmap branch on x86_64, > > and tried to see how caches work across platforms. I copied > > /usr/share/fonts/bitstream-vera/fonts.cache-2 from i386 over, and ran > > fc-cache -f /usr/share/fonts/bitstream-vera/fonts.cache-2 > > > > It did in fact update the fonts.cache-2 file, but the result did only > > have one section for x86_64, the i386 section was gone. Am I missing > > something ? How am I supposed to create a multi-platform fonts.cache-2 > > file ? > > That''s the proper way to create a multi-platform fonts.cache-2 file, but > it must be a bug; I''ve changed the code a lot and haven''t necessarily > tested it. I''ll look into it later today. > > patJust trying this again, it worked when copying an x86_64 cache to a i386 machine. Let me try the other way around again. I notice though, that FcDirCacheValid() needs to become a bit smarter. It should not consider a cache valid if it doesn''t contain a section for the current platform. Otherwise, fc-cache may fail to add a section for the current plaform, since the cache file appears uptodate, timestamp-wise. Matthias
Matthias Clasen wrote:> Ok, I have finally gotten around to compile the mmap branch on x86_64, > and tried to see how caches work across platforms. I copied > /usr/share/fonts/bitstream-vera/fonts.cache-2 from i386 over, and ran > fc-cache -f /usr/share/fonts/bitstream-vera/fonts.cache-2 > > It did in fact update the fonts.cache-2 file, but the result did only > have one section for x86_64, the i386 section was gone. Am I missing > something ? How am I supposed to create a multi-platform fonts.cache-2 > file ?I take it you were using the branch code, not 2.3.90. I introduced a bug into the branch while introducing atomic updates to cache files. I''ve now fixed this bug; let me know if it works now. pat
>>>>> "Keith" == Keith Packard <keithp@keithp.com> writes:Keith> I suggest that allow each configured font directory to be Keith> mirrored into the /var/cache hierarchy and store the cache Keith> files there; essentially doing a directory prefix mapping Keith> (/usr/share/fonts -> /var/cache/fontconfig) and appending Keith> the rest of the path normally. This will allow the existing Keith> naming mechanisms to be used as-is. Perhaps it should be $FOO -> /var/cache/fontconfig/$FOO so that all of the various paths can be handled. At least /usr/local/share/fonts and /local/share/fonts will get used and should be supported. I wonder though how much of a performance hit we''ll see by abusing the kernels'' filesystem caches by doubling the number if directories searched each time a fontconfig-linked program starts? As an example, given a reasonably large number of font directories (each dist package has its own under /usr/share/fonts, I tend to keep my ~/.fonts organized like that, etc) and the large number of libraries most GUI apps link against these days whenever I emerge anything on my gentoo laptop with X running it takes minutes each time ldconfig is run versus deciseconds when X is not running. Keith> If we do this, what would you think about removing the whole Keith> /var/cache/fontconfig heirarchy when fontconfig was purged? Probably should happen; even if it is being purged just to make sure a re-install is clean the user probably wants to regenerate all of the cache files. But that should be an issue for the packager rather than for the developer. -JimC -- James H. Cloos, Jr. <cloos@jhcloos.com>
Le mercredi 05 octobre 2005 ? 10:44 -0700, Keith Packard a ?crit :> I think the architectural neutrality isn''t the real issue here, but > rather the lack of formal package management for these files.Both are issues. The need to generate cache files for all architectures is a waste of disk space and doesn''t account for the way packaging systems work.> I suggest that allow each configured font directory to be mirrored into > the /var/cache hierarchy and store the cache files there; essentially > doing a directory prefix mapping (/usr/share/fonts > -> /var/cache/fontconfig) and appending the rest of the path normally. > This will allow the existing naming mechanisms to be used as-is.It would indeed solve both problems simultaneously. Maybe, for performance, would it be better to use /var/cache/fontconfig/$hash.cache files, $hash being generated from the original pathname ?> Directories which do not have such a mapping (~/.fonts) will be handled > as they are today, with cache files stored in the directories > themselves.> Should I give this a try? Does it seem like a good solution? > > If we do this, what would you think about removing the > whole /var/cache/fontconfig heirarchy when fontconfig was purged?Yes, that would be mandated, just like for any /var/cache hierarchy. Regards, -- .''''`. Josselin Mouette /\./\ : :'' : josselin.mouette@ens-lyon.org `. `'' joss@debian.org `- Debian GNU/Linux -- The power of freedom -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part Url : http://lists.freedesktop.org/archives/fontconfig/attachments/20051009/4ddf1409/attachment.pgp
>>>>> "Keith" == Keith Packard <keithp@keithp.com> writes:Keith> You could do that by mapping / -> /var/cache/fontconfig Yup. But I''d expect is should be only for ''system'' dirs. (Those covered by fc-cache(1)''s --system-only flag.)>> But that should be an issue for the packager rather than for the developer.Keith> Yes, of course (he says as debian packager for fontconfig). Seems I failed a due diligence roll. ;^) -JimC -- James H. Cloos, Jr. <cloos@jhcloos.com>
On Wed, 2005-10-05 at 15:19 -0400, James Cloos wrote:> Perhaps it should be $FOO -> /var/cache/fontconfig/$FOO so that all > of the various paths can be handled. At least /usr/local/share/fonts > and /local/share/fonts will get used and should be supported.You could do that by mapping / -> /var/cache/fontconfig> I wonder though how much of a performance hit we''ll see by abusing > the kernels'' filesystem caches by doubling the number if directories > searched each time a fontconfig-linked program starts?The alternative is to attempt to construct a file naming scheme that would map directories onto unique filenames. I don''t see a trivial way to do that; we could use device-number/inode pairs or some such...> As an example, given a reasonably large number of font directories > (each dist package has its own under /usr/share/fonts, I tend to > keep my ~/.fonts organized like that, etc) and the large number of > libraries most GUI apps link against these days whenever I emerge > anything on my gentoo laptop with X running it takes minutes each > time ldconfig is run versus deciseconds when X is not running.yeah, reducing the amount of FS thrashing would be nice.> Keith> If we do this, what would you think about removing the whole > Keith> /var/cache/fontconfig heirarchy when fontconfig was purged? > > Probably should happen; even if it is being purged just to make sure a > re-install is clean the user probably wants to regenerate all of the > cache files. But that should be an issue for the packager rather than > for the developer.Yes, of course (he says as debian packager for fontconfig). -keith -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part Url : http://lists.freedesktop.org/archives/fontconfig/attachments/20051005/f7f35942/attachment.pgp
On Sun, 2005-10-09 at 00:39 +0200, Josselin Mouette wrote:> It would indeed solve both problems simultaneously. > Maybe, for performance, would it be better to > use /var/cache/fontconfig/$hash.cache files, $hash being generated from > the original pathname ?cryptographic hashes to the rescue. Yes, this will work quite nicely. -keith -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part Url : http://lists.freedesktop.org/archives/fontconfig/attachments/20051009/f3b7a778/attachment.pgp
Josselin Mouette wrote:> It would indeed solve both problems simultaneously. > Maybe, for performance, would it be better to > use /var/cache/fontconfig/$hash.cache files, $hash being generated from > the original pathname ?How does the attached patch look? Is fontconfig supposed to be responsible for creating /var/cache/fontconfig, or can it rely on the packager? Also is hardcoding /var/cache/fontconfig the right thing? pat -------------- next part -------------- Index: fontconfig/fontconfig.h ==================================================================RCS file: /cvs/fontconfig/fontconfig/fontconfig/fontconfig.h,v retrieving revision 1.69.2.15 diff -u -p -r1.69.2.15 fontconfig.h --- fontconfig/fontconfig.h 24 Nov 2005 21:40:20 -0000 1.69.2.15 +++ fontconfig/fontconfig.h 7 Dec 2005 04:00:31 -0000 @@ -104,6 +104,8 @@ typedef int FcBool; #define FC_EMBOLDEN "embolden" /* Bool - true if emboldening needed*/ #define FC_EMBEDDED_BITMAP "embeddedbitmap" /* Bool - true to enable embedded bitmaps */ +#define FC_VAR_CACHE_DIR "/var/cache/fontconfig/" +#define FC_CACHE_SUFFIX ".cache-"FC_CACHE_VERSION #define FC_DIR_CACHE_FILE "fonts.cache-"FC_CACHE_VERSION #define FC_USER_CACHE_FILE ".fonts.cache-"FC_CACHE_VERSION Index: src/fccache.c ==================================================================RCS file: /cvs/fontconfig/fontconfig/src/fccache.c,v retrieving revision 1.23.4.37 diff -u -p -r1.23.4.37 fccache.c --- src/fccache.c 29 Nov 2005 14:57:10 -0000 1.23.4.37 +++ src/fccache.c 7 Dec 2005 04:00:31 -0000 @@ -723,6 +723,8 @@ FcBool FcDirCacheRead (FcFontSet * set, FcStrSet * dirs, const FcChar8 *dir) { char *cache_file = (char *)FcStrPlus (dir, (FcChar8 *) "/" FC_DIR_CACHE_FILE); + char hash[9]; + char *cache_hashed; int fd; char * current_arch_machine_name; char candidate_arch_machine_name[9+MACHINE_SIGNATURE_SIZE]; @@ -732,10 +734,17 @@ FcDirCacheRead (FcFontSet * set, FcStrSe if (!cache_file) goto bail; + sprintf (hash, "%8x", FcStringHash ((FcChar8 *)cache_file)); + cache_hashed = (char *)FcStrPlus ((FcChar8 *)FC_VAR_CACHE_DIR, FcStrPlus ((FcChar8 *)hash, (FcChar8 *)FC_CACHE_SUFFIX)); + current_arch_machine_name = FcCacheMachineSignature(); - fd = open(cache_file, O_RDONLY); + fd = open(cache_hashed, O_RDONLY); if (fd == -1) - goto bail; + { + fd = open (cache_file, O_RDONLY); + if (fd == -1) + goto bail; + } current_arch_start = FcCacheSkipToArch(fd, current_arch_machine_name); if (current_arch_start < 0) @@ -845,6 +854,8 @@ FcBool FcDirCacheWrite (FcFontSet *set, FcStrSet *dirs, const FcChar8 *dir) { FcChar8 *cache_file = FcStrPlus (dir, (FcChar8 *) "/" FC_DIR_CACHE_FILE); + char hash[9]; + FcChar8 *cache_hashed; int fd, fd_orig, i, dirs_count; FcAtomic *atomic; FcCache metadata; @@ -856,6 +867,9 @@ FcDirCacheWrite (FcFontSet *set, FcStrSe if (!cache_file) goto bail; + sprintf (hash, "%8x", FcStringHash ((FcChar8 *)cache_file)); + cache_hashed = FcStrPlus ((FcChar8 *)FC_VAR_CACHE_DIR, FcStrPlus ((FcChar8 *)hash, (FcChar8 *)FC_CACHE_SUFFIX)); + current_dir_block = FcDirCacheProduce (set, &metadata); if (metadata.count && !current_dir_block) @@ -864,9 +878,13 @@ FcDirCacheWrite (FcFontSet *set, FcStrSe if (FcDebug () & FC_DBG_CACHE) printf ("FcDirCacheWriteDir cache_file \"%s\"\n", cache_file); - atomic = FcAtomicCreate (cache_file); + atomic = FcAtomicCreate (cache_hashed); if (!atomic) - goto bail0; + { + atomic = FcAtomicCreate (cache_file); + if (!atomic) + goto bail0; + } if (!FcAtomicLock (atomic)) goto bail1;
On Tue, 2005-12-06 at 23:04 -0500, Patrick Lam wrote:> Josselin Mouette wrote: > > It would indeed solve both problems simultaneously. > > Maybe, for performance, would it be better to > > use /var/cache/fontconfig/$hash.cache files, $hash being generated from > > the original pathname ? > > How does the attached patch look? Is fontconfig supposed to be > responsible for creating /var/cache/fontconfig, or can it rely on the > packager? Also is hardcoding /var/cache/fontconfig the right thing?You want to use a cryptographically secure hash to avoid collisions, as long as you survive collisions, you can use MD5, SHA1 or SHA256 as you please. -keith -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part Url : http://lists.freedesktop.org/archives/fontconfig/attachments/20051206/c0174a87/attachment.pgp
Le mardi 06 d?cembre 2005 ? 23:04 -0500, Patrick Lam a ?crit :> Josselin Mouette wrote: > > It would indeed solve both problems simultaneously. > > Maybe, for performance, would it be better to > > use /var/cache/fontconfig/$hash.cache files, $hash being generated from > > the original pathname ? > > How does the attached patch look?Great ! I don''t have enough knowledge of the fontconfig internals, but it looks simple enough to work. Although, I''d have the same remark as Keith about the hash: using a MD5 hash would be much better, as the code isn''t collision-safe.> Is fontconfig supposed to be > responsible for creating /var/cache/fontconfig, or can it rely on the > packager? Also is hardcoding /var/cache/fontconfig the right thing?I think you should use ${localstatedir}/cache/fontconfig instead. Furthermore, the makefiles should create this directory in DESTDIR in the install target. Packagers will either use this directory or create it themselves, this isn''t a problem. Regards, -- .''''`. Josselin Mouette /\./\ : :'' : josselin.mouette@ens-lyon.org `. `'' joss@debian.org `- Debian GNU/Linux -- The power of freedom
On Wed, 2005-12-07 at 09:40 +0100, Josselin Mouette wrote:> I think you should use ${localstatedir}/cache/fontconfig instead. > Furthermore, the makefiles should create this directory in DESTDIR in > the install target. Packagers will either use this directory or create > it themselves, this isn''t a problem.Yes, that way it will land in /var/cache/fontconfig where it belongs. -keith -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part Url : http://lists.freedesktop.org/archives/fontconfig/attachments/20051207/867b4dca/attachment.pgp
Josselin Mouette wrote:> Great ! I don''t have enough knowledge of the fontconfig internals, but > it looks simple enough to work. Although, I''d have the same remark as > Keith about the hash: using a MD5 hash would be much better, as the code > isn''t collision-safe.I''ve got a patch here which uses md5 and handles collisions.> I think you should use ${localstatedir}/cache/fontconfig instead. > Furthermore, the makefiles should create this directory in DESTDIR in > the install target. Packagers will either use this directory or create > it themselves, this isn''t a problem.How do I make ${localstatedir} available to the C source code? pat
> How do I make ${localstatedir} available to the C source code? > > patAdd to configure.{ac,in}: pkglocalstatedir=''${localstatedir}/lib/''${PACKAGE_NAME} AC_SUBST(pkglocalstatedir) and to src/Makefile.am: AM_CPPFLAGS = \ -DPKGLOCALSTATEDIR=''"${pkglocalstatedir}"'' --behdad http://behdad.org/ "Commandment Three says Do Not Kill, Amendment Two says Blood Will Spill" -- Dan Bern, "New American Language"
Le vendredi 09 d?cembre 2005 ? 01:17 -0500, Behdad Esfahbod a ?crit :> > How do I make ${localstatedir} available to the C source code? > > > > pat > > Add to configure.{ac,in}: > > pkglocalstatedir=''${localstatedir}/lib/''${PACKAGE_NAME} > AC_SUBST(pkglocalstatedir) > > > and to src/Makefile.am: > > AM_CPPFLAGS = \ > -DPKGLOCALSTATEDIR=''"${pkglocalstatedir}"''Which would translate here as: pkgcachedir=''${localstatedir}/cache/''${PACKAGE_NAME} AC_SUBST(pkgcachedir) AM_CPPFLAGS = -DPKGCACHEDIR=''"${pkgcachedir}"'' Regards, -- .''''`. Josselin Mouette /\./\ : :'' : josselin.mouette@ens-lyon.org `. `'' joss@debian.org `- Debian GNU/Linux -- The power of freedom
Josselin Mouette wrote:> Which would translate here as: > pkgcachedir=''${localstatedir}/cache/''${PACKAGE_NAME} > AC_SUBST(pkgcachedir) > > AM_CPPFLAGS = -DPKGCACHEDIR=''"${pkgcachedir}"''This doesn''t quite work. ${localstatedir} is ${prefix}/var ${prefix} is /usr so it''s looking for /usr/var/cache/fontconfig, which isn''t right. Should I use something other than ${localstatedir}? pat
Le vendredi 09 d?cembre 2005 ? 10:10 -0500, Patrick Lam a ?crit :> Josselin Mouette wrote: > > Which would translate here as: > > pkgcachedir=''${localstatedir}/cache/''${PACKAGE_NAME} > > AC_SUBST(pkgcachedir) > > > > AM_CPPFLAGS = -DPKGCACHEDIR=''"${pkgcachedir}"'' > > This doesn''t quite work. > > ${localstatedir} is ${prefix}/var > ${prefix} is /usr > > so it''s looking for /usr/var/cache/fontconfig, which isn''t right. > > Should I use something other than ${localstatedir}?It''s just like when you forget to set --sysconfdir=/etc and end up with configuration in /etc. It''s just autoconf being brain-dead. Regards, -- .''''`. Josselin Mouette /\./\ : :'' : josselin.mouette@ens-lyon.org `. `'' joss@debian.org `- Debian GNU/Linux -- The power of freedom
I''ve committed my patch to replace /fontdir/fonts.cache-2 with /var/cache/fontconfig/(md5-hash).cache-2. All caches need to be rebuilt in their new locations. I forgot to make the Makefile create the /var/cache/fontconfig directory though. Can someone send a patch to do that? Also if anyone has any comments on this new approach, it would be good to send them to the list now. pat