Fontconfig - Nov 2008 - Interaction of (web) document font families and fontconfig rules

I''m looking for comments on how fontconfig users would prefer font
families defined in web documents through @font-face (or defined
by other means in other documents) to be presented in FcPattern *p
for FcConfigSubstitute(FcConfig*, p, FcMatchPattern).

The question basically comes down to whether
author-defined/provided fonts should be treated in the same way as
locally installed fonts, or whether they should be considered
special.


Locally installed fonts only:
----------------------------

For web documents using only locally installed font families
through CSS 2.1, there is a reasonably well defined algorithm for
prioritizing fonts according to the font-family property, which is
a list of family names, and other style properties.

In the spirit of giving the user the final say, the CSS properties
are converted to an FcPattern and presented to
FcConfigSubstitute(,,FcMatchPattern), and the result is used in
the equivalent of an FcFontSetSort.

For example, with

  <p lang="en" style="font-family: Helvetica,
sans-serif">
    Paragraph.
  </p>

After FcConfigSubstitute, the FC_FAMILY property for the element
might be something like "Helvetica(s), Nimbus Sans L(s), Bitstream
Vera Sans(w)" (followed by several other family names).  Here
Nimbus Sans L is a possible close-match to the author-requested
Helvetica (for use when Helvetica is not installed), and
Bitstream Vera Sans is the default font for lang="en".  These
families are considered in this order when finding glyphs for
characters.


@font-face:
----------

What the @font-face CSS rule adds is the ability for the author to describe
the properties of a automatically-downloadable font face and the family to
which it belongs.  This font face is only available to the document(s) that
use this style rule.  The following web page contains an example, which uses
the BPG Ucnobi U font:

http://www.w3.org/International/tests/test-webfonts/test-georgian-ucnobi.html

Consider now if the author had included this style rule:

 @font-face {
    font-family: "BPG Ucnobi U";
    font-style:  normal;
    font-weight: normal;
    src:
url(http://www.w3.org/International/tests/test-webfonts/BPG-Ucnobi-U.ttf);
 }

Note that the value of the font-family descriptor here may or may
not be the same as the family name in the font referenced by the
src descriptor.  It is the value in the @font-face rule, not the
name from the font, that is matched against any HTML element''s
font-family style properties, when deciding whether to use this
font-face.  Equivalently the family name in font pattern
constructed to represent this face comes from the @font-face rule
not the font.

(In the @font-face rule above the descriptor value is the same as
the font family name, but the page referenced above actually used
a different font-family value.)

Now consider if a user has the following fontconfig rule:

        <alias>
                <family>sans-serif</family>
                <prefer>
                        <family>BPG Ucnobi U</family>
                </prefer>
        </alias>

so that FcConfigSubstitute turns font-family:sans-serif into
"Bitstream Vera Sans(w), BPG Ucnobi U(w)", which roughly means use
Bitstream Vera Sans for Latin languages and BPG Ucnobi U for the
Georgian language.

Now, if no BPG Ucnobi U font is installed locally, and a page
includes an @font-face rule with family BPG Ucnobi U, should the
element

 <p lang="ka"
style="font-family:sans-serif">????????</p>

be rendered with the font from
http://www.w3.org/International/tests/test-webfonts/BPG-Ucnobi-U.ttf
or with a locally installed fallback font?

And, if there were a locally installed version of BPG Ucnobi U,
how would the user choose between the document font and the
locally installed font?  (Currently this is decided by the web
browser.  The spec is changing and currently contradictory.)


Discussion:
----------

Part of me would like to think that document fonts are just like
any other fonts but I think there are differences.

* The user has control over locally installed fonts, and can set
  up fontconfig rules appropriately.  Document fonts change as
  often as users view different documents and do not get parsed by
  acceptfont/rejectfont rules.

* Document fonts may not be what they seem.

  * The font-family descriptor may have little to do with the font
    itself.

  * The document font may be a trimmed down version of the font
    with only enough glyphs to display a heading, for example.


Document fonts are already treated differently in that they are
not considered for fallback when looking for support for
characters not supported by the specified fonts.
(The draft spec http://dev.w3.org/csswg/css3-fonts/#font-matching
is changing but currently says "the user agent performs a system
font fallback procedure to find the best match for the character
to be rendered".)  I imagine the reason for not considering
document fonts is that the author may (dare I mention it) play
tricks using Unicode code point assignments for unrelated glyphs.


My current thinking is that the FC_FAMILY value in the font
pattern representing fonts from document @font-face rules would
have a namespace. e.g.

  family: "@font-face:BPG Ucnobi U"

and, when the CSS style for an HTML element requests
"font-family:BPG Ucnobi U" (and an @font-face rule exists), the
FC_FAMILY property of the pattern passed to
FcConfigSubstitute(,,FcMatchPattern) would include
"@font-face:BPG Ucnobi U".

Le Mer 19 novembre 2008 03:29, Karl Tomlinson a ?crit :
> Discussion:
> ----------
>
> Part of me would like to think that document fonts are just like
> any other fonts but I think there are differences.
>
> * The user has control over locally installed fonts, and can set
>   up fontconfig rules appropriately.  Document fonts change as
>   often as users view different documents and do not get parsed by
>   acceptfont/rejectfont rules.
>
> * Document fonts may not be what they seem.
>
>   * The font-family descriptor may have little to do with the font
>     itself.
>
>   * The document font may be a trimmed down version of the font
>     with only enough glyphs to display a heading, for example.
Since you''re looking at this, please consider:

1. it is highly possible @font-face will generate blocking extensions
like noscripts or flashblock eventually

2. there will soon be a third font category, system fonts installed on
demand
http://fedoraproject.org/wiki/Features/AutomaticFontInstallation
It is a potential competitor of @font-face, which will work in every
application of the system, not just the browser. And it will keep the
user in control.

However, any automated font installation system will involve a
resolution and downloading step which will be more expensive than a
simple local call (or a simple "download this blackbox file, trust
me"), so the associated fonts probably can''t be treated just as
system
fonts, and apps must decide when this cost is worth it.

Deciding how to priorize or even blacklist @font-face fonts, system
fonts and auto-installed fonts is going to be fun.

-- 
Nicolas Mailhot

Another consideration to keep in mind is that many users configure their
browsers to ignore the documents'' font specifications and instead use a
(set of) local fonts they prefer.

Browsers must not lose that ability when adding support for @font-face.

-JimC
-- 
James Cloos <cloos at jhcloos.com>         OpenPGP: 1024D/ED7DAEA6

Hi Karl,

Sorry to take so long to answer.

Karl Tomlinson wrote:
> I''m looking for comments on how fontconfig users would prefer font
> families defined in web documents through @font-face (or defined
> by other means in other documents) to be presented in FcPattern *p
> for FcConfigSubstitute(FcConfig*, p, FcMatchPattern).
> 
> The question basically comes down to whether
> author-defined/provided fonts should be treated in the same way as
> locally installed fonts, or whether they should be considered
> special.
My take on it is that they should be considered special.  And that''s
the API
I''m going to implement in Pango.  Actually the proposed Pango API is
very
close to CSS.  Or I think it is.  Anyway, the bug to track it is here:

  http://bugzilla.gnome.org/show_bug.cgi?id=347237

> Locally installed fonts only:
> ----------------------------
> 
> For web documents using only locally installed font families
> through CSS 2.1, there is a reasonably well defined algorithm for
> prioritizing fonts according to the font-family property, which is
> a list of family names, and other style properties.
> 
> In the spirit of giving the user the final say, the CSS properties
> are converted to an FcPattern and presented to
> FcConfigSubstitute(,,FcMatchPattern), and the result is used in
> the equivalent of an FcFontSetSort.
> 
> For example, with
> 
>   <p lang="en" style="font-family: Helvetica,
sans-serif">
>     Paragraph.
>   </p>
> 
> After FcConfigSubstitute, the FC_FAMILY property for the element
> might be something like "Helvetica(s), Nimbus Sans L(s), Bitstream
> Vera Sans(w)" (followed by several other family names).  Here
> Nimbus Sans L is a possible close-match to the author-requested
> Helvetica (for use when Helvetica is not installed), and
> Bitstream Vera Sans is the default font for lang="en".  These
> families are considered in this order when finding glyphs for
> characters.
You do a FcConfigSubstitute(,,FcMatchFont) in the resulting fonts too, right?

> @font-face:
> ----------
> 
> What the @font-face CSS rule adds is the ability for the author to describe
> the properties of a automatically-downloadable font face and the family to
> which it belongs.  This font face is only available to the document(s) that
> use this style rule.  The following web page contains an example, which
uses
> the BPG Ucnobi U font:
> 
>
http://www.w3.org/International/tests/test-webfonts/test-georgian-ucnobi.html
> 
> Consider now if the author had included this style rule:
> 
>  @font-face {
>     font-family: "BPG Ucnobi U";
>     font-style:  normal;
>     font-weight: normal;
>     src:
url(http://www.w3.org/International/tests/test-webfonts/BPG-Ucnobi-U.ttf);
>  }
> 
> Note that the value of the font-family descriptor here may or may
> not be the same as the family name in the font referenced by the
> src descriptor.  It is the value in the @font-face rule, not the
> name from the font, that is matched against any HTML element''s
> font-family style properties, when deciding whether to use this
> font-face.  Equivalently the family name in font pattern
> constructed to represent this face comes from the @font-face rule
> not the font.
> 
> (In the @font-face rule above the descriptor value is the same as
> the font family name, but the page referenced above actually used
> a different font-family value.)
> 
> Now consider if a user has the following fontconfig rule:
> 
>         <alias>
>                 <family>sans-serif</family>
>                 <prefer>
>                         <family>BPG Ucnobi U</family>
>                 </prefer>
>         </alias>
> 
> so that FcConfigSubstitute turns font-family:sans-serif into
> "Bitstream Vera Sans(w), BPG Ucnobi U(w)", which roughly means
use
> Bitstream Vera Sans for Latin languages and BPG Ucnobi U for the
> Georgian language.
> 
> Now, if no BPG Ucnobi U font is installed locally, and a page
> includes an @font-face rule with family BPG Ucnobi U, should the
> element
> 
>  <p lang="ka"
style="font-family:sans-serif">????????</p>
> 
> be rendered with the font from
> http://www.w3.org/International/tests/test-webfonts/BPG-Ucnobi-U.ttf
> or with a locally installed fallback font?
I''d say the locally installed font.

> And, if there were a locally installed version of BPG Ucnobi U,
> how would the user choose between the document font and the
> locally installed font?  (Currently this is decided by the web
> browser.  The spec is changing and currently contradictory.)
The document one used if user specified that font family explicitly.  The
system one used as a result of fontconfig configuration.  I always think of
the @font-face families as living in separate namespace.

> Discussion:
> ----------
> 
> Part of me would like to think that document fonts are just like
> any other fonts but I think there are differences.
> 
> * The user has control over locally installed fonts, and can set
>   up fontconfig rules appropriately.  Document fonts change as
>   often as users view different documents and do not get parsed by
>   acceptfont/rejectfont rules.
> 
> * Document fonts may not be what they seem.
> 
>   * The font-family descriptor may have little to do with the font
>     itself.
> 
>   * The document font may be a trimmed down version of the font
>     with only enough glyphs to display a heading, for example.
Exactly.  A good way to think about them is like fonts embedded in a PDF, or a
SVG.  Do you expect your PDF/SVG.  That makes things much easier.

> Document fonts are already treated differently in that they are
> not considered for fallback when looking for support for
> characters not supported by the specified fonts.
> (The draft spec http://dev.w3.org/csswg/css3-fonts/#font-matching
> is changing but currently says "the user agent performs a system
> font fallback procedure to find the best match for the character
> to be rendered".)  I imagine the reason for not considering
> document fonts is that the author may (dare I mention it) play
> tricks using Unicode code point assignments for unrelated glyphs.
> 
> 
> My current thinking is that the FC_FAMILY value in the font
> pattern representing fonts from document @font-face rules would
> have a namespace. e.g.
> 
>   family: "@font-face:BPG Ucnobi U"
Exactly my thinking in the context of how this will be used with Pango.

> and, when the CSS style for an HTML element requests
> "font-family:BPG Ucnobi U" (and an @font-face rule exists), the
> FC_FAMILY property of the pattern passed to
> FcConfigSubstitute(,,FcMatchPattern) would include
> "@font-face:BPG Ucnobi U".
You sure can do that.  But I''m not sure how useful that would be.

behdad

On Wed, 19 Nov 2008 10:14:21 +0100 (CET), Nicolas Mailhot wrote:
> Since you''re looking at this, please consider:
>
> 1. it is highly possible @font-face will generate blocking extensions
> like noscripts or flashblock eventually
In Mozilla, @font-face can be disabled completely by toggling
gfx.downloadable_fonts.enabled and restarting the browser.

But yes, extensions to control use of src:url() on a per site
basis may become desirable.

On Thu, 20 Nov 2008 00:26:44 -0500, James Cloos wrote:
> Another consideration to keep in mind is that many users configure their
> browsers to ignore the documents'' font specifications and instead
use a
> (set of) local fonts they prefer.
>
> Browsers must not lose that ability when adding support for @font-face.
In Mozilla, @font-face does not remove the ability for users to
disable author fonts (on Linux, at least - I haven''t checked
implementations for other platforms).

This is still available through either of the following:

 Edit -> Preferences -> Content -> Advanced ->
 "Allow pages to choose their own fonts, instead of my selections
above."

 View -> Page Style -> No Style

On Fri, 21 Nov 2008 00:08:36 +0100, Juliusz Chroboczek wrote:
>> I''m looking for comments on how fontconfig users would prefer
font
>> families defined in web documents through @font-face (or defined
>> by other means in other documents) to be presented in FcPattern *p
>> for FcConfigSubstitute(FcConfig*, p, FcMatchPattern).
>
> They should most certainly be treated specially.
>
> Consider the following.  There are googols of fonts called
``Garamond''''.
> If two different web pages provide a font called Garamond, there''s
no
> reason to conclude that it''s the same font; one could be an
authentic
> French Renaissance font, while the other could be a stylistically very
> different 20th-century revival of the French Renaissance style.
>
> If both pages are viewed simultaneously in multiple browser windows, it
> should not possible for the two fonts to get mixed up -- each page should
> receive its own font --,
Yes, definitely.
> let alone for either to prevent me from using my locally
> installed font called Garamond (which happens to be a 20th
> century revival, but most probably one that is different from
> the second page''s font).
There are a number of related scenarios here, and I''m not clear
exactly to which you are referring.

There is no way that a Garamond font provided by one web page
will affect which Garamond font you get on another web page or in
another app.

Normally your Garamond font will only be used for web content if
requested by the author, or if you have configured Garamond as a
default generic font and author-specified fonts are either
non-existing, disabled, or lacking support the character.  So there
is not really a use of the local Garamond to "prevent" here.

On Wed, 17 Dec 2008 00:08:56 -0500, Behdad Esfahbod wrote:
> Karl Tomlinson wrote:
>> I''m looking for comments on how fontconfig users would prefer
font
>> families defined in web documents through @font-face (or defined
>> by other means in other documents) to be presented in FcPattern *p
>> for FcConfigSubstitute(FcConfig*, p, FcMatchPattern).
>> 
>> The question basically comes down to whether
>> author-defined/provided fonts should be treated in the same way as
>> locally installed fonts, or whether they should be considered
>> special.
>
> My take on it is that they should be considered special.  And
that''s the API
> I''m going to implement in Pango.  Actually the proposed Pango API
is very
> close to CSS.  Or I think it is.  Anyway, the bug to track it is here:
>
>   http://bugzilla.gnome.org/show_bug.cgi?id=347237
Thanks, Behdad.

The fontconfig API that was useful for Mozilla was FcFontSetSort so
that both system and document-specific FcFontSets could be
included in the same sort without polluting all font sorts with
the fonts from one document.

I don''t really have any good suggestions on how to do something
similar with a PangoFontMap, unfortunately.  It may be nice to
have a single FontMap throughout the app so that caches are shared
- a separate FontMap for each document may get heavy.

Perhaps an API to "Add font file and/or cairo font face to
catalog" could be used if there was a flag to say whether or not
it should be used for fallback.  In our case we want the font
to only be used if the font was explicitly requested in the description.
> You do a FcConfigSubstitute(,,FcMatchFont) in the resulting fonts too,
right?
Yes, though there were speed and memory gains from only doing this
for fonts that actually get used to render glyphs.
>> Part of me would like to think that document fonts are just like
>> any other fonts but I think there are differences.
>> 
>> * The user has control over locally installed fonts, and can set
>>   up fontconfig rules appropriately.  Document fonts change as
>>   often as users view different documents and do not get parsed by
>>   acceptfont/rejectfont rules.
>> 
>> * Document fonts may not be what they seem.
>> 
>>   * The font-family descriptor may have little to do with the font
>>     itself.
>> 
>>   * The document font may be a trimmed down version of the font
>>     with only enough glyphs to display a heading, for example.
>
> Exactly.  A good way to think about them is like fonts embedded in a PDF,
or a
> SVG.  Do you expect your PDF/SVG.  That makes things much easier.
CSS may be a little different here (but I don''t know PDF/SVG in
detail).

I imagine a PDF would specify one font for each character (or run
of characters).

CSS OTOH specifies a number of fonts for a block of characters.
The intention is that the fonts that are actually used depend on
which fonts are actually available and support the characters.
>> My current thinking is that the FC_FAMILY value in the font
>> pattern representing fonts from document @font-face rules would
>> have a namespace. e.g.
>> 
>>   family: "@font-face:BPG Ucnobi U"
>
> Exactly my thinking in the context of how this will be used with Pango.
>
>
>> and, when the CSS style for an HTML element requests
>> "font-family:BPG Ucnobi U" (and an @font-face rule exists),
the
>> FC_FAMILY property of the pattern passed to
>> FcConfigSubstitute(,,FcMatchPattern) would include
>> "@font-face:BPG Ucnobi U".
>
> You sure can do that.  But I''m not sure how useful that would be.
We kind of need to do this because of the way CSS''s font-family
can include a list of both system and document font families (in
any order).  If we are going to pass the system font families
through FcConfigSubstitute, then the document font families need
to passed also.

A fontconfig API that might provide more flexibility here would be
something like FcValueGetBinding or perhaps FcPatternGetBinding.
At present the only way to properly interpret the families from
FcConfigSubstitute is to use FcFontSetMatch or FcFontSetSort.
Though, that would only make the sorting more flexible - I think
all families still need to be passed to one FcConfigSubstitute.

karlt+freedesktop at karlt.net wrote:
> On Wed, 17 Dec 2008 00:08:56 -0500, Behdad Esfahbod wrote:
> 
>> Karl Tomlinson wrote:
>>> I''m looking for comments on how fontconfig users would
prefer font
>>> families defined in web documents through @font-face (or defined
>>> by other means in other documents) to be presented in FcPattern *p
>>> for FcConfigSubstitute(FcConfig*, p, FcMatchPattern).
>>>
>>> The question basically comes down to whether
>>> author-defined/provided fonts should be treated in the same way as
>>> locally installed fonts, or whether they should be considered
>>> special.
>> My take on it is that they should be considered special.  And
that''s the API
>> I''m going to implement in Pango.  Actually the proposed Pango
API is very
>> close to CSS.  Or I think it is.  Anyway, the bug to track it is here:
>>
>>   http://bugzilla.gnome.org/show_bug.cgi?id=347237
> 
> Thanks, Behdad.
> 
> The fontconfig API that was useful for Mozilla was FcFontSetSort so
> that both system and document-specific FcFontSets could be
> included in the same sort without polluting all font sorts with
> the fonts from one document.
I see.
> I don''t really have any good suggestions on how to do something
> similar with a PangoFontMap, unfortunately.  It may be nice to
> have a single FontMap throughout the app so that caches are shared
> - a separate FontMap for each document may get heavy.
Yeah.  The way I thought this may be used is that the application assigns a
unique family name to each document font.  And application fonts are not used
in fallback (because they are not added to fontconfig catalog at all).
> Perhaps an API to "Add font file and/or cairo font face to
> catalog" could be used if there was a flag to say whether or not
> it should be used for fallback.  In our case we want the font
> to only be used if the font was explicitly requested in the description.
> 
>> You do a FcConfigSubstitute(,,FcMatchFont) in the resulting fonts too,
right?
> 
> Yes, though there were speed and memory gains from only doing this
> for fonts that actually get used to render glyphs.
> 
>>> Part of me would like to think that document fonts are just like
>>> any other fonts but I think there are differences.
>>>
>>> * The user has control over locally installed fonts, and can set
>>>   up fontconfig rules appropriately.  Document fonts change as
>>>   often as users view different documents and do not get parsed by
>>>   acceptfont/rejectfont rules.
>>>
>>> * Document fonts may not be what they seem.
>>>
>>>   * The font-family descriptor may have little to do with the font
>>>     itself.
>>>
>>>   * The document font may be a trimmed down version of the font
>>>     with only enough glyphs to display a heading, for example.
>> Exactly.  A good way to think about them is like fonts embedded in a
PDF, or a
>> SVG.  Do you expect your PDF/SVG.  That makes things much easier.
> 
> CSS may be a little different here (but I don''t know PDF/SVG in
> detail).
> 
> I imagine a PDF would specify one font for each character (or run
> of characters).
> 
> CSS OTOH specifies a number of fonts for a block of characters.
> The intention is that the fonts that are actually used depend on
> which fonts are actually available and support the characters.
Yeah, the fallback makes it quite complicated.  I''m not sure how useful
fallback to @font-face fonts are, but as you say, it''s CSS spec and you
have
to implement it :).
>>> My current thinking is that the FC_FAMILY value in the font
>>> pattern representing fonts from document @font-face rules would
>>> have a namespace. e.g.
>>>
>>>   family: "@font-face:BPG Ucnobi U"
>> Exactly my thinking in the context of how this will be used with Pango.
>>
>>
>>> and, when the CSS style for an HTML element requests
>>> "font-family:BPG Ucnobi U" (and an @font-face rule
exists), the
>>> FC_FAMILY property of the pattern passed to
>>> FcConfigSubstitute(,,FcMatchPattern) would include
>>> "@font-face:BPG Ucnobi U".
>> You sure can do that.  But I''m not sure how useful that would
be.
> 
> We kind of need to do this because of the way CSS''s font-family
> can include a list of both system and document font families (in
> any order).  If we are going to pass the system font families
> through FcConfigSubstitute, then the document font families need
> to passed also.
> 
> A fontconfig API that might provide more flexibility here would be
> something like FcValueGetBinding or perhaps FcPatternGetBinding.
> At present the only way to properly interpret the families from
> FcConfigSubstitute is to use FcFontSetMatch or FcFontSetSort.
> Though, that would only make the sorting more flexible - I think
> all families still need to be passed to one FcConfigSubstitute.
I''m not sure GetBinding would be that useful.  The configuration can do
binding=same or binding=strong easily.

Cheers,
behdad