Fontconfig - Nov 2005 - Xft - (incorrect ?) subpixel filtering.

Around 22 o''clock on May 19, Yaroslav Rastrigin wrote:
> Limitations and drawbacks of this approach are obvious. First and foremost
> is unacessibility of pixel values outside of glyph''s bitmap. 
I''ve thought about using a wider filter; I''m glad you had a
chance to
experiment with this.  There''s no reason you couldn''t
''widen'' glyphs by a
pixel to make the filtering work better; simply change the size of the 
output RGBA image and adjust all of the metrics.  That should let you
produce full RGBA pixels just beyond the current glyph bounds.

I do see an error in your code which may improve the output a bit:

                    hi1[0] = in[x ? x - 1 : x];

I think this should be:

		    hi1[0] = x ? in[x-1] : 0;

(similarly on the other end of the range).  The input is transparent 
beyond the bounds of the glyph.

Because all of the glyphs are precomposed before being applied to the 
screen, this should do the right thing when displaying multiple glyphs.

-keith

Several moments:
1. Filtering algorithm in xftglyphs.c lines 508..520 (approx.) don''t
produces
correct results.
(Definition of "correct" in this case is very subjective, though :-),
but
overall readability improvement could serve as a fair measurement.). Best 
seen on thin (one pixel wide)glyphs with vertical elements. F.e.
''i'' or ''l''.
While nonvertical elements of these glyphs are subpixel AA''ed,
verticals do
not. This behavior produces easily noticeable ''artifacts'',
visually marked as
gray or slightly colored dots on or near purely black glyph.
I''ve implemented (very simple and inefficient, both in performance and
''color
fringe'' removal) 3-level filter, provided below. While incomplete and
very
limited in usage, it often gives "better" results (again,
questionable, but
at least it smoothes glyphs more evenly).
              	for (x = 0; x < width * hmul; x += hmul)
                {
#if 0
                    red = green = blue = 0;
                    o = 0;
                    for (s = 0; s < 3; s++)
                    {
                        red += filters[rf][s]*in[x+o];
                        green += filters[gf][s]*in[x+o];
                        blue += filters[bf][s]*in[x+o];
                        o += os;
                    }
                    red = red / 65536;
                    green = green / 65536;
                    blue = blue / 65536;
                    red += in[x+o];
                    green += in[x+o];
                    blue += in[x+o];
                    *out++ = (green << 24) | (red << 16) | (green
<< 8) |
blue;                                              
#endif
                    hi1[0] = in[x ? x - 1 : x];
                    hi1[1] = in[x];
                    hi1[2] = in[x+1];
                    hi1[3] = in[x+2];
                    hi1[4] = in[x+3 >= width*hmul ? x + 2 : x + 3];
                    red = ((hi1[0] * 65536)/5) + ((hi1[1] * 65536 * 3)/5) + 
((hi1[2] * 65536)/5);                            
                    green = ((hi1[1] * 65536)/5) + ((hi1[2] * 65536 * 3)/5) + 
((hi1[3] * 65536)/5);                          
                    blue = ((hi1[2] * 65536)/5) + ((hi1[3] * 65536 * 3)/5) + 
((hi1[4] * 65536)/5);                           
                    *out++ = ((red / 65536) << 16) | ((green / 65536)
<< 8) |
(blue/65536);                                  
                }
 Limitations and drawbacks of this approach are obvious. First and foremost is 
unacessibility of pixel values outside of glyph''s bitmap. Again, this
is
mostly visible on sans-serif fonts (serifs are ''widening'' the
bitmap, so
vertical stems could be properly filtered on both sides) and thin vertical 
glyphs (Tahoma 8-10 pt. ''i'',''l'' or
''H''). This is incurable with current
implementation, when we''re filtering each glyph separately. I think
this
needs to be done on the whole string. This will help also to ensure more 
smooth and visually pleasing transition from one glyph to another when 
they''re near (0 or one pixel between them).


-- 
With all the best, yarick at relex dot ru.