theora dev - Jul 2006 - [theora in hardware] Contents of YUV_BUFFER_ENTRY

Hi everyone, it's my first e-mail to this list (my classmates from college
have sent a few though), so i'ld like first to greet everyone and
congratulate all the people responsible for all the work with this theora
project.

So, to my question:

As you probably know, we are working on an implementation of the theora
codec on hardware. I'm currently responsible for getting the function
ReconRefFrame in dct_decode.c working, and after i understood the code and
finished it in VHDL i found out (after 3 painfull debugging days) that i
don't understand at all what is in LastFrameRecon, ThisFrameRecon and
GoldenFrame (which types are YUV_BUFFER_ENTRY). Generally what i want to
know is: What is inside YUV_BUFFER_ENTRY? More specifically, keep reading...

I know it is a buffer for the pixels, but i just recently found out that
there are more things then just the pixels i need.  This is a complicated
issue, so i'll explain step by step what i understand (which i might be
wrong) and finally to what i dont understand.

1) What i understand: recon_pixel_index_table(i) returns the absolute
location of a relative location. Example: if i do
recon_pixel_Index_table(0), i would get the first pixel of the frame
(although i dont quite get the visual idea of what is the first pixel. Is it
in that black area around the frame? is it the first real pixel? If i do
recon_pixel_index_table(YPlaneFragments) do i get the first pixel in U
Plane?)

2) What i dont understand: i expected to YUV_BUFFER_ENTRY to contain only
the pixels I'ld be using. So in my mind, if i did LastFrameRecon(0) to
LastFrameRecon(YPlaneFragments), for example, i'ld get all the Y pixels. I
realized after careful study of the recon_pixel_index_table that if i wanted
the first pixel of the Y Plane it would not be 0, but in fact,
ReconYDataOffset.
Now, why is there such an offset? What is in between 0 and ReconYDataOffset?
Is there any more "unexpected" data mixed in the pixels im using?
Where am i
going to be needing these pixels? It would help a bunch if i could eliminate
them in hardware, since we're short on memory

Sorry for the terribly long e-mail, but my last attempt at explaining all
this in a hurry resulted in 2 very confused people with communication
problems. So i put it as clear as i could
Thanks a lot for the help

Arthur Valadares
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://lists.xiph.org/pipermail/theora-dev/attachments/20060724/9a97f212/attachment.html

Arthur Valadares wrote:
> 1) What i understand: recon_pixel_index_table(i) returns the absolute 
> location of a relative location. Example: if i do 
> recon_pixel_Index_table(0), i would get the first pixel of the frame 
> (although i dont quite get the visual idea of what is the first pixel. 
> Is it in that black area around the frame? is it the first real pixel? 
> If i do recon_pixel_index_table(YPlaneFragments) do i get the first 
> pixel in U Plane?)
recon_pixel_index_table gives the offset for the pixel in the 
lower-right of the given fragment (remember Theora stores the image from 
bottom to top internally). Be careful, as the fragments are not arranged 
in row-major order as one might expect, but "Hilbert-curve order".
This
is explained in the spec in section 2.3. If what I just said doesn't 
make any sense, I recommend reading all of Chapter 2. frinit.c is also a 
good section of code to read, as this is what sets up all of these offsets.

The "black area" around the frame is added solely for video whose 
dimensions are not multiples of 16. Internally Theora only deals with 
images that are a multiple of 16, so the "first pixel" will indeed be 
inside this area, and not the first "real" pixel. However, the decoder
shouldn't care whether a pixel is "real" or not... it must operate
on
the full, padded frame until it is time for display, when it is supposed 
to crop it for display (the cropped data can be used to predict future 
frames, though, and so _cannot_ be ignored to save a few bytes).
> 2) What i dont understand: i expected to YUV_BUFFER_ENTRY to contain 
> only the pixels I'ld be using. So in my mind, if i did
LastFrameRecon(0)
> to LastFrameRecon(YPlaneFragments), for example, i'ld get all the Y 
> pixels. I realized after careful study of the recon_pixel_index_table 
> that if i wanted the first pixel of the Y Plane it would not be 0, but 
> in fact, ReconYDataOffset.
> Now, why is there such an offset? What is in between 0 and 
> ReconYDataOffset? Is there any more "unexpected" data mixed in
the
> pixels im using? Where am i going to be needing these pixels? It would 
> help a bunch if i could eliminate them in hardware, since we're short
on
> memory
This makes more sense if you look up at ExpandBlock(), in particular the 
part where it computes MVOffset. When motion compensation is used, a 
fragment is reconstructed from an _offset_ version of pixels in one of 
the reconstruction buffers. This offset can quite legally point outside 
of the image proper, so an extra "border" is added all around the
image
to give it some valid data to point to. This is the "unexpected" data 
you describe.

If you don't want to add this border, you will be required to clamp the 
indices of the pixels you're reading to lie within the image, for 
_every_ pixel you read (see, e.g., Section 7.9.1 of the spec). In C, 
it's obviously much faster to copy the edge pixels into this border and 
avoid the clamping.

Thanks for the help, it took me a while to understand everything you said,
but after a little reading (especially the 7.9.1 you recommended) i think i
got it. I still have a question about the definition you sent, it doesn`t
fit with what i`m reading on the code:


recon_pixel_index_table gives the offset for the pixel in
the
> lower-right of the given fragment (remember Theora stores the image from
> bottom to top internally). Be careful, as the fragments are not arranged


 Please look at this part of the code: (imagine PlaneFragOffset = 0 for
example)
> PixelIndex = pbi->recon_pixel_index_table[PlaneFragOffset];
>
> SrcPtr1 = &DestReconPtr[ PixelIndex ];
>
> DestPtr1 = &DestReconPtr[ PixelIndex - PlaneBorderWidth ];
>
> PixelIndex = pbi->recon_pixel_index_table[PlaneFragOffset +
>
> LineFragments - 1] +
>
> (HFRAGPIXELS - 1);
>
> SrcPtr2 = &DestReconPtr[ PixelIndex ];
>
> DestPtr2 = &DestReconPtr[ PixelIndex + 1 ];
>
 it seems that recon_pixel_index returns the lower LEFT pixel. I noticed how
he sets the first pixelindex as
pbi->recon_pixel_index_table[PlaneFragOffset].
If that is the lower left pixel, it makes sense, since this code is suposed
to copy the first pixel on the left over to the MV edges. It then gets the
destination pointer as the lower left - 16 (left edge border)

On the second pixelindex, it does PixelIndex
pbi->recon_pixel_index_table[PlaneFragOffset +LineFragments - 1] +
(HFRAGPIXELS
- 1) which makes sense again if it returns the lower left pixel, so this
could be interpreted as 1) get the lower left pixel on the fragment that
represents the last fragment on the line and 2) Add the number of pixels on
a fragment to make it go from the lower left to the lower right, and then
set the destination as +1, which would be just by it`s side to fill the
right edge border

Correct me if im wrong on my interpretation, but it seems to me that it fits
perfectly

Arthur Valadares
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://lists.xiph.org/pipermail/theora-dev/attachments/20060725/3a37f539/attachment.html