Vorbis - Oct 2007 - Vorbis granule position

I have a technical question about the vorbis granule position,
but I would like to put the question into context.

When a ogg vorbis stream is ripped using wget, fetch, or
streamripper under Linux or FreeBSD, the resulting file has
problems both with granule position and with a missing EOS. I
think I can figure out how to add an EOS, but is there a way
to determine the granule position in a stream if both the
previous and next granule positions are correct?

I am not the first person to notice this problem:

http://streamripper.sourceforge.net/phpBB2/viewtopic.php?t=1458

I understand that granule position of pages containing Vorbis
audio is in units of PCM audio samples, but I do not
understand if this quantity can only be calculated during
encoding or if it is possible to calculate it from the
captured file itself.
                                             
Finally, does there already exist a tool (sort of an 'fsck'
for ogg vorbis files) that will fix these files?

Regards,
 JM Rotenberry
 rotenber@io.com

On Tue, Oct 23, 2007 at 02:44:44PM -0500, James M. Rotenberry wrote:
> When a ogg vorbis stream is ripped using wget, fetch, or
> streamripper under Linux or FreeBSD, the resulting file has
> problems both with granule position and with a missing EOS.
A missing EOS isn't really a problem, it just indicates the stream is 
incomplete. There are a couple of scripts to set the EOS flag. Mine is 
in http://downloads.xiph.org/releases/rogg/rogg-0.1.tar.gz

Decreasing granulepos is a problem, and shouldn't happen. What stream 
are you seeing this from?

It is possible to retimestamp a stream, but I'm not aware of any tools 
which do so. Running it through gstreamer's muxer or liboggz after 
setting the packet timestamps to "-1" might help.

 -r

Ralph Giles

Thank you for your comments.
> Decreasing granulepos is a problem, and shouldn't happen. What stream
> are you seeing this from?
Based on my experience and the results of the tests I have appended
to this message, I believe that EVERY rip of streaming ogg vorbis
will have this problem.

The tests I ran used three different ogg vorbis streams, three
different ripping programs (wget, streamripper, and fetch), and
two different operating systems. In all three cases there is a
granulepos problem and, consequently, incorrect values for the
Playback length and the Average bitrate.

Regards,
 JM Rotenberry
 rotenber@io.com


---------------------------------------------------------------------
http://vorbis.nm.cbc.ca/cbcr1-toronto.ogg

Using wget on Linux:

ogginfo ./cbcr1-toronto.ogg
Processing file "./cbcr1-toronto.ogg"...

New logical stream (#1, serial: 0cc1016f): type vorbis
Vorbis headers parsed for stream 1, information follows...
Version: 0
Vendor: AO; aoTuV b4b [20051117] (based on Xiph.Org's libVorbis)
Channels: 1
Rate: 22050

Nominal bitrate: 24.000000 kb/s
Upper bitrate: 24.000000 kb/s
Lower bitrate: 24.000000 kb/s
User comments section follows...
        TITLE        ARTIST        ENCODEDBY=oddcastv3
Warning: sequence number gap in stream 1. Got page 3705920 when expecting page
2. Indicates missing data.
Warning: EOS not set on stream 1
Vorbis stream 1:
        Total data length: 118558 bytes
        Playback length: 31552m:45.766s
        Average bitrate: 0.000501 kb/s
--------------------------------------------------------------------------
http://audio-ogg.ibiblio.org:8000/wcpe.ogg

Using streamripper on FreeBSD:

ogginfo sr_program_2007_10_24_10_25_51.ogg
Processing file "sr_program_2007_10_24_10_25_51.ogg"...

New logical stream (#1, serial: 643c9869): type vorbis
s headers parsed for stream 1, information follows...
Version: 0
Vendor: Xiph.Org libVorbis I 20020717 (1.0)
Channels: 2
Rate: 44100

Nominal bitrate: 55.999000 kb/s
Upper bitrate not set
Lower bitrate not set
User comments section follows...
        TITLE=WCPE
Warning: sequence number gap in stream 1. Got page 37948 when expecting page 2.
Indicates missing data.
Warning: EOS not set on stream 1
Vorbis stream 1:
        Total data length: 200607 bytes
        Playback length: 326m:02.141s
        Average bitrate: 0.082039 kb/s
--------------------------------------------------------------------------
http://aud1.kpfa.org:8090/kpfa.ogg

Using fetch on FreeBSD:


gginfo ./kpfa.ogg
Processing file "./kpfa.ogg"...

New logical stream (#1, serial: 046cdbe2): type vorbis
Vorbis headers parsed for stream 1, information follows...
Version: 0
Vendor: Xiph.Org libVorbis I 20050304
Channels: 2
Rate: 18000

Nominal bitrate: 56.000000 kb/s
Upper bitrate not set
Lower bitrate not set
Warning: sequence number gap in stream 1. Got page 253700 when expecting page 2.
 Indicates missing data.
Warning: EOS not set on stream 1
Vorbis stream 1:
        Total data length: 141662 bytes
        Playback length: 2173m:44.056s
        Average bitrate: 0.008689 kb/s