ogg dev - Jul 2009 - Fixing ogg vorbis corruption caused by bad metadata

ogg.k.ogg.k at googlemail.com wrote, on 7/14/2009 7:16
AM:
>> easy to replace. The second packet is the metadata, which we can lose.
>> It's just the third packet that needs to be reconstructed. After
that,
>> you could start at any packet division in the rest of the file and it
>> would play fine? So this generic restore tool that I'm positing
would
>> just need to try every known codebook set until a valid file was
>> produced? How hard would that be?
> You'd need to renumber and regenerate CRCs. rogg might do all that
> (a set of tools somewhere on svn.xiph.org).
>> What I'm puzzled by, based on the information above, is why my
attempted
>> fixes haven't worked: pick a valid ogg vorbis file that I believe
was
>> ripped with the same tool around the same time, swap in the first three
>> packets from that file, modify the serial number to match the new file.
>> Thus far this technique hasn't actually fixed any files.
> CRCs ? If they're wrong (and they'd become wrong if you fiddle with
the
> serial number without regenerating them), libogg will ignore the page.
Thank you! I'd never heard of rogg. That definitely gets me one step 
closer to a solution. Taking a file where I had substituted the header 
from a good ogg into the "bad" ogg, and then using rogg to fix the
crc,
I now have a playable file. What's odd is it starts out, for about a 
second, as audible and normal, and then the audio is scrambled -- 
perhaps a bitrate issue?

Here's a sample of the result of this process:

http://adam.rosi-kessel.org/bugs/liboggz/484/fixed_with_rogg.ogg

So I think this might just need a little more tweaking to be turned into 
an automated fix script. Any ideas based on that file exactly what is 
wrong and how to correct it?

Adam

Adam Rosi-Kessel wrote, on 7/14/2009 7:55 AM:
> ogg.k.ogg.k at googlemail.com wrote, on 7/14/2009 7:16 AM:
>>> easy to replace. The second packet is the metadata, which we can
lose.
>>> It's just the third packet that needs to be reconstructed.
After that,
>>> you could start at any packet division in the rest of the file and
it
>>> would play fine? So this generic restore tool that I'm positing
would
>>> just need to try every known codebook set until a valid file was
>>> produced? How hard would that be?
>> You'd need to renumber and regenerate CRCs. rogg might do all that
>> (a set of tools somewhere on svn.xiph.org).
>>> What I'm puzzled by, based on the information above, is why my
attempted
>>> fixes haven't worked: pick a valid ogg vorbis file that I
believe was
>>> ripped with the same tool around the same time, swap in the first
three
>>> packets from that file, modify the serial number to match the new
file.
>>> Thus far this technique hasn't actually fixed any files.
>> CRCs ? If they're wrong (and they'd become wrong if you fiddle
with the
>> serial number without regenerating them), libogg will ignore the page.
>
> Thank you! I'd never heard of rogg. That definitely gets me one step
> closer to a solution. Taking a file where I had substituted the header
> from a good ogg into the "bad" ogg, and then using rogg to fix
the crc,
> I now have a playable file. What's odd is it starts out, for about a
> second, as audible and normal, and then the audio is scrambled --
> perhaps a bitrate issue?
OK, now I've actually successfully repaired a file with this method. 
Rather than use rogg_crc to fix the CRC after the fact, I'm using 
rogg_serial to replace the serial number in the "good" ogg before 
swapping the first three packets into the "bad" ogg. The result
appears
to be an ogg that plays perfectly.

This fix should be easy to script -- then I'll just need to use some 
other means to retag the files since they will have lost their metadata. 
I suppose if I wanted to be clever, I could extract from the binary all 
the key tags which are still fine (album, artist, genre, trackno, etc) 
and the retag the file with a standard tool after the first three 
packets are replaced.

The only issue I'm noticing is ogginfo reports:

Warning: sequence number gap in stream 1. Got page 14 when expecting 
page 2. Indicates missing data.
Warning: discontinuity in stream (1)

This doesn't seem to interfere with proper playing, but does anyone have 
a suggestion for a simple fix here? I suppose I just need to renumber 
the pages?

Adam

On Tue, Jul 14, 2009 at 9:48 AM, Adam Rosi-Kessel<adam at rosi-kessel.org>
wrote:
> The only issue I'm noticing is ogginfo reports:
>
> Warning: sequence number gap in stream 1. Got page 14 when expecting
> page 2. Indicates missing data.
> Warning: discontinuity in stream (1)
I'd guess this is flagging the data that was overwritten by the bad
tagging code. Some audio is missing, but 12 pages isn't enough to be
noticeable in most files.
> This doesn't seem to interfere with proper playing, but does anyone
have
> a suggestion for a simple fix here? I suppose I just need to renumber
> the pages?
Well, it's a warning for a reason; the file is still valid and will
play, it's just telling you some data is missing compared to the
original encode. Renumbering the pages (and fixing the crc's after the
change) would remove the warning, if that's what you want.

Cool that you've gotten them fixed!

 -r