ogg dev - Nov 2018 - How to concatenate Ogg in the browser JS?

Please see the documentation: https://xiph.org/ogg/doc/framing.html

I would encourage you to use random serial numbers, as intended, also, 
as any downstream consumers of your files will face limitations similar 
to the ones you are facing if they want to do anything more with them.

But before you go too far down the route of changing the serial numbers, 
can you tell us what software is producing Oggs with a hard-coded serial 
number to begin with?

Vitaly Zdanevich wrote:
> Thank you again for your help, now I am thinking about much simpler
solution - because all my Oggs comes from the the one source looks like I can
alter serial and crc32 at their hardcoded positions. Please see this screenshot
https://giphy.com/gifs/9AIe7ksYBwiYQoLVv9/fullscreen with serials 11111111 and
22222222- it will be correct approach? And can you please say me from what
region I need to calculate crc32? And also as we can see on the screenshot above
- changed two groups of numbers beside the serials - what is that? Thank you
very much for help.
> 
> 16.11.2018, 04:01, "Ralph Giles" <giles at thaumas.net>:
>> On 2018-11-15 11:01 a.m., Vitaly Zdanevich wrote:
>>>   Maybe you know some existing code snippet or library for
Javascript for altering serial and crc?
>>
>> I don't, sorry.
>>
>>>   Just now I compiled your quick script to wasm but the size is
28K, > I think about simple correct solution for that case.
>>
>> If you strip out the fprintf()s for progress and error reporting, it
>> might get smaller. More so if you replace the open/memmap step with
>> direct access to the blob as an array. But it's not very much code
to
>> just port the functions to pure js.
>>
>>    -r
> _______________________________________________
> ogg-dev mailing list
> ogg-dev at xiph.org
> http://lists.xiph.org/mailman/listinfo/ogg-dev
>

AWS Polly produces Oggs with hardcoded serial numbers as zeros.

16.11.2018, 17:57, "Timothy B. Terriberry" <tterribe at
xiph.org>:
> Please see the documentation: https://xiph.org/ogg/doc/framing.html
>
> I would encourage you to use random serial numbers, as intended, also,
> as any downstream consumers of your files will face limitations similar
> to the ones you are facing if they want to do anything more with them.
>
> But before you go too far down the route of changing the serial numbers,
> can you tell us what software is producing Oggs with a hard-coded serial
> number to begin with?
>
> Vitaly Zdanevich wrote:
>>  Thank you again for your help, now I am thinking about much simpler
solution - because all my Oggs comes from the the one source looks like I can
alter serial and crc32 at their hardcoded positions. Please see this screenshot
https://giphy.com/gifs/9AIe7ksYBwiYQoLVv9/fullscreen with serials 11111111 and
22222222- it will be correct approach? And can you please say me from what
region I need to calculate crc32? And also as we can see on the screenshot above
- changed two groups of numbers beside the serials - what is that? Thank you
very much for help.
>>
>>  16.11.2018, 04:01, "Ralph Giles" <giles at
thaumas.net>:
>>>  On 2018-11-15 11:01 a.m., Vitaly Zdanevich wrote:
>>>>    Maybe you know some existing code snippet or library for
Javascript for altering serial and crc?
>>>
>>>  I don't, sorry.
>>>
>>>>    Just now I compiled your quick script to wasm but the size
is 28K, > I think about simple correct solution for that case.
>>>
>>>  If you strip out the fprintf()s for progress and error reporting,
it
>>>  might get smaller. More so if you replace the open/memmap step
with
>>>  direct access to the blob as an array. But it's not very much
code to
>>>  just port the functions to pure js.
>>>
>>>     -r
>>  _______________________________________________
>>  ogg-dev mailing list
>>  ogg-dev at xiph.org
>>  http://lists.xiph.org/mailman/listinfo/ogg-dev

On 2018-11-16 7:02 a.m., Vitaly Zdanevich wrote:
> AWS Polly produces Oggs with hardcoded serial numbers as zeros.
Your earlier ogginfo post said the file was created by ffmpeg. There was 
a bug like this in ffmpeg, fixed in 2010. I hope they're not using an 
ffmpeg build that old! Or the low-quality 'vorbis' encoder included with
the ffmpeg source instead of 'libvorbis'.

It's also possible something about their runtime environment is 
providing bad random number seeds. Either way, it's an issue which 
should be reported.

Pity they don't support Opus. It's quite a bit more efficient with
speech.

  -r

I faced with problem about CRC32 calculation, I see that lookup table is
different here https://github.com/rillian/rogg/blob/master/rogg.c#L196
end for example here
https://stackoverflow.com/a/18639975/1879101 - JS code on this page for building
the table produces the same result. I think that generating the table is better
than having hardcoded one - for security against the case when somebody
accidentally change one character and this table will continue to work in 99% of
cases. My current JS code for building table:

function _makeCRCTable() {
    const CRCTable = new Uint32Array(256);
    for (let i = 256; i--;) {
        let char = i;
        for (let j = 8; j--;) {
            char = char & 1 ? 3988292384 ^ char >>> 1 : char
>>> 1;
        }
        CRCTable[i] = char;
    }
    return CRCTable;
}

I need to fix something here?

16.11.2018, 17:57, "Timothy B. Terriberry" <tterribe at
xiph.org>:
> Please see the documentation: https://xiph.org/ogg/doc/framing.html
>
> I would encourage you to use random serial numbers, as intended, also,
> as any downstream consumers of your files will face limitations similar
> to the ones you are facing if they want to do anything more with them.
>
> But before you go too far down the route of changing the serial numbers,
> can you tell us what software is producing Oggs with a hard-coded serial
> number to begin with?
>
> Vitaly Zdanevich wrote:
>>  Thank you again for your help, now I am thinking about much simpler
solution - because all my Oggs comes from the the one source looks like I can
alter serial and crc32 at their hardcoded positions. Please see this screenshot
https://giphy.com/gifs/9AIe7ksYBwiYQoLVv9/fullscreen with serials 11111111 and
22222222- it will be correct approach? And can you please say me from what
region I need to calculate crc32? And also as we can see on the screenshot above
- changed two groups of numbers beside the serials - what is that? Thank you
very much for help.
>>
>>  16.11.2018, 04:01, "Ralph Giles" <giles at
thaumas.net>:
>>>  On 2018-11-15 11:01 a.m., Vitaly Zdanevich wrote:
>>>>    Maybe you know some existing code snippet or library for
Javascript for altering serial and crc?
>>>
>>>  I don't, sorry.
>>>
>>>>    Just now I compiled your quick script to wasm but the size
is 28K, > I think about simple correct solution for that case.
>>>
>>>  If you strip out the fprintf()s for progress and error reporting,
it
>>>  might get smaller. More so if you replace the open/memmap step
with
>>>  direct access to the blob as an array. But it's not very much
code to
>>>  just port the functions to pure js.
>>>
>>>     -r
>>  _______________________________________________
>>  ogg-dev mailing list
>>  ogg-dev at xiph.org
>>  http://lists.xiph.org/mailman/listinfo/ogg-dev

I found how to build CRC32 table for Ogg in JS, if anyone interested:

function _makeCRC32Table() {
    // From
https://stackoverflow.com/questions/53438815/hot-to-build-crc32-table-for-ogg
    const polynomial = 79764919;
    const mask = 2147483648;
    const CRCTable = new Uint32Array(256);
    for (let i = 256; i--;) {
        let char = i << 24;
        for (let j = 8; j--;) {
            char = char & mask ? polynomial ^ char << 1 : char
<< 1;
        }
        CRCTable[i] = char;
    }
    return CRCTable;
}


22.11.2018, 17:43, "Vitaly Zdanevich" <zdanevich.vitaly at
ya.ru>:
> I faced with problem about CRC32 calculation, I see that lookup table is
different here https://github.com/rillian/rogg/blob/master/rogg.c#L196
> end for example here
> https://stackoverflow.com/a/18639975/1879101 - JS code on this page for
building the table produces the same result. I think that generating the table
is better than having hardcoded one - for security against the case when
somebody accidentally change one character and this table will continue to work
in 99% of cases. My current JS code for building table:
>
> function _makeCRCTable() {
>     const CRCTable = new Uint32Array(256);
>     for (let i = 256; i--;) {
>         let char = i;
>         for (let j = 8; j--;) {
>             char = char & 1 ? 3988292384 ^ char >>> 1 : char
>>> 1;
>         }
>         CRCTable[i] = char;
>     }
>     return CRCTable;
> }
>
> I need to fix something here?
>
> 16.11.2018, 17:57, "Timothy B. Terriberry" <tterribe at
xiph.org>:
>>  Please see the documentation: https://xiph.org/ogg/doc/framing.html
>>
>>  I would encourage you to use random serial numbers, as intended, also,
>>  as any downstream consumers of your files will face limitations
similar
>>  to the ones you are facing if they want to do anything more with them.
>>
>>  But before you go too far down the route of changing the serial
numbers,
>>  can you tell us what software is producing Oggs with a hard-coded
serial
>>  number to begin with?
>>
>>  Vitaly Zdanevich wrote:
>>>   Thank you again for your help, now I am thinking about much
simpler solution - because all my Oggs comes from the the one source looks like
I can alter serial and crc32 at their hardcoded positions. Please see this
screenshot https://giphy.com/gifs/9AIe7ksYBwiYQoLVv9/fullscreen with serials
11111111 and 22222222- it will be correct approach? And can you please say me
from what region I need to calculate crc32? And also as we can see on the
screenshot above - changed two groups of numbers beside the serials - what is
that? Thank you very much for help.
>>>
>>>   16.11.2018, 04:01, "Ralph Giles" <giles at
thaumas.net>:
>>>>   On 2018-11-15 11:01 a.m., Vitaly Zdanevich wrote:
>>>>>     Maybe you know some existing code snippet or library
for Javascript for altering serial and crc?
>>>>
>>>>   I don't, sorry.
>>>>
>>>>>     Just now I compiled your quick script to wasm but the
size is 28K, > I think about simple correct solution for that case.
>>>>
>>>>   If you strip out the fprintf()s for progress and error
reporting, it
>>>>   might get smaller. More so if you replace the open/memmap
step with
>>>>   direct access to the blob as an array. But it's not very
much code to
>>>>   just port the functions to pure js.
>>>>
>>>>      -r
>>>   _______________________________________________
>>>   ogg-dev mailing list
>>>   ogg-dev at xiph.org
>>>   http://lists.xiph.org/mailman/listinfo/ogg-dev