Alexander Lubyagin
2011-Apr-05 11:25 UTC
[theora-dev] quantize after fdct, _dequant table, and idct
1) What are you doing "mathematically" in a procedure x86/x86enquant: oc_enc_quantize_sse2()? This - the assembler code, and I do not understand mathematically - that's going on there. --- A: 120 121 28 73 -20 -99 -98 -100 123 122 112 108 73 -32 -102 -98 123 123 117 121 100 74 -15 -97 124 125 125 124 123 116 74 -33 125 125 125 125 124 124 100 74 124 124 124 125 125 124 121 114 125 125 124 125 125 125 120 117 125 125 125 125 125 125 123 113 --- _dct: 2767 1165 -439 154 15 24 -87 -53 -1254 1080 -245 0 125 -15 -101 -67 -512 260 212 -207 200 -49 -80 -55 -111 -111 300 -166 125 -17 -70 -45 -54 -69 127 18 -31 60 -82 -30 -50 -30 40 83 -74 72 -68 -35 -31 -33 37 48 -40 40 -28 -44 -21 -13 29 12 1 -3 11 -44 --- _dequant: 24 32 32 36 32 32 40 36 36 36 40 40 40 44 48 56 48 48 48 48 48 56 56 56 52 56 60 56 60 60 60 56 60 56 60 64 64 64 60 60 64 60 56 60 64 64 64 64 64 64 64 64 64 60 64 64 64 64 64 64 64 64 64 64 --- _qdct: 115 36 -39 -14 34 -14 4 -7 7 -3 -1 -3 5 0 0 0 3 -4 6 -1 -1 -1 -1 2 -3 4 0 -2 -1 -2 -1 2 0 1 -1 0 0 1 1 -1 0 -1 -1 -1 -1 1 -1 1 0 0 -1 1 -1 -1 0 -1 1 0 0 0 -1 -1 0 -1 So, _qdct[i][j] = _dct[i][j] / _dequant[i][j] ? 2) Second question - you are using: 24 20 20 20 20 16 24 20 20 20 28 24 24 28 32 40 32 32 28 28 32 44 40 40 36 40 44 44 44 44 44 44 44 44 44 48 48 48 44 44 48 44 44 44 48 48 48 48 48 48 48 48 48 44 48 48 48 48 48 48 48 48 48 48 , or 24 32 32 36 32 32 40 36 36 36 40 40 40 44 48 56 48 48 48 48 48 56 56 56 52 56 60 56 60 60 60 56 60 56 60 64 64 64 60 60 64 60 56 60 64 64 64 64 64 64 64 64 64 60 64 64 64 64 64 64 64 64 64 64 for luma plane, and 28 28 28 36 32 36 52 40 40 52 60 56 52 56 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60 for chroma plane? 3) Third question. How do you restore the original matrix after fdct and quantize ... Used dequantize + idct?