freebsd stable - Dec 2010 - aesni(?) corrupts data on 8.2-BETA1

I just upgraded my main laptop from 8.1-RELEASE (GENERIC, amd64) to 
8.2-BETA1 and added aesni_load="YES" to my /boot/loader.conf.

(If my interpretation is correct:) With aesni loaded, I see many files 
corrupted on my geli encrypted volume. Without aesni loaded, they are ok.

I have got a journaling UFS2 on gjournal on geli on a FreeBSD partition 
on a MBR slice on a disk with ahci loaded.

Story: First I noticed some weirdness of Thunderbird not showing the 
"upgraded" message properly and reloading IMAP messages that have 
already been read, but did not think of anything. Only during my usual 
rsyncing of the encrypted volume, I saw that some files could not be 
read (invalid file descriptor?). I rebooted without aesni and got a 
different error message.

I created checksums of all files on that encrypted volume with and 
without aesni loaded (rebooting in between): 150 Differences (one files 
could not be read in both cases).

Just to make sure, I tried to rsync with "--checksum" and
"--dry-run" to
the other machine that is supposed to have the same files: With aesni, 
many files were scheduled to be synced and one could not be read, but 
without aesni, only that one file was scheduled to be synced -- it 
probably got corrupted for good with aesni loaded. It is especially 
weird that I did not attempt to write to the file that got corrupted on 
disk with aesni loaded.

Is there anything I am doing wrong or is it really aesni or the 
processor failing?

The processor is a Core i7-M620 (with AESNI of course).

Before I investigate any further, I have to make a real backup... 
rsyncing does not prevent silent corruption. I am lucky that it was not 
so silent after all.

Jan Henrik

Actually, I just noticed something like this as well with ssh via
cryptodev and rsync as well. It was erroring out. eg.


Dec 10 16:50:01 backup3 sshd[13120]: Corrupted MAC on input.
Dec 10 16:50:01 backup3 sshd[13120]: Finished discarding for 64.x.x.x

I had a few ssh sessions die as well.  It was working ok with a kernel /
world from last week.  I was going to try and see if I can narrow it
down, but it seemed to had been working fine with world from last week.
 Not sure if its the openssl update ?  But if you are seeing issues with
geli, then I doubt its openssl.

	---Mike

On 12/10/2010 7:49 PM, Jan Henrik Sylvester wrote:
> I just upgraded my main laptop from 8.1-RELEASE (GENERIC, amd64) to
> 8.2-BETA1 and added aesni_load="YES" to my /boot/loader.conf.
> 
> (If my interpretation is correct:) With aesni loaded, I see many files
> corrupted on my geli encrypted volume. Without aesni loaded, they are ok.
> 
> I have got a journaling UFS2 on gjournal on geli on a FreeBSD partition
> on a MBR slice on a disk with ahci loaded.
> 
> Story: First I noticed some weirdness of Thunderbird not showing the
> "upgraded" message properly and reloading IMAP messages that have
> already been read, but did not think of anything. Only during my usual
> rsyncing of the encrypted volume, I saw that some files could not be
> read (invalid file descriptor?). I rebooted without aesni and got a
> different error message.
> 
> I created checksums of all files on that encrypted volume with and
> without aesni loaded (rebooting in between): 150 Differences (one files
> could not be read in both cases).
> 
> Just to make sure, I tried to rsync with "--checksum" and
"--dry-run" to
> the other machine that is supposed to have the same files: With aesni,
> many files were scheduled to be synced and one could not be read, but
> without aesni, only that one file was scheduled to be synced -- it
> probably got corrupted for good with aesni loaded. It is especially
> weird that I did not attempt to write to the file that got corrupted on
> disk with aesni loaded.
> 
> Is there anything I am doing wrong or is it really aesni or the
> processor failing?
> 
> The processor is a Core i7-M620 (with AESNI of course).
> 
> Before I investigate any further, I have to make a real backup...
> rsyncing does not prevent silent corruption. I am lucky that it was not
> so silent after all.
> 
> Jan Henrik
> _______________________________________________
> freebsd-stable@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-stable
> To unsubscribe, send any mail to
"freebsd-stable-unsubscribe@freebsd.org"
> 
>

On Fri, Dec 10, 2010 at 08:43:18PM -0500, Mike Tancsa
wrote:
> Actually, I just noticed something like this as well with ssh via
> cryptodev and rsync as well. It was erroring out. eg.
> 
> 
> Dec 10 16:50:01 backup3 sshd[13120]: Corrupted MAC on input.
> Dec 10 16:50:01 backup3 sshd[13120]: Finished discarding for 64.x.x.x
> 
> I had a few ssh sessions die as well.  It was working ok with a kernel /
> world from last week.  I was going to try and see if I can narrow it
> down, but it seemed to had been working fine with world from last week.
>  Not sure if its the openssl update ?  But if you are seeing issues with
> geli, then I doubt its openssl.
> 
> 	---Mike
> 
> On 12/10/2010 7:49 PM, Jan Henrik Sylvester wrote:
> > I just upgraded my main laptop from 8.1-RELEASE (GENERIC, amd64) to
> > 8.2-BETA1 and added aesni_load="YES" to my
/boot/loader.conf.
> > 
> > (If my interpretation is correct:) With aesni loaded, I see many files
> > corrupted on my geli encrypted volume. Without aesni loaded, they are
ok.
> > 
> > I have got a journaling UFS2 on gjournal on geli on a FreeBSD
partition
> > on a MBR slice on a disk with ahci loaded.
> > 
> > Story: First I noticed some weirdness of Thunderbird not showing the
> > "upgraded" message properly and reloading IMAP messages that
have
> > already been read, but did not think of anything. Only during my usual
> > rsyncing of the encrypted volume, I saw that some files could not be
> > read (invalid file descriptor?). I rebooted without aesni and got a
> > different error message.
> > 
> > I created checksums of all files on that encrypted volume with and
> > without aesni loaded (rebooting in between): 150 Differences (one
files
> > could not be read in both cases).
> > 
> > Just to make sure, I tried to rsync with "--checksum" and
"--dry-run" to
> > the other machine that is supposed to have the same files: With aesni,
> > many files were scheduled to be synced and one could not be read, but
> > without aesni, only that one file was scheduled to be synced -- it
> > probably got corrupted for good with aesni loaded. It is especially
> > weird that I did not attempt to write to the file that got corrupted
on
> > disk with aesni loaded.
> > 
> > Is there anything I am doing wrong or is it really aesni or the
> > processor failing?
> > 
> > The processor is a Core i7-M620 (with AESNI of course).
> > 
> > Before I investigate any further, I have to make a real backup...
> > rsyncing does not prevent silent corruption. I am lucky that it was
not
> > so silent after all.
I have no access to AESNI hardware. For start, you may use
src/tools/tools/crypto/cryptotest
to somewhat verify the sanity of the driver.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 196 bytes
Desc: not available
Url :
http://lists.freebsd.org/pipermail/freebsd-stable/attachments/20101211/9a6c8d80/attachment.pgp

On 12/11/2010 11:01 AM, Kostik Belousov wrote:
> 
> I have no access to AESNI hardware. For start, you may use
> src/tools/tools/crypto/cryptotest
> to somewhat verify the sanity of the driver.
I doesnt happen every time, but one out of 5 or so 

 ./cryptotest -c -z -t 10
   0.000 sec,      20    aes crypts,      16 bytes,  4637681 byte/sec,    35.4
Mb/sec
   0.000 sec,      20    aes crypts,      32 bytes,  9014085 byte/sec,    68.8
Mb/sec
   0.000 sec,      20    aes crypts,      64 bytes, 17777778 byte/sec,   135.6
Mb/sec
   0.000 sec,      20    aes crypts,     128 bytes, 33684211 byte/sec,   257.0
Mb/sec
   0.000 sec,      20    aes crypts,     256 bytes, 67368421 byte/sec,   514.0
Mb/sec
   0.000 sec,      20    aes crypts,     512 bytes, 116363636 byte/sec,   887.8
Mb/sec
   0.000 sec,      20    aes crypts,    1024 bytes, 179649123 byte/sec,  1370.6
Mb/sec
   0.000 sec,      20    aes crypts,    2048 bytes, 201773399 byte/sec,  1539.4
Mb/sec
decrypt mismatch:
original:
0000: 35 38 65 74 61 69 75 62 6e 31 6e 6a 6a 33 37 38
0010: 6f 38 32 75 74 74 6a 34 62 62 62 21 69 74 6e 6f
0020: 61 38 32 39 6a 6f 6f 73 6e 31 65 74 73 62 6f 75
0030: 69 37 39 73 74 37 35 75 6f 73 6e 75 31 6f 68 6e
0040: 33 30 35 31 6f 30 68 61 31 33 35 35 6f 30 6a 65
0050: 34 38 6f 61 68 34 31 34 61 73 37 6e 62 35 65 6f
0060: 36 69 6e 74 73 37 65 61 74 73 75 36 73 61 73 61
0070: 62 33 75 74 37 73 38 32 34 61 61 69 74 75 6f 32
0080: 75 61 6f 30 65 61 65 62 35 61 31 36 6e 68 74 61
0090: 61 74 6f 6f 74 6e 74 65 31 65 35 34 61 69 75 68
00a0: 61 36 68 74 74 21 32 65 61 21 38 74 31 6a 32 36
00b0: 68 74 31 6f 75 68 30 73 69 65 6a 6a 32 31 6e 74
00c0: 74 34 30 30 73 62 32 35 6f 75 6f 21 31 61 65 32
00d0: 65 6f 37 61 31 34 62 31 6f 69 36 68 68 65 6f 61
00e0: 6e 35 61 75 34 69 65 39 21 34 62 74 36 38 39 37
00f0: 30 69 65 38 38 37 6f 74 38 31 6f 62 6f 69 30 35
0100: 6f 61 68 39 6f 21 62 61 73 34 36 6e 65 68 33 32
0110: 61 65 75 62 36 35 6e 33 6e 31 6f 35 34 6f 30 6f
0120: 30 6a 62 35 6a 38 21 6f 65 65 68 73 69 31 6f 61
0130: 6f 62 74 35 74 33 61 33 73 69 6a 6a 6f 61 37 35
0140: 21 6e 30 34 30 74 39 32 6f 74 6f 6f 65 35 69 31
0150: 74 73 6e 6a 74 31 31 37 31 6e 35 6e 37 62 68 74
0160: 35 6a 31 65 61 37 73 69 38 33 65 39 61 65 74 36
0170: 32 33 69 36 74 31 6f 68 74 34 34 31 69 36 38 35
0180: 30 36 74 35 6f 38 35 30 74 69 75 36 36 69 38 38
0190: 33 38 69 73 6f 6f 35 32 37 39 33 74 68 37 32 65
01a0: 6f 62 69 65 33 6f 62 73 65 35 6e 73 35 69 6f 6a
01b0: 31 6f 65 6e 31 73 6a 6a 21 31 35 68 6f 37 21 38
01c0: 33 21 21 6e 6a 6f 75 75 30 32 74 35 32 36 21 33
01d0: 73 65 39 61 32 73 21 65 35 61 21 30 62 39 38 6f
01e0: 62 73 31 69 32 38 30 62 75 37 73 62 6f 73 6f 75
01f0: 32 38 38 34 65 37 6e 69 62 6e 69 34 65 38 6f 38
0200: 65 69 38 74 38 34 21 34 68 30 21 74 74 6a 69 39
0210: 6f 6a 69 31 35 75 31 74 62 31 61 37 6f 73 68 74
0220: 34 39 34 65 69 73 6f 65 74 32 62 6a 37 6f 37 75
0230: 74 31 37 32 73 6a 33 6f 62 34 75 73 35 6a 31 39
0240: 37 35 38 37 36 74 62 6f 39 6a 74 33 37 6a 39 34
0250: 36 33 75 39 38 6e 6f 33 37 68 37 32 39 61 6a 33
0260: 75 36 37 6f 6f 6e 31 32 74 65 65 34 75 68 6f 65
0270: 31 65 38 62 38 6f 65 68 73 36 69 30 30 68 21 33
0280: 73 74 68 75 21 39 6f 74 38 37 68 73 6f 75 32 6e
0290: 6a 75 61 6f 36 74 73 6e 62 34 6e 39 32 6e 62 6e
02a0: 39 74 6f 62 21 38 6e 73 61 37 38 21 69 75 6e 6f
02b0: 6f 37 31 61 74 68 74 21 36 69 62 6a 6e 38 74 35
02c0: 61 21 34 31 38 30 61 62 73 62 36 73 35 6a 32 34
02d0: 35 21 62 6e 68 21 36 69 33 31 6a 62 35 62 74 30
02e0: 6f 61 38 6f 68 6f 6f 65 6a 73 6e 69 65 61 38 74
02f0: 68 69 39 39 21 61 6f 6f 36 31 6a 38 75 6f 38 68
0300: 30 65 34 68 31 6f 69 39 33 6e 6f 68 37 74 6a 6f
0310: 74 38 38 75 62 37 62 68 6a 69 62 34 32 33 36 32
0320: 61 75 30 6e 36 30 35 35 6f 34 75 30 38 61 35 61
0330: 62 21 31 38 74 74 36 68 30 34 32 62 37 34 38 32
0340: 38 6a 61 33 6a 21 6f 31 74 68 38 6a 6e 39 6f 68
0350: 62 73 6e 32 74 68 69 37 6e 32 75 21 74 73 65 34
0360: 65 75 73 74 6f 62 34 33 74 6f 68 33 38 74 68 74
0370: 34 37 36 68 35 6e 62 38 6a 65 38 68 36 39 32 34
0380: 6f 75 38 65 65 65 35 62 30 33 32 38 69 30 31 61
0390: 37 74 6f 62 6f 6a 74 37 21 34 62 32 30 31 75 39
03a0: 73 73 68 6f 32 33 68 62 6e 30 75 6e 30 34 37 73
03b0: 38 30 33 74 6a 21 73 61 73 39 74 74 37 33 30 31
03c0: 69 68 30 32 34 65 31 74 6f 37 39 62 6a 74 6f 36
03d0: 69 32 32 6a 65 33 6f 32 32 39 33 39 62 21 37 62
03e0: 31 37 31 65 36 65 35 34 6e 31 21 73 38 32 75 38
03f0: 68 35 6e 33 6a 6f 32 37 61 65 75 69 68 69 39 32
0400: 38 30 74 33 32 36 37 61 33 69 6f 31 6f 68 35 73
0410: 74 31 69 68 75 36 6f 31 37 65 31 6f 65 30 37 39
0420: 37 38 6f 6e 21 75 74 33 30 36 74 6f 37 32 33 74
0430: 69 75 65 75 32 37 37 6e 62 61 61 74 65 6f 74 62
0440: 33 39 68 39 32 65 62 62 37 6e 6a 34 6f 31 68 65
0450: 34 69 6a 75 37 65 62 6e 33 30 33 61 62 21 30 65
0460: 36 68 38 61 21 74 69 37 6f 6f 6a 31 6e 21 39 30
0470: 21 37 33 74 36 32 39 62 32 62 30 38 62 37 6e 31
0480: 35 73 36 35 74 6e 62 21 34 74 31 74 33 30 37 6f
0490: 34 74 6e 21 36 65 6f 6a 31 36 32 75 6f 6f 37 38
04a0: 32 33 6f 39 62 68 62 21 35 75 75 6a 74 30 38 38
04b0: 37 30 37 39 32 6f 37 21 6a 75 6e 37 61 21 61 69
04c0: 6f 34 39 6a 32 6a 38 73 68 31 62 61 31 75 32 61
04d0: 75 39 34 35 6f 6a 68 33 34 37 30 36 37 65 62 75
04e0: 6e 6a 61 61 33 62 36 31 21 34 36 38 74 38 32 69
04f0: 69 33 36 32 6a 74 35 32 31 36 61 61 73 30 31 75
0500: 35 32 31 6a 68 75 36 31 74 31 6f 38 6f 38 6a 6f
0510: 74 69 74 6f 30 6f 38 38 61 32 32 6f 62 69 61 73
0520: 32 32 6f 69 36 73 61 6e 61 21 74 21 37 6f 6f 61
0530: 65 36 21 6f 6a 73 75 39 6f 35 38 65 6f 62 36 38
0540: 68 34 31 74 6a 32 39 21 32 75 39 6e 61 34 37 33
0550: 30 37 65 61 31 61 34 73 21 6f 36 35 34 31 6f 36
0560: 65 69 62 30 6f 61 21 61 68 36 37 6e 74 61 39 74
0570: 65 38 6a 39 32 62 33 6f 34 35 69 6f 33 6e 74 61
0580: 75 39 61 62 73 31 30 35 69 37 68 30 6f 37 69 74
0590: 61 68 75 69 37 6e 65 68 6a 32 75 31 6a 74 36 31
05a0: 6e 37 75 75 61 33 6f 6a 37 75 6a 36 69 68 6f 61
05b0: 21 36 6a 69 68 75 68 39 6f 38 74 69 34 6f 31 74
05c0: 75 38 21 31 74 6a 39 38 34 73 30 34 36 6f 62 65
05d0: 6a 69 35 6e 30 75 32 6f 73 62 6f 61 6a 21 6a 34
05e0: 37 34 32 34 6f 68 62 69 35 62 6a 37 6a 74 62 6f
05f0: 37 73 6f 37 74 37 32 37 33 38 65 31 37 75 32 31
0600: 31 34 35 30 36 74 30 37 39 6a 68 73 65 30 75 62
0610: 31 65 6f 34 62 68 65 65 62 69 21 6e 74 65 69 34
0620: 6f 36 31 61 6e 31 65 32 39 33 6e 38 30 6f 33 38
0630: 34 36 65 21 69 69 65 6f 73 6a 35 37 75 62 68 62
0640: 34 32 69 74 21 65 62 36 35 35 33 65 38 6e 6e 65
0650: 6a 38 68 6f 6a 21 38 6f 34 6f 36 74 6a 74 39 62
0660: 36 6f 6e 65 61 61 38 75 21 74 61 73 69 74 32 6a
0670: 61 74 33 6f 30 74 31 68 30 33 65 6a 75 6a 73 65
0680: 37 21 73 6f 31 6a 61 38 35 6e 68 6f 33 73 69 6a
0690: 34 75 37 31 73 35 62 73 6a 31 6f 61 36 32 33 6f
06a0: 6f 69 68 21 68 6f 73 74 36 65 65 6f 36 65 75 75
06b0: 6f 73 34 32 36 21 6f 69 37 31 61 6f 69 6a 36 65
06c0: 6a 69 6a 21 6e 65 33 68 73 61 74 6e 74 33 39 65
06d0: 69 30 74 62 21 33 39 37 73 73 75 73 6f 65 6f 74
06e0: 65 74 33 61 39 6e 6f 21 37 39 32 74 6f 68 62 6f
06f0: 6a 39 6f 61 62 34 6f 21 61 61 69 69 69 6e 74 35
0700: 69 21 21 6f 32 6f 32 6e 65 34 74 31 6f 36 74 37
0710: 65 62 65 74 21 30 74 34 35 74 34 35 33 31 30 75
0720: 37 30 36 6e 39 38 32 31 62 62 62 6a 31 39 35 6f
0730: 61 73 30 34 74 74 6f 36 73 33 37 73 74 38 30 6a
0740: 34 6e 73 69 30 6f 31 36 68 21 69 39 36 36 65 30
0750: 39 6f 31 21 32 21 32 6f 39 39 75 75 69 74 31 38
0760: 39 65 31 39 34 32 65 6f 6f 35 65 34 37 30 34 33
0770: 62 36 33 38 65 65 30 38 38 69 73 68 74 65 62 6e
0780: 69 74 65 38 62 30 30 37 36 65 68 39 32 6f 6f 38
0790: 61 32 65 75 73 6f 74 6a 65 31 39 6f 6f 35 31 6e
07a0: 65 21 73 74 21 74 34 65 6f 6f 68 68 35 74 36 21
07b0: 6f 37 6f 75 36 21 34 34 74 74 39 69 36 37 39 37
07c0: 69 21 34 21 74 6f 65 21 38 21 31 6f 74 75 62 39
07d0: 30 6a 32 36 38 33 75 6f 69 6e 6f 36 33 38 6f 75
07e0: 38 69 6e 61 32 34 34 75 74 35 39 37 6a 61 75 6a
07f0: 21 35 30 37 6f 74 32 65 39 74 38 32 35 74 35 39
0800: 74 31 73 39 32 6a 36 39 21 68 74 38 75 21 32 6f
0810: 35 32 21 69 36 62 65 61 6e 39 69 31 33 6f 30 69
0820: 75 73 39 35 6f 68 34 6a 33 31 36 6e 38 34 6f 6f
0830: 69 61 6f 62 30 34 74 75 74 74 73 69 39 34 69 32
0840: 61 6f 37 30 73 6a 6a 69 39 74 39 69 61 34 6e 6a
0850: 6e 6f 73 6e 6f 31 32 33 68 75 33 74 37 75 39 6f
0860: 36 33 6f 75 68 69 21 37 73 6e 74 35 74 75 69 6f
0870: 68 73 62 74 61 38 73 6e 74 69 69 38 30 6f 33 36
0880: 6f 68 65 73 68 6f 31 35 31 68 74 38 34 74 74 36
0890: 31 6e 6e 36 69 75 6a 34 30 68 6f 74 31 68 32 30
08a0: 21 31 73 31 6e 68 33 74 6f 30 31 69 37 68 62 6a
08b0: 37 68 69 38 61 6a 6f 61 75 74 68 73 35 30 73 61
08c0: 38 38 32 30 62 65 74 6a 65 6a 68 62 6e 74 69 30
08d0: 61 69 34 39 65 6f 74 6a 39 6f 65 34 62 36 36 33
08e0: 21 61 69 35 75 74 33 35 68 31 74 38 38 34 34 32
08f0: 34 6f 6f 6e 31 62 74 74 38 62 6a 33 31 69 6f 74
0900: 61 6e 36 6a 21 35 21 31 6e 30 35 30 31 39 6f 36
0910: 34 38 6a 35 75 62 62 74 34 74 30 36 74 33 6f 65
0920: 36 35 21 35 30 39 33 33 6e 6a 33 37 34 32 39 35
0930: 21 73 37 33 6f 6e 69 73 6f 73 39 39 69 6f 31 62
0940: 69 37 31 74 33 73 69 39 6f 33 33 39 36 62 61 65
0950: 6f 65 61 32 38 6f 6e 30 62 65 6e 32 30 37 38 73
0960: 31 6f 31 34 62 31 30 38 73 33 69 6e 6f 6a 6a 31
0970: 75 6f 33 73 38 36 74 33 37 74 65 38 74 21 6a 6a
0980: 6a 36 32 37 75 62 68 74 6f 61 33 74 33 38 61 6e
0990: 74 38 30 68 30 37 34 37 74 75 68 38 6e 6e 6e 74
09a0: 6a 61 34 34 73 36 6a 62 37 31 6e 74 35 74 6f 65
09b0: 61 6f 21 61 74 34 62 73 37 74 65 21 21 34 30 38
09c0: 62 68 32 39 74 74 6f 73 38 31 74 39 68 69 38 6f
09d0: 6f 38 34 6e 65 33 30 36 75 6f 35 6e 6f 65 73 6a
09e0: 33 6f 34 74 39 39 37 69 30 61 39 6a 6a 74 37 37
09f0: 34 68 30 6f 31 30 6a 38 6f 21 69 38 65 73 36 68
0a00: 33 21 6a 62 36 36 62 36 74 6a 30 68 35 38 6f 39
0a10: 36 35 32 74 35 74 34 30 74 34 34 35 35 30 75 6a
0a20: 75 68 37 32 69 6f 34 30 31 68 68 6e 36 75 65 31
0a30: 65 73 38 69 38 6f 74 34 33 61 39 6a 68 32 6f 61
0a40: 30 75 69 69 39 34 69 30 36 68 6f 38 6a 34 62 75
0a50: 6f 33 21 6e 65 74 74 68 65 6f 39 21 68 6f 34 65
0a60: 35 38 35 31 32 36 31 34 69 69 32 68 38 38 34 6f
0a70: 74 31 35 32 38 6f 30 6e 31 39 6e 39 61 30 31 75
0a80: 61 6e 37 69 6a 61 38 65 6a 21 21 32 73 34 65 61
0a90: 35 69 69 6f 62 73 62 75 21 68 36 34 68 75 74 6f
0aa0: 39 34 6e 37 62 73 62 69 30 36 6e 34 21 34 36 61
0ab0: 34 62 73 33 6f 39 21 39 74 35 30 62 65 31 68 38
0ac0: 35 38 61 31 6a 74 34 61 6e 74 36 36 31 31 74 36
0ad0: 74 37 35 6e 74 61 35 37 74 36 6f 6f 33 34 6e 61
0ae0: 6f 37 36 33 38 30 34 30 74 30 32 61 38 62 74 34
0af0: 33 65 74 21 74 36 69 30 38 75 35 74 37 38 36 32
0b00: 61 38 65 39 38 6e 39 68 6e 6f 6e 73 73 6f 61 74
0b10: 34 35 75 38 38 30 38 35 33 6f 6f 74 30 61 62 65
0b20: 32 37 31 30 33 74 68 6e 35 37 75 39 75 31 6e 74
0b30: 75 39 35 74 6e 6f 36 62 32 61 36 62 39 31 74 65
0b40: 6a 68 6f 38 35 75 69 30 69 21 6e 75 38 32 74 73
0b50: 68 6f 31 37 6a 33 6f 30 6a 61 62 37 32 21 36 74
0b60: 38 73 34 6f 30 6a 21 30 34 36 75 65 38 30 32 36
0b70: 6f 33 39 33 33 6f 33 31 21 62 6a 65 35 74 6f 6f
0b80: 34 21 32 61 34 65 61 6f 73 6a 37 6f 6a 6e 73 33
0b90: 6e 30 75 62 6f 6e 21 35 61 34 74 75 61 32 6e 39
0ba0: 65 6a 74 6e 75 65 31 30 75 61 6f 31 31 39 34 74
0bb0: 39 37 36 34 21 65 6e 68 6e 33 38 38 65 69 69 30
0bc0: 6a 73 6e 31 32 69 68 62 6f 75 21 39 36 74 6f 68
0bd0: 34 6a 6f 34 21 6f 36 32 6a 33 75 21 69 21 21 6a
0be0: 30 32 39 39 6e 74 6a 31 74 34 74 39 35 39 37 39
0bf0: 37 21 69 69 35 36 32 33 36 62 21 62 35 6e 74 32
0c00: 6f 6e 68 31 33 39 6f 74 69 35 6f 35 34 21 61 68
0c10: 75 61 68 68 74 30 31 6f 36 74 68 31 74 61 33 65
0c20: 38 34 6f 37 30 31 73 69 6e 33 36 74 33 30 6a 6e
0c30: 68 6e 37 6f 6e 61 31 65 36 62 39 62 74 62 74 6a
0c40: 33 21 35 33 37 6e 33 33 62 6f 21 6f 35 34 34 69
0c50: 74 6a 6f 39 33 21 38 35 35 74 73 36 36 31 69 35
0c60: 74 36 61 38 6a 61 37 69 21 69 32 65 34 33 35 38
0c70: 6a 69 74 38 21 61 39 35 6a 33 74 74 74 31 36 31
0c80: 75 37 35 34 65 62 34 68 32 75 21 33 75 61 74 31
0c90: 35 73 35 68 6f 34 75 69 37 73 73 38 61 6e 35 38
0ca0: 33 74 6f 68 36 6f 21 34 6e 39 37 32 34 34 21 6e
0cb0: 6a 61 37 69 62 74 74 6f 34 74 74 62 75 6f 75 75
0cc0: 35 61 38 74 37 37 68 21 74 35 6f 38 39 6a 69 73
0cd0: 39 37 37 6a 38 74 37 65 31 73 37 34 32 30 74 73
0ce0: 68 61 31 62 32 32 35 36 73 73 33 33 65 33 73 68
0cf0: 30 38 6e 61 6f 69 75 39 73 74 30 75 74 74 74 32
0d00: 36 6a 38 34 74 6f 21 31 75 39 74 35 32 34 75 62
0d10: 32 62 69 74 32 74 75 75 37 74 39 38 73 6e 75 6e
0d20: 74 73 74 38 69 75 6f 21 36 62 61 34 69 38 69 32
0d30: 74 74 39 36 69 36 65 37 62 38 65 33 74 6a 36 31
0d40: 62 6f 6f 32 36 74 65 6a 6f 69 32 6f 38 36 61 34
0d50: 39 73 74 39 6e 35 74 35 33 6f 61 69 33 74 31 65
0d60: 39 75 73 65 21 6f 21 32 6f 65 61 30 34 6e 68 69
0d70: 21 6a 39 6f 33 6e 34 6f 31 36 6f 31 32 73 6f 6f
0d80: 30 69 30 75 6f 21 62 65 68 69 6f 32 75 21 36 6f
0d90: 34 35 35 34 38 39 74 36 61 75 75 73 30 36 32 74
0da0: 36 62 33 61 6f 6f 75 75 62 65 62 35 6a 31 38 6f
0db0: 6f 21 69 37 62 30 6f 74 75 36 31 33 38 33 75 30
0dc0: 6f 75 68 74 68 61 21 30 75 35 65 34 75 21 69 36
0dd0: 39 74 6f 61 73 65 65 74 37 6f 74 65 32 33 6f 74
0de0: 6e 21 73 34 34 73 34 74 62 75 35 65 6f 35 38 6f
0df0: 32 30 21 6e 32 65 6f 6e 31 39 34 69 39 6f 74 65
0e00: 6f 61 6e 69 35 37 34 74 30 6e 6f 6f 31 74 38 21
0e10: 74 37 36 36 36 6f 6f 74 34 6f 30 69 32 37 32 68
0e20: 65 6a 61 74 35 36 38 35 68 74 34 39 69 65 62 74
0e30: 6e 61 6f 68 21 38 6a 74 74 61 31 62 32 33 30 74
0e40: 38 65 34 33 37 6f 61 36 35 6e 61 35 38 69 65 30
0e50: 61 31 65 31 6f 75 6f 36 6a 39 31 30 62 6a 37 75
0e60: 21 68 6e 74 21 38 32 68 69 69 75 38 73 35 6a 6f
0e70: 6e 21 75 68 6f 61 69 74 74 31 37 39 62 61 36 6f
0e80: 6f 6f 32 6f 6e 31 74 39 68 73 74 35 62 61 30 68
0e90: 31 33 39 73 38 6f 31 6a 74 6f 73 39 33 21 61 36
0ea0: 6a 69 61 73 31 36 21 32 39 75 37 61 68 34 6e 62
0eb0: 37 32 6f 65 68 73 75 65 62 74 68 65 74 6f 73 68
0ec0: 6e 36 65 74 38 68 39 69 38 33 6a 36 37 74 61 61
0ed0: 6f 73 75 31 34 6f 6f 33 21 73 61 75 75 36 38 62
0ee0: 38 39 6e 35 74 65 6f 35 61 33 74 62 30 65 69 21
0ef0: 6e 75 74 30 36 33 69 36 73 61 65 74 74 75 6e 34
0f00: 36 6f 39 33 30 34 6a 61 73 35 69 31 37 69 74 33
0f10: 75 38 21 32 37 33 34 31 61 6f 61 68 62 74 6e 68
0f20: 36 65 31 36 33 62 74 69 74 37 34 38 74 68 37 73
0f30: 36 73 6f 33 74 74 74 6a 6e 35 21 65 32 32 21 38
0f40: 74 74 21 69 6e 74 73 75 73 61 31 31 75 61 68 6f
0f50: 6f 73 68 32 37 32 30 30 73 75 65 75 6f 68 74 39
0f60: 65 74 39 38 73 33 74 38 6a 68 35 31 33 74 39 68
0f70: 37 31 30 68 21 74 68 69 69 33 75 32 31 37 68 6f
0f80: 34 74 74 6a 73 37 32 6f 36 73 39 6f 37 6f 75 61
0f90: 6e 69 6f 36 74 73 6e 73 74 32 6e 34 6e 73 74 30
0fa0: 61 30 61 35 73 74 30 39 38 39 34 61 34 38 6e 74
0fb0: 31 6f 6f 6a 62 6f 65 32 38 6a 75 69 65 21 74 33
0fc0: 21 35 61 73 62 61 21 33 74 31 38 35 35 74 32 6f
0fd0: 36 61 74 61 30 32 74 38 74 69 38 36 33 6a 35 21
0fe0: 30 30 69 36 68 21 6f 32 74 6e 37 32 33 39 61 62
0ff0: 74 35 30 37 34 74 68 31 74 39 33 30 73 6f 75 74
cleartext:
0000: 35 38 65 74 61 69 75 62 6e 31 6e 6a 6a 33 37 38
0010: 6f 38 32 75 74 74 6a 34 62 62 62 21 69 74 6e 6f
0020: 61 38 32 39 6a 6f 6f 73 6e 31 65 74 73 62 6f 75
0030: 69 37 39 73 74 37 35 75 6f 73 6e 75 31 6f 68 6e
0040: 33 30 35 31 6f 30 68 61 31 33 35 35 6f 30 6a 65
0050: 34 38 6f 61 68 34 31 34 61 73 37 6e 62 35 65 6f
0060: 36 69 6e 74 73 37 65 61 74 73 75 36 73 61 73 61
0070: 62 33 75 74 37 73 38 32 34 61 61 69 74 75 6f 32
0080: 75 61 6f 30 65 61 65 62 35 61 31 36 6e 68 74 61
0090: 61 74 6f 6f 74 6e 74 65 31 65 35 34 61 69 75 68
00a0: 61 36 68 74 74 21 32 65 61 21 38 74 31 6a 32 36
00b0: 68 74 31 6f 75 68 30 73 69 65 6a 6a 32 31 6e 74
00c0: 74 34 30 30 73 62 32 35 6f 75 6f 21 31 61 65 32
00d0: 65 6f 37 61 31 34 62 31 6f 69 36 68 68 65 6f 61
00e0: 6e 35 61 75 34 69 65 39 21 34 62 74 36 38 39 37
00f0: 30 69 65 38 38 37 6f 74 38 31 6f 62 6f 69 30 35
0100: cd ba bb d9 ef fa 5e c4 cd d3 8a 9e cd 69 5f 39
0110: 61 65 75 62 36 35 6e 33 6e 31 6f 35 34 6f 30 6f
0120: 30 6a 62 35 6a 38 21 6f 65 65 68 73 69 31 6f 61
0130: 6f 62 74 35 74 33 61 33 73 69 6a 6a 6f 61 37 35
0140: 21 6e 30 34 30 74 39 32 6f 74 6f 6f 65 35 69 31
0150: 74 73 6e 6a 74 31 31 37 31 6e 35 6e 37 62 68 74
0160: 35 6a 31 65 61 37 73 69 38 33 65 39 61 65 74 36
0170: 32 33 69 36 74 31 6f 68 74 34 34 31 69 36 38 35
0180: 30 36 74 35 6f 38 35 30 74 69 75 36 36 69 38 38
0190: 33 38 69 73 6f 6f 35 32 37 39 33 74 68 37 32 65
01a0: 6f 62 69 65 33 6f 62 73 65 35 6e 73 35 69 6f 6a
01b0: 31 6f 65 6e 31 73 6a 6a 21 31 35 68 6f 37 21 38
01c0: 33 21 21 6e 6a 6f 75 75 30 32 74 35 32 36 21 33
01d0: 73 65 39 61 32 73 21 65 35 61 21 30 62 39 38 6f
01e0: 62 73 31 69 32 38 30 62 75 37 73 62 6f 73 6f 75
01f0: 32 38 38 34 65 37 6e 69 62 6e 69 34 65 38 6f 38
0200: 65 69 38 74 38 34 21 34 68 30 21 74 74 6a 69 39
0210: 6f 6a 69 31 35 75 31 74 62 31 61 37 6f 73 68 74
0220: 34 39 34 65 69 73 6f 65 74 32 62 6a 37 6f 37 75
0230: 74 31 37 32 73 6a 33 6f 62 34 75 73 35 6a 31 39
0240: 37 35 38 37 36 74 62 6f 39 6a 74 33 37 6a 39 34
0250: 36 33 75 39 38 6e 6f 33 37 68 37 32 39 61 6a 33
0260: 75 36 37 6f 6f 6e 31 32 74 65 65 34 75 68 6f 65
0270: 31 65 38 62 38 6f 65 68 73 36 69 30 30 68 21 33
0280: 73 74 68 75 21 39 6f 74 38 37 68 73 6f 75 32 6e
0290: 6a 75 61 6f 36 74 73 6e 62 34 6e 39 32 6e 62 6e
02a0: 39 74 6f 62 21 38 6e 73 61 37 38 21 69 75 6e 6f
02b0: 6f 37 31 61 74 68 74 21 36 69 62 6a 6e 38 74 35
02c0: 61 21 34 31 38 30 61 62 73 62 36 73 35 6a 32 34
02d0: 35 21 62 6e 68 21 36 69 33 31 6a 62 35 62 74 30
02e0: 6f 61 38 6f 68 6f 6f 65 6a 73 6e 69 65 61 38 74
02f0: 68 69 39 39 21 61 6f 6f 36 31 6a 38 75 6f 38 68
0300: 30 65 34 68 31 6f 69 39 33 6e 6f 68 37 74 6a 6f
0310: 74 38 38 75 62 37 62 68 6a 69 62 34 32 33 36 32
0320: 61 75 30 6e 36 30 35 35 6f 34 75 30 38 61 35 61
0330: 62 21 31 38 74 74 36 68 30 34 32 62 37 34 38 32
0340: 38 6a 61 33 6a 21 6f 31 74 68 38 6a 6e 39 6f 68
0350: 62 73 6e 32 74 68 69 37 6e 32 75 21 74 73 65 34
0360: 65 75 73 74 6f 62 34 33 74 6f 68 33 38 74 68 74
0370: 34 37 36 68 35 6e 62 38 6a 65 38 68 36 39 32 34
0380: 6f 75 38 65 65 65 35 62 30 33 32 38 69 30 31 61
0390: 37 74 6f 62 6f 6a 74 37 21 34 62 32 30 31 75 39
03a0: 73 73 68 6f 32 33 68 62 6e 30 75 6e 30 34 37 73
03b0: 38 30 33 74 6a 21 73 61 73 39 74 74 37 33 30 31
03c0: 69 68 30 32 34 65 31 74 6f 37 39 62 6a 74 6f 36
03d0: 69 32 32 6a 65 33 6f 32 32 39 33 39 62 21 37 62
03e0: 31 37 31 65 36 65 35 34 6e 31 21 73 38 32 75 38
03f0: 68 35 6e 33 6a 6f 32 37 61 65 75 69 68 69 39 32
0400: 38 30 74 33 32 36 37 61 33 69 6f 31 6f 68 35 73
0410: 74 31 69 68 75 36 6f 31 37 65 31 6f 65 30 37 39
0420: 37 38 6f 6e 21 75 74 33 30 36 74 6f 37 32 33 74
0430: 69 75 65 75 32 37 37 6e 62 61 61 74 65 6f 74 62
0440: 33 39 68 39 32 65 62 62 37 6e 6a 34 6f 31 68 65
0450: 34 69 6a 75 37 65 62 6e 33 30 33 61 62 21 30 65
0460: 36 68 38 61 21 74 69 37 6f 6f 6a 31 6e 21 39 30
0470: 21 37 33 74 36 32 39 62 32 62 30 38 62 37 6e 31
0480: 35 73 36 35 74 6e 62 21 34 74 31 74 33 30 37 6f
0490: 34 74 6e 21 36 65 6f 6a 31 36 32 75 6f 6f 37 38
04a0: 32 33 6f 39 62 68 62 21 35 75 75 6a 74 30 38 38
04b0: 37 30 37 39 32 6f 37 21 6a 75 6e 37 61 21 61 69
04c0: 6f 34 39 6a 32 6a 38 73 68 31 62 61 31 75 32 61
04d0: 75 39 34 35 6f 6a 68 33 34 37 30 36 37 65 62 75
04e0: 6e 6a 61 61 33 62 36 31 21 34 36 38 74 38 32 69
04f0: 69 33 36 32 6a 74 35 32 31 36 61 61 73 30 31 75
0500: 35 32 31 6a 68 75 36 31 74 31 6f 38 6f 38 6a 6f
0510: 74 69 74 6f 30 6f 38 38 61 32 32 6f 62 69 61 73
0520: 32 32 6f 69 36 73 61 6e 61 21 74 21 37 6f 6f 61
0530: 65 36 21 6f 6a 73 75 39 6f 35 38 65 6f 62 36 38
0540: 68 34 31 74 6a 32 39 21 32 75 39 6e 61 34 37 33
0550: 30 37 65 61 31 61 34 73 21 6f 36 35 34 31 6f 36
0560: 65 69 62 30 6f 61 21 61 68 36 37 6e 74 61 39 74
0570: 65 38 6a 39 32 62 33 6f 34 35 69 6f 33 6e 74 61
0580: 75 39 61 62 73 31 30 35 69 37 68 30 6f 37 69 74
0590: 61 68 75 69 37 6e 65 68 6a 32 75 31 6a 74 36 31
05a0: 6e 37 75 75 61 33 6f 6a 37 75 6a 36 69 68 6f 61
05b0: 21 36 6a 69 68 75 68 39 6f 38 74 69 34 6f 31 74
05c0: 75 38 21 31 74 6a 39 38 34 73 30 34 36 6f 62 65
05d0: 6a 69 35 6e 30 75 32 6f 73 62 6f 61 6a 21 6a 34
05e0: 37 34 32 34 6f 68 62 69 35 62 6a 37 6a 74 62 6f
05f0: 37 73 6f 37 74 37 32 37 33 38 65 31 37 75 32 31
0600: 31 34 35 30 36 74 30 37 39 6a 68 73 65 30 75 62
0610: 31 65 6f 34 62 68 65 65 62 69 21 6e 74 65 69 34
0620: 6f 36 31 61 6e 31 65 32 39 33 6e 38 30 6f 33 38
0630: 34 36 65 21 69 69 65 6f 73 6a 35 37 75 62 68 62
0640: 34 32 69 74 21 65 62 36 35 35 33 65 38 6e 6e 65
0650: 6a 38 68 6f 6a 21 38 6f 34 6f 36 74 6a 74 39 62
0660: 36 6f 6e 65 61 61 38 75 21 74 61 73 69 74 32 6a
0670: 61 74 33 6f 30 74 31 68 30 33 65 6a 75 6a 73 65
0680: 37 21 73 6f 31 6a 61 38 35 6e 68 6f 33 73 69 6a
0690: 34 75 37 31 73 35 62 73 6a 31 6f 61 36 32 33 6f
06a0: 6f 69 68 21 68 6f 73 74 36 65 65 6f 36 65 75 75
06b0: 6f 73 34 32 36 21 6f 69 37 31 61 6f 69 6a 36 65
06c0: 6a 69 6a 21 6e 65 33 68 73 61 74 6e 74 33 39 65
06d0: 69 30 74 62 21 33 39 37 73 73 75 73 6f 65 6f 74
06e0: 65 74 33 61 39 6e 6f 21 37 39 32 74 6f 68 62 6f
06f0: 6a 39 6f 61 62 34 6f 21 61 61 69 69 69 6e 74 35
0700: 69 21 21 6f 32 6f 32 6e 65 34 74 31 6f 36 74 37
0710: 65 62 65 74 21 30 74 34 35 74 34 35 33 31 30 75
0720: 37 30 36 6e 39 38 32 31 62 62 62 6a 31 39 35 6f
0730: 61 73 30 34 74 74 6f 36 73 33 37 73 74 38 30 6a
0740: 34 6e 73 69 30 6f 31 36 68 21 69 39 36 36 65 30
0750: 39 6f 31 21 32 21 32 6f 39 39 75 75 69 74 31 38
0760: 39 65 31 39 34 32 65 6f 6f 35 65 34 37 30 34 33
0770: 62 36 33 38 65 65 30 38 38 69 73 68 74 65 62 6e
0780: 69 74 65 38 62 30 30 37 36 65 68 39 32 6f 6f 38
0790: 61 32 65 75 73 6f 74 6a 65 31 39 6f 6f 35 31 6e
07a0: 65 21 73 74 21 74 34 65 6f 6f 68 68 35 74 36 21
07b0: 6f 37 6f 75 36 21 34 34 74 74 39 69 36 37 39 37
07c0: 69 21 34 21 74 6f 65 21 38 21 31 6f 74 75 62 39
07d0: 30 6a 32 36 38 33 75 6f 69 6e 6f 36 33 38 6f 75
07e0: 38 69 6e 61 32 34 34 75 74 35 39 37 6a 61 75 6a
07f0: 21 35 30 37 6f 74 32 65 39 74 38 32 35 74 35 39
0800: 74 31 73 39 32 6a 36 39 21 68 74 38 75 21 32 6f
0810: 35 32 21 69 36 62 65 61 6e 39 69 31 33 6f 30 69
0820: 75 73 39 35 6f 68 34 6a 33 31 36 6e 38 34 6f 6f
0830: 69 61 6f 62 30 34 74 75 74 74 73 69 39 34 69 32
0840: 61 6f 37 30 73 6a 6a 69 39 74 39 69 61 34 6e 6a
0850: 6e 6f 73 6e 6f 31 32 33 68 75 33 74 37 75 39 6f
0860: 36 33 6f 75 68 69 21 37 73 6e 74 35 74 75 69 6f
0870: 68 73 62 74 61 38 73 6e 74 69 69 38 30 6f 33 36
0880: 6f 68 65 73 68 6f 31 35 31 68 74 38 34 74 74 36
0890: 31 6e 6e 36 69 75 6a 34 30 68 6f 74 31 68 32 30
08a0: 21 31 73 31 6e 68 33 74 6f 30 31 69 37 68 62 6a
08b0: 37 68 69 38 61 6a 6f 61 75 74 68 73 35 30 73 61
08c0: 38 38 32 30 62 65 74 6a 65 6a 68 62 6e 74 69 30
08d0: 61 69 34 39 65 6f 74 6a 39 6f 65 34 62 36 36 33
08e0: 21 61 69 35 75 74 33 35 68 31 74 38 38 34 34 32
08f0: 34 6f 6f 6e 31 62 74 74 38 62 6a 33 31 69 6f 74
0900: 61 6e 36 6a 21 35 21 31 6e 30 35 30 31 39 6f 36
0910: 34 38 6a 35 75 62 62 74 34 74 30 36 74 33 6f 65
0920: 36 35 21 35 30 39 33 33 6e 6a 33 37 34 32 39 35
0930: 21 73 37 33 6f 6e 69 73 6f 73 39 39 69 6f 31 62
0940: 69 37 31 74 33 73 69 39 6f 33 33 39 36 62 61 65
0950: 6f 65 61 32 38 6f 6e 30 62 65 6e 32 30 37 38 73
0960: 31 6f 31 34 62 31 30 38 73 33 69 6e 6f 6a 6a 31
0970: 75 6f 33 73 38 36 74 33 37 74 65 38 74 21 6a 6a
0980: 6a 36 32 37 75 62 68 74 6f 61 33 74 33 38 61 6e
0990: 74 38 30 68 30 37 34 37 74 75 68 38 6e 6e 6e 74
09a0: 6a 61 34 34 73 36 6a 62 37 31 6e 74 35 74 6f 65
09b0: 61 6f 21 61 74 34 62 73 37 74 65 21 21 34 30 38
09c0: 62 68 32 39 74 74 6f 73 38 31 74 39 68 69 38 6f
09d0: 6f 38 34 6e 65 33 30 36 75 6f 35 6e 6f 65 73 6a
09e0: 33 6f 34 74 39 39 37 69 30 61 39 6a 6a 74 37 37
09f0: 34 68 30 6f 31 30 6a 38 6f 21 69 38 65 73 36 68
0a00: 33 21 6a 62 36 36 62 36 74 6a 30 68 35 38 6f 39
0a10: 36 35 32 74 35 74 34 30 74 34 34 35 35 30 75 6a
0a20: 75 68 37 32 69 6f 34 30 31 68 68 6e 36 75 65 31
0a30: 65 73 38 69 38 6f 74 34 33 61 39 6a 68 32 6f 61
0a40: 30 75 69 69 39 34 69 30 36 68 6f 38 6a 34 62 75
0a50: 6f 33 21 6e 65 74 74 68 65 6f 39 21 68 6f 34 65
0a60: 35 38 35 31 32 36 31 34 69 69 32 68 38 38 34 6f
0a70: 74 31 35 32 38 6f 30 6e 31 39 6e 39 61 30 31 75
0a80: 61 6e 37 69 6a 61 38 65 6a 21 21 32 73 34 65 61
0a90: 35 69 69 6f 62 73 62 75 21 68 36 34 68 75 74 6f
0aa0: 39 34 6e 37 62 73 62 69 30 36 6e 34 21 34 36 61
0ab0: 34 62 73 33 6f 39 21 39 74 35 30 62 65 31 68 38
0ac0: 35 38 61 31 6a 74 34 61 6e 74 36 36 31 31 74 36
0ad0: 74 37 35 6e 74 61 35 37 74 36 6f 6f 33 34 6e 61
0ae0: 6f 37 36 33 38 30 34 30 74 30 32 61 38 62 74 34
0af0: 33 65 74 21 74 36 69 30 38 75 35 74 37 38 36 32
0b00: 61 38 65 39 38 6e 39 68 6e 6f 6e 73 73 6f 61 74
0b10: 34 35 75 38 38 30 38 35 33 6f 6f 74 30 61 62 65
0b20: 32 37 31 30 33 74 68 6e 35 37 75 39 75 31 6e 74
0b30: 75 39 35 74 6e 6f 36 62 32 61 36 62 39 31 74 65
0b40: 6a 68 6f 38 35 75 69 30 69 21 6e 75 38 32 74 73
0b50: 68 6f 31 37 6a 33 6f 30 6a 61 62 37 32 21 36 74
0b60: 38 73 34 6f 30 6a 21 30 34 36 75 65 38 30 32 36
0b70: 6f 33 39 33 33 6f 33 31 21 62 6a 65 35 74 6f 6f
0b80: 34 21 32 61 34 65 61 6f 73 6a 37 6f 6a 6e 73 33
0b90: 6e 30 75 62 6f 6e 21 35 61 34 74 75 61 32 6e 39
0ba0: 65 6a 74 6e 75 65 31 30 75 61 6f 31 31 39 34 74
0bb0: 39 37 36 34 21 65 6e 68 6e 33 38 38 65 69 69 30
0bc0: 6a 73 6e 31 32 69 68 62 6f 75 21 39 36 74 6f 68
0bd0: 34 6a 6f 34 21 6f 36 32 6a 33 75 21 69 21 21 6a
0be0: 30 32 39 39 6e 74 6a 31 74 34 74 39 35 39 37 39
0bf0: 37 21 69 69 35 36 32 33 36 62 21 62 35 6e 74 32
0c00: 6f 6e 68 31 33 39 6f 74 69 35 6f 35 34 21 61 68
0c10: 75 61 68 68 74 30 31 6f 36 74 68 31 74 61 33 65
0c20: 38 34 6f 37 30 31 73 69 6e 33 36 74 33 30 6a 6e
0c30: 68 6e 37 6f 6e 61 31 65 36 62 39 62 74 62 74 6a
0c40: 33 21 35 33 37 6e 33 33 62 6f 21 6f 35 34 34 69
0c50: 74 6a 6f 39 33 21 38 35 35 74 73 36 36 31 69 35
0c60: 74 36 61 38 6a 61 37 69 21 69 32 65 34 33 35 38
0c70: 6a 69 74 38 21 61 39 35 6a 33 74 74 74 31 36 31
0c80: 75 37 35 34 65 62 34 68 32 75 21 33 75 61 74 31
0c90: 35 73 35 68 6f 34 75 69 37 73 73 38 61 6e 35 38
0ca0: 33 74 6f 68 36 6f 21 34 6e 39 37 32 34 34 21 6e
0cb0: 6a 61 37 69 62 74 74 6f 34 74 74 62 75 6f 75 75
0cc0: 35 61 38 74 37 37 68 21 74 35 6f 38 39 6a 69 73
0cd0: 39 37 37 6a 38 74 37 65 31 73 37 34 32 30 74 73
0ce0: 68 61 31 62 32 32 35 36 73 73 33 33 65 33 73 68
0cf0: 30 38 6e 61 6f 69 75 39 73 74 30 75 74 74 74 32
0d00: 36 6a 38 34 74 6f 21 31 75 39 74 35 32 34 75 62
0d10: 32 62 69 74 32 74 75 75 37 74 39 38 73 6e 75 6e
0d20: 74 73 74 38 69 75 6f 21 36 62 61 34 69 38 69 32
0d30: 74 74 39 36 69 36 65 37 62 38 65 33 74 6a 36 31
0d40: 62 6f 6f 32 36 74 65 6a 6f 69 32 6f 38 36 61 34
0d50: 39 73 74 39 6e 35 74 35 33 6f 61 69 33 74 31 65
0d60: 39 75 73 65 21 6f 21 32 6f 65 61 30 34 6e 68 69
0d70: 21 6a 39 6f 33 6e 34 6f 31 36 6f 31 32 73 6f 6f
0d80: 30 69 30 75 6f 21 62 65 68 69 6f 32 75 21 36 6f
0d90: 34 35 35 34 38 39 74 36 61 75 75 73 30 36 32 74
0da0: 36 62 33 61 6f 6f 75 75 62 65 62 35 6a 31 38 6f
0db0: 6f 21 69 37 62 30 6f 74 75 36 31 33 38 33 75 30
0dc0: 6f 75 68 74 68 61 21 30 75 35 65 34 75 21 69 36
0dd0: 39 74 6f 61 73 65 65 74 37 6f 74 65 32 33 6f 74
0de0: 6e 21 73 34 34 73 34 74 62 75 35 65 6f 35 38 6f
0df0: 32 30 21 6e 32 65 6f 6e 31 39 34 69 39 6f 74 65
0e00: 6f 61 6e 69 35 37 34 74 30 6e 6f 6f 31 74 38 21
0e10: 74 37 36 36 36 6f 6f 74 34 6f 30 69 32 37 32 68
0e20: 65 6a 61 74 35 36 38 35 68 74 34 39 69 65 62 74
0e30: 6e 61 6f 68 21 38 6a 74 74 61 31 62 32 33 30 74
0e40: 38 65 34 33 37 6f 61 36 35 6e 61 35 38 69 65 30
0e50: 61 31 65 31 6f 75 6f 36 6a 39 31 30 62 6a 37 75
0e60: 21 68 6e 74 21 38 32 68 69 69 75 38 73 35 6a 6f
0e70: 6e 21 75 68 6f 61 69 74 74 31 37 39 62 61 36 6f
0e80: 6f 6f 32 6f 6e 31 74 39 68 73 74 35 62 61 30 68
0e90: 31 33 39 73 38 6f 31 6a 74 6f 73 39 33 21 61 36
0ea0: 6a 69 61 73 31 36 21 32 39 75 37 61 68 34 6e 62
0eb0: 37 32 6f 65 68 73 75 65 62 74 68 65 74 6f 73 68
0ec0: 6e 36 65 74 38 68 39 69 38 33 6a 36 37 74 61 61
0ed0: 6f 73 75 31 34 6f 6f 33 21 73 61 75 75 36 38 62
0ee0: 38 39 6e 35 74 65 6f 35 61 33 74 62 30 65 69 21
0ef0: 6e 75 74 30 36 33 69 36 73 61 65 74 74 75 6e 34
0f00: 36 6f 39 33 30 34 6a 61 73 35 69 31 37 69 74 33
0f10: 75 38 21 32 37 33 34 31 61 6f 61 68 62 74 6e 68
0f20: 36 65 31 36 33 62 74 69 74 37 34 38 74 68 37 73
0f30: 36 73 6f 33 74 74 74 6a 6e 35 21 65 32 32 21 38
0f40: 74 74 21 69 6e 74 73 75 73 61 31 31 75 61 68 6f
0f50: 6f 73 68 32 37 32 30 30 73 75 65 75 6f 68 74 39
0f60: 65 74 39 38 73 33 74 38 6a 68 35 31 33 74 39 68
0f70: 37 31 30 68 21 74 68 69 69 33 75 32 31 37 68 6f
0f80: 34 74 74 6a 73 37 32 6f 36 73 39 6f 37 6f 75 61
0f90: 6e 69 6f 36 74 73 6e 73 74 32 6e 34 6e 73 74 30
0fa0: 61 30 61 35 73 74 30 39 38 39 34 61 34 38 6e 74
0fb0: 31 6f 6f 6a 62 6f 65 32 38 6a 75 69 65 21 74 33
0fc0: 21 35 61 73 62 61 21 33 74 31 38 35 35 74 32 6f
0fd0: 36 61 74 61 30 32 74 38 74 69 38 36 33 6a 35 21
0fe0: 30 30 69 36 68 21 6f 32 74 6e 37 32 33 39 61 62
0ff0: 74 35 30 37 34 74 68 31 74 39 33 30 73 6f 75 74
   0.002 sec,      20    aes crypts,    4096 bytes, 40756219 byte/sec,   310.9
Mb/sec
   0.001 sec,      20    aes crypts,    8192 bytes, 124498480 byte/sec,   949.8
Mb/sec
   0.000 sec,      20 aes192 crypts,      16 bytes,  2990654 byte/sec,    22.8
Mb/sec
   0.000 sec,      20 aes192 crypts,      32 bytes,  8888889 byte/sec,    67.8
Mb/sec
   0.000 sec,      20 aes192 crypts,      64 bytes, 18550725 byte/sec,   141.5
Mb/sec
   0.000 sec,      20 aes192 crypts,     128 bytes, 32820513 byte/sec,   250.4
Mb/sec
   0.000 sec,      20 aes192 crypts,     256 bytes, 62439024 byte/sec,   476.4
Mb/sec
   0.000 sec,      20 aes192 crypts,     512 bytes, 106666667 byte/sec,   813.8
Mb/sec
   0.000 sec,      20 aes192 crypts,    1024 bytes, 179649123 byte/sec,  1370.6
Mb/sec
   0.000 sec,      20 aes192 crypts,    2048 bytes, 182857143 byte/sec,  1395.1
Mb/sec
   0.000 sec,      20 aes192 crypts,    4096 bytes, 273979933 byte/sec,  2090.3
Mb/sec
   0.001 sec,      20 aes192 crypts,    8192 bytes, 259651347 byte/sec,  1981.0
Mb/sec
   0.000 sec,      20 aes256 crypts,      16 bytes,  4210526 byte/sec,    32.1
Mb/sec
   0.000 sec,      20 aes256 crypts,      32 bytes,  9142857 byte/sec,    69.8
Mb/sec
   0.000 sec,      20 aes256 crypts,      64 bytes, 18823529 byte/sec,   143.6
Mb/sec
   0.000 sec,      20 aes256 crypts,     128 bytes, 32820513 byte/sec,   250.4
Mb/sec
   0.000 sec,      20 aes256 crypts,     256 bytes, 63209877 byte/sec,   482.3
Mb/sec
   0.000 sec,      20 aes256 crypts,     512 bytes, 112527473 byte/sec,   858.5
Mb/sec
   0.000 sec,      20 aes256 crypts,    1024 bytes, 170666667 byte/sec,  1302.1
Mb/sec
   0.000 sec,      20 aes256 crypts,    2048 bytes, 119766082 byte/sec,   913.7
Mb/sec
   0.000 sec,      20 aes256 crypts,    4096 bytes, 298978102 byte/sec,  2281.0
Mb/sec
   0.001 sec,      20 aes256 crypts,    8192 bytes, 248242424 byte/sec,  1893.9
Mb/sec

On Sat, Dec 11, 2010 at 06:08:08PM -0500, Mike Tancsa
wrote:
> On 12/11/2010 11:01 AM, Kostik Belousov wrote:
> > 
> > I have no access to AESNI hardware. For start, you may use
> > src/tools/tools/crypto/cryptotest
> > to somewhat verify the sanity of the driver.
> 
> I doesnt happen every time, but one out of 5 or so 
> 
First, which arch is it, amd64 or i386 ?

Also, please revert r216162 and do the same tests.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 196 bytes
Desc: not available
Url :
http://lists.freebsd.org/pipermail/freebsd-stable/attachments/20101211/aa7f4609/attachment.pgp

On 12/12/2010 15:26, Mike Tancsa wrote:
> On 12/12/2010 3:43 AM, Kostik Belousov wrote:
>> Please try this patch on the latest HEAD or RELENG_8.
>>
>
> I tried both on i386 and amd64 and all looks good!  I did 1000
> iterations of
> cryptotest -c -z -t 10
>
> without issue!  Thanks for the quick fix!
I redid my testing on a (read-only attached) geli provider (now with XTS 
instead of CBC) taking checksums of all files:

8.2-BETA1 without patch without aesni loaded: all checksums ok.
8.2-BETA1 with patch without aesni loaded: all checksums ok.
8.2-BETA1 without patch with aesni loaded: 124 of 18209 failures.
8.2-BETA1 with patch with aesni loaded: all checksums ok.

The patch seems to fix my geli problem on 8.2-BETA1 amd64, too.

Thanks!
Jan Henrik

On 12/16/2010 8:14 PM, jhell wrote:
> On 12/16/2010 20:10, jhell wrote:
>> Regarding this patch(r216455) and r216162 I have had to back both of
>> them out of my local tree to avoid panics on a ZFS & UFS2 i386
system.

I have zfs & UFS on AMD64 and have not seen any issues.  How do you
trigger the issue in i386 ?   Or does it just randomly happen ?

	---Mike

On 12/17/2010 09:01, Mike Tancsa wrote:
> On 12/16/2010 8:14 PM, jhell wrote:
>> On 12/16/2010 20:10, jhell wrote:
>>> Regarding this patch(r216455) and r216162 I have had to back both
of
>>> them out of my local tree to avoid panics on a ZFS & UFS2 i386
system.
> 
> 
> I have zfs & UFS on AMD64 and have not seen any issues.  How do you
> trigger the issue in i386 ?   Or does it just randomly happen ?
> 
> 	---Mike
I can simply run a scrub on the pool and within a little while within
12% - 18% it goes ker-plunk!. Not that it really has to do with the
percentage rage as it does the timing.

I do have prefetch enabled when this happens and currently running with
it turned off and will booting so said kernel.bad in a little while to
see if that makes a difference.

-- 

 jhell,v