Libguestfs - Oct 2013 - Re: Hivex - Trailing garbage at the end of hive file

(Resending since my message from 2 days ago didn't make it to the list.
Perhaps because of the log file attachment)

Hi Rich,

Sorry about the delayed response. I understand that since hive formats are
not open, the current tack of warning about the hive file is a better one

To respond to you question about "whether it fails", hivexsh is unable
to
open the hive file and it prints this message and exits. I've attached the
verbose logs as requested

This extract from the logs shows that the hivexsh complains content after
file offset 0x77c000 is garbage

hivex: badsys: trailing garbage at end of file (at 0x77c000, after 1849
pages)

So, I went ahead and truncated the contents of the file after that file
offset and hivex was able to successfully open the new hive file

The sizes of the different hive files shown here

[root@ip-10-66-209-246 ~]# ls -als bads*
7664 -rw--w--w- 1 root root 7847936 Oct 1 21:51 badsys (SYSTEM hive file
after truncating trailing garbage)
7680 -rw------- 1 root root 7864320 Oct 1 21:21 badsysback (original
SYSTEM hive file)
16 -rw-r--r-- 1 root root 16384 Oct 1 21:55 badsysgarb (16K of truncated
garbage)


Looking into hive file, the last 16K is what hivex complains to be
trailing garbage. All of that 16K is zeroes.

[root@ip-10-66-209-246 ~]# dd if=./badsysback of=./badsysgarb bs=4k
skip=1916
4+0 records in
4+0 records out
16384 bytes (16 kB) copied, 9.1288e-05 s, 179 MB/s


[root@ip-10-66-209-246 ~]# hexdump badsysgarb
0000000 0000 0000 0000 0000 0000 0000 0000 0000
*
0004000

I'm not sure what the filesystem cluster size is but could the SYSTEM hive
file be padded with 16K of zeroes to match 32k cluster allocation (or
something like that?)

I imported both hive files (original one that hivex complained about and
the truncated one) into a windows regedit and the tool opened them both
fine and showed pretty much the same content

I'm going to see if the hive file I have has any sensitive customer
information and will try to share it if it does not

Thanks for you help!
~ Hari



On 9/25/13 4:59 AM, "Richard W.M. Jones" <rjones@redhat.com>
wrote:
>On Tue, Sep 24, 2013 at 09:31:26PM +0000, Subramanian, Hari wrote:
>> Is there a reason why a previous version of hivex ignored this
condition
>> while more recent ones throw an error?
>
>We don't understand all aspects of the hive format, so it's better
to
>warn about these things than ignore them.
>
>> I do have some hive files that are failing (I'm yet to validate if
they
>> are good or were corrupted some other way). I will do so and share my
>> findings
>
>When you say "that are failing" do you mean they actually fail
(and if
>so how -- full verbose logs please) or they print this warning, which
>is not a failure?
>
>Rich.
>
>-- 
>Richard Jones, Virtualization Group, Red Hat
>http://people.redhat.com/~rjones
>virt-top is 'top' for virtual machines.  Tiny program with many
>powerful monitoring features, net stats, disk stats, logging, etc.
>http://people.redhat.com/~rjones/virt-top

On Fri, Oct 04, 2013 at 02:12:08PM +0000, Subramanian, Hari
wrote:
> To respond to you question about "whether it fails", hivexsh is
unable to
> open the hive file and it prints this message and exits. I've attached
the
> verbose logs as requested
OK, I understand it now.  It is in fact failing, setting
errno = ENOSYS and returning an error.
> This extract from the logs shows that the hivexsh complains content after
> file offset 0x77c000 is garbage
> 
> hivex: badsys: trailing garbage at end of file (at 0x77c000, after 1849
> pages)
> 
> So, I went ahead and truncated the contents of the file after that file
> offset and hivex was able to successfully open the new hive file
I guess if the hive comes from a real guest we can change this to warn
but not fail.

Should be a pretty simple patch.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
virt-df lists disk usage of guests without needing to install any
software inside the virtual machine.  Supports Linux and Windows.
http://people.redhat.com/~rjones/virt-df/

Rich,

Our workflow is something like this:

0. Start with a fresh copy of windows server 2k8
1. We read the system hive and then write to it a bunch of times
2. Boot windows
3. Read from the system hive

Hivex reports the failure at step #3. I also noticed that the size of the
registry hive observed in step #3 is the same as step #0. Is it possible
that hivex issues write that cause a hive file to shrink in size and while
compacting the hive file, it retains the size but zeroes out the end of
the file? That would point to the trailing zeroes getting introduced in
step #1

It's also possible that windows is padding those zeroes in step #2. I'm
adding some instrumentation to confirm this

Thanks for your help!
~ Hari

On 10/4/13 11:01 AM, "Richard W.M. Jones" <rjones@redhat.com>
wrote:
>On Fri, Oct 04, 2013 at 02:12:08PM +0000, Subramanian, Hari wrote:
>> To respond to you question about "whether it fails", hivexsh
is unable
>>to
>> open the hive file and it prints this message and exits. I've
attached
>>the
>> verbose logs as requested
>
>OK, I understand it now.  It is in fact failing, setting
>errno = ENOSYS and returning an error.
>
>> This extract from the logs shows that the hivexsh complains content
>>after
>> file offset 0x77c000 is garbage
>> 
>> hivex: badsys: trailing garbage at end of file (at 0x77c000, after 1849
>> pages)
>> 
>> So, I went ahead and truncated the contents of the file after that file
>> offset and hivex was able to successfully open the new hive file
>
>I guess if the hive comes from a real guest we can change this to warn
>but not fail.
>
>Should be a pretty simple patch.
>
>Rich.
>
>-- 
>Richard Jones, Virtualization Group, Red Hat
>http://people.redhat.com/~rjones
>virt-df lists disk usage of guests without needing to install any
>software inside the virtual machine.  Supports Linux and Windows.
>http://people.redhat.com/~rjones/virt-df/