Libguestfs - Sep 2013 - Re: Hivex - Trailing garbage at the end of hive file

Is there a reason why a previous version of hivex ignored this condition
while more recent ones throw an error?

I do have some hive files that are failing (I'm yet to validate if they
are good or were corrupted some other way). I will do so and share my
findings


Thanks
~ Hari 

On 9/24/13 5:20 PM, "Richard W.M. Jones" <rjones@redhat.com>
wrote:
>On Tue, Sep 24, 2013 at 10:14:12PM +0100, Richard W.M. Jones wrote:
>> [Forwarding to the proper list]
>>
>> We use hivex for one of our projects and I see the following diff
>> committed to the hivex project (source link provided) that ignores
>> trailing garbage.  But this code has been reverted to no ignore this
>> condition in later version of hivex
>>
>> -      /* This error is seemingly common in uncorrupt registry files.
*/
>> -      /*
>> -      fprintf (stderr, "hivex: %s: ignoring trailing garbage at
end of
>>file (at %zu, after %zu pages)\n",
>> -               filename, off, h->pages);
>> -      */
>> +      /* NB: This error is seemingly common in uncorrupt registry
>>files. */
>> +      if (h->msglvl >= 2)
>> +        fprintf (stderr, "hivex: %s: ignoring trailing garbage at
end
>>of file (at %zu, after %zu pages)\n",
>> +                 filename, off, h->pages);
>>        Break;
>> 
>> 
>>http://git.annexia.org/?p=hivex.git;a=commitdiff;h=5439842dc591accbf620a5
>>f76dd6f04ad4d5c971
>
>> In your experience, is this a common condition seen in some windows
>> hive files and is it safe to ignore when this condition is
>> encountered?
>
>I have seen this, rarely, in registry hives.  I don't understand at
>all why it would happen, but would welcome known-good hives that
>demonstrate the problem.  Remember that the registry format is
>completely undocumented, so it's all guesswork :-(
>
>The commit you mention just changes the message so it is printed when
>debugging is enabled, but is otherwise the same, so I'm not sure what
>precisely the problem is.  It should just print the message (if
>debugging is enabled) and continue.
>
>Rich.
>
>-- 
>Richard Jones, Virtualization Group, Red Hat
>http://people.redhat.com/~rjones
>virt-top is 'top' for virtual machines.  Tiny program with many
>powerful monitoring features, net stats, disk stats, logging, etc.
>http://people.redhat.com/~rjones/virt-top

On Tue, Sep 24, 2013 at 09:31:26PM +0000, Subramanian, Hari
wrote:
> Is there a reason why a previous version of hivex ignored this condition
> while more recent ones throw an error?
We don't understand all aspects of the hive format, so it's better to
warn about these things than ignore them.
> I do have some hive files that are failing (I'm yet to validate if they
> are good or were corrupted some other way). I will do so and share my
> findings
When you say "that are failing" do you mean they actually fail (and if
so how -- full verbose logs please) or they print this warning, which
is not a failure?

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
virt-top is 'top' for virtual machines.  Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top

Hi Rich,

Sorry about the delayed response. I understand that since hive formats are
not open, the current tack of warning about the hive file is a better one

To respond to you question about "whether it fails", hivexsh is unable
to
open the hive file and it prints this message and exits. I've attached the
verbose logs as requested

This extract from the logs shows that the hivexsh complains content after
file offset 0x77c000 is garbage

hivex: badsys: trailing garbage at end of file (at 0x77c000, after 1849
pages) 

So, I went ahead and truncated the contents of the file after that file
offset and hivex was able to successfully open the new hive file

The sizes of the different hive files shown here

[root@ip-10-66-209-246 ~]# ls -als bads*
7664 -rw--w--w- 1 root root 7847936 Oct 1 21:51 badsys (SYSTEM hive file
after truncating trailing garbage)
7680 -rw------- 1 root root 7864320 Oct 1 21:21 badsysback (original
SYSTEM hive file)
16 -rw-r--r-- 1 root root 16384 Oct 1 21:55 badsysgarb (16K of truncated
garbage)


Looking into hive file, the last 16K is what hivex complains to be
trailing garbage. All of that 16K is zeroes.
 

[root@ip-10-66-209-246 ~]# dd if=./badsysback of=./badsysgarb bs=4k
skip=1916
4+0 records in
4+0 records out
16384 bytes (16 kB) copied, 9.1288e-05 s, 179 MB/s


[root@ip-10-66-209-246 ~]# hexdump badsysgarb
0000000 0000 0000 0000 0000 0000 0000 0000 0000
*
0004000

I'm not sure what the filesystem cluster size is but could the SYSTEM hive
file be padded with 16K of zeroes to match 32k cluster allocation (or
something like that?)

I imported both hive files (original one that hivex complained about and
the truncated one) into a windows regedit and the tool opened them both
fine and showed pretty much the same content

I'm going to see if the hive file I have has any sensitive customer
information and will try to share it if it does not

Thanks for you help!
~ Hari


On 9/25/13 4:59 AM, "Richard W.M. Jones" <rjones@redhat.com>
wrote:
>On Tue, Sep 24, 2013 at 09:31:26PM +0000, Subramanian, Hari wrote:
>> Is there a reason why a previous version of hivex ignored this
condition
>> while more recent ones throw an error?
>
>We don't understand all aspects of the hive format, so it's better
to
>warn about these things than ignore them.
>
>> I do have some hive files that are failing (I'm yet to validate if
they
>> are good or were corrupted some other way). I will do so and share my
>> findings
>
>When you say "that are failing" do you mean they actually fail
(and if
>so how -- full verbose logs please) or they print this warning, which
>is not a failure?
>
>Rich.
>
>-- 
>Richard Jones, Virtualization Group, Red Hat
>http://people.redhat.com/~rjones
>virt-top is 'top' for virtual machines.  Tiny program with many
>powerful monitoring features, net stats, disk stats, logging, etc.
>http://people.redhat.com/~rjones/virt-top

(Resending since my message from 2 days ago didn't make it to the list.
Perhaps because of the log file attachment)

Hi Rich,

Sorry about the delayed response. I understand that since hive formats are
not open, the current tack of warning about the hive file is a better one

To respond to you question about "whether it fails", hivexsh is unable
to
open the hive file and it prints this message and exits. I've attached the
verbose logs as requested

This extract from the logs shows that the hivexsh complains content after
file offset 0x77c000 is garbage

hivex: badsys: trailing garbage at end of file (at 0x77c000, after 1849
pages)

So, I went ahead and truncated the contents of the file after that file
offset and hivex was able to successfully open the new hive file

The sizes of the different hive files shown here

[root@ip-10-66-209-246 ~]# ls -als bads*
7664 -rw--w--w- 1 root root 7847936 Oct 1 21:51 badsys (SYSTEM hive file
after truncating trailing garbage)
7680 -rw------- 1 root root 7864320 Oct 1 21:21 badsysback (original
SYSTEM hive file)
16 -rw-r--r-- 1 root root 16384 Oct 1 21:55 badsysgarb (16K of truncated
garbage)


Looking into hive file, the last 16K is what hivex complains to be
trailing garbage. All of that 16K is zeroes.

[root@ip-10-66-209-246 ~]# dd if=./badsysback of=./badsysgarb bs=4k
skip=1916
4+0 records in
4+0 records out
16384 bytes (16 kB) copied, 9.1288e-05 s, 179 MB/s


[root@ip-10-66-209-246 ~]# hexdump badsysgarb
0000000 0000 0000 0000 0000 0000 0000 0000 0000
*
0004000

I'm not sure what the filesystem cluster size is but could the SYSTEM hive
file be padded with 16K of zeroes to match 32k cluster allocation (or
something like that?)

I imported both hive files (original one that hivex complained about and
the truncated one) into a windows regedit and the tool opened them both
fine and showed pretty much the same content

I'm going to see if the hive file I have has any sensitive customer
information and will try to share it if it does not

Thanks for you help!
~ Hari



On 9/25/13 4:59 AM, "Richard W.M. Jones" <rjones@redhat.com>
wrote:
>On Tue, Sep 24, 2013 at 09:31:26PM +0000, Subramanian, Hari wrote:
>> Is there a reason why a previous version of hivex ignored this
condition
>> while more recent ones throw an error?
>
>We don't understand all aspects of the hive format, so it's better
to
>warn about these things than ignore them.
>
>> I do have some hive files that are failing (I'm yet to validate if
they
>> are good or were corrupted some other way). I will do so and share my
>> findings
>
>When you say "that are failing" do you mean they actually fail
(and if
>so how -- full verbose logs please) or they print this warning, which
>is not a failure?
>
>Rich.
>
>-- 
>Richard Jones, Virtualization Group, Red Hat
>http://people.redhat.com/~rjones
>virt-top is 'top' for virtual machines.  Tiny program with many
>powerful monitoring features, net stats, disk stats, logging, etc.
>http://people.redhat.com/~rjones/virt-top