Xen users - Aug 2009 - Bootable LiveDVD w/ Xen that boots Windows Image?

Hey all,

Need to know if ths sounds like a crazy idea or not.  Not really messed with Xen
a whole lot, but I have played around with other virtualization technologies, so
I''m not 100% if I can do what I''m after.

Basic gist is, I want to create a small Xen dom0 install, primarily using Debian
Lenny 5.0.2, and from it, boot a Windows XP domU instance, but do all of this
from a single bootable DVD disc. I think I''ve got the basics down
behind setting up a bootable Linux DVD, as I''ve built netboots before
and found some some scripts which might help.

The tricky part is going to be the Windows install.  I figure I can install it
on a read-write virtual disk, and later package that image up for booting off of
the DVD.  The only curiosity is going to be how well Windows takes it, because
it scibbles all over the place (Registry mostly) when it boots. 
There''s a specific application I need to be available when this whole
mess boots up.  I won''t use said app very often (1-2 times/month), but
I don''t want to keep an entire Windows install around for it. 
I''m thinking it''d be easier to somehow get this onto a
bootable medium, and since Windows wasn''t designed to boot off of CD
and DVD media, I figured that this would be the next best idea.

Or if people have better ideas, I am all ears...

Thanks!,

Joshua Kinard


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Hi Joshua!
 The new Xen Live CD v2.0 does this, but with a Ubuntu domU instead of
Windows domU...

 Take a look at the Xen Live CD wiki page, so you can learn how to buid your
own Xen Live CD with your Windows Guest installed...

 Xen Live CD: http://wiki.xensource.com/xenwiki/LiveCD

 The domU guests has a read-write root file system powered by SquashFS, so
you must use something like qcow2 file image, I don''t know... I hope
that
the Xen Live CD can help you!

Cheers!
Thiago

2009/8/7 Joshua Kinard <joshua.kinard@sdc-world.com>
>  Hey all,
>
> Need to know if ths sounds like a crazy idea or not.  Not really messed
> with Xen a whole lot, but I have played around with other virtualization
> technologies, so I''m not 100% if I can do what I''m after.
>
> Basic gist is, I want to create a small Xen dom0 install, primarily using
> Debian Lenny 5.0.2, and from it, boot a Windows XP domU instance, but do
all
> of this from a single bootable DVD disc. I think I''ve got the
basics down
> behind setting up a bootable Linux DVD, as I''ve built netboots
before and
> found some some scripts which might help.
>
> The tricky part is going to be the Windows install.  I figure I can install
> it on a read-write virtual disk, and later package that image up for
booting
> off of the DVD.  The only curiosity is going to be how well Windows takes
> it, because it scibbles all over the place (Registry mostly) when it boots.
> There''s a specific application I need to be available when this
whole mess
> boots up.  I won''t use said app very often (1-2 times/month), but
I don''t
> want to keep an entire Windows install around for it.  I''m
thinking it''d be
> easier to somehow get this onto a bootable medium, and since Windows
wasn''t
> designed to boot off of CD and DVD media, I figured that this would be the
> next best idea.
>
> Or if people have better ideas, I am all ears...
>
> Thanks!,
>
> Joshua Kinard
>
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xensource.com
> http://lists.xensource.com/xen-users
>

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On Fri, Aug 7, 2009 at 11:05 PM, Joshua
Kinard<joshua.kinard@sdc-world.com> wrote:
> The tricky part is going to be the Windows install.  I figure I can install
> it on a read-write virtual disk, and later package that image up for
booting
> off of the DVD.  The only curiosity is going to be how well Windows takes
> it, because it scibbles all over the place (Registry mostly) when it boots.
It''d be easier to use tap:qcow (or similar) so that the main image can
be read only, while the changing part is located in disk/RAM. Problem
is last time I check tap:qcow has some problems (like the fact that it
doesn''t work with PV domU or Windows+GPLPV)
> There''s a specific application I need to be available when this
whole mess
> boots up.  I won''t use said app very often (1-2 times/month), but
I don''t
> want to keep an entire Windows install around for it.  I''m
thinking it''d be
> easier to somehow get this onto a bootable medium, and since Windows
wasn''t
> designed to boot off of CD and DVD media, I figured that this would be the
> next best idea.
>
> Or if people have better ideas, I am all ears...
Why not simply :
- buy a large enough external USB HDD (70G will do)
- install Ubuntu (or your favorite distro) on it
- install Xen (method varies, depending on distro)
- install Windows HVM domU

So instead of a bootable DVD you have a bootable USB HDD. This way you
can use domU as you usually do without having to worry about tap:qcow
woes.

If you want it to be even more generic, use Virtualbox instead of Xen.
That way you can have Windows guest even on non-VT/SVM hardware
(netbooks come to mind).

-- 
Fajar

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Hey Thiago,

I actually had come across that and was looking at it.  I think it might do most
or part of what I''m looking to setup (the Linux LiveCD part), as
I''m using Debian Leny for the Linux side.

I''ll have to check out this qcow2 file image setup some more.  Fajar
mentioned in a mail after this one that feature may still have problems.  Are
there any documents someplace detailing the qcow2 setup?

Speed isn''t really going to be a factor for me, even though
it''ll likely be dog-slow anyways (CD-ROM read speed + squashfs
decompression + Windows'' sasquatch-sized footprint + specialized
application we need to run).  But if it works, that''ll be awesome.  At
least I''ll get a coffee break while the whole mess would boot up....

Thanks!,

--J

________________________________
From: Thiago Camargo Martins Cordeiro [thiagocmartinsc@gmail.com]
Sent: Saturday, August 08, 2009 2:31 AM
To: Joshua Kinard
Cc: xen-users@lists.xensource.com
Subject: Re: [Xen-users] Bootable LiveDVD w/ Xen that boots Windows Image?

Hi Joshua!

 The new Xen Live CD v2.0 does this, but with a Ubuntu domU instead of Windows
domU...

 Take a look at the Xen Live CD wiki page, so you can learn how to buid your own
Xen Live CD with your Windows Guest installed...

 Xen Live CD: http://wiki.xensource.com/xenwiki/LiveCD

 The domU guests has a read-write root file system powered by SquashFS, so you
must use something like qcow2 file image, I don''t know... I hope that
the Xen Live CD can help you!

Cheers!
Thiago

2009/8/7 Joshua Kinard
<joshua.kinard@sdc-world.com<mailto:joshua.kinard@sdc-world.com>>
Hey all,

Need to know if ths sounds like a crazy idea or not.  Not really messed with Xen
a whole lot, but I have played around with other virtualization technologies, so
I''m not 100% if I can do what I''m after.

Basic gist is, I want to create a small Xen dom0 install, primarily using Debian
Lenny 5.0.2, and from it, boot a Windows XP domU instance, but do all of this
from a single bootable DVD disc. I think I''ve got the basics down
behind setting up a bootable Linux DVD, as I''ve built netboots before
and found some some scripts which might help.

The tricky part is going to be the Windows install.  I figure I can install it
on a read-write virtual disk, and later package that image up for booting off of
the DVD.  The only curiosity is going to be how well Windows takes it, because
it scibbles all over the place (Registry mostly) when it boots. 
There''s a specific application I need to be available when this whole
mess boots up.  I won''t use said app very often (1-2 times/month), but
I don''t want to keep an entire Windows install around for it. 
I''m thinking it''d be easier to somehow get this onto a
bootable medium, and since Windows wasn''t designed to boot off of CD
and DVD media, I figured that this would be the next best idea.

Or if people have better ideas, I am all ears...

Thanks!,

Joshua Kinard

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com<mailto:Xen-users@lists.xensource.com>
http://lists.xensource.com/xen-users



_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Hi Fajar,

Is tap:qcow2 something that has to be done when installing the Windows domU, or
is that configured when building the LiveCD?  I''ll google around for
some information, but if you''ve run into any pitfalls, quirks, or
specific configs that work, I''d be curious to read about them.

I can''t use the USB option, as our specialized application is mostly a
system checking/validation tool, and so has to be pretty static to make sure
that as we check various systems, nothing changes on the scanning system.  The
systems it does scan are all on stand-alone networks, and we want to avoid the
tool from uneccesarily modifying them (I know it won''t, but
I''m the techie -- the ones requiring this don''t understand it
at that level).

We have other methods available if this idea doesn''t pan
out....it''ll just mean additional hardware and Windows installs to
maintain.  But if this works out, it''ll save us some headaches. 
VirtualBox is also out because of PUEL licensing getting in the way.  Ditto for
VMWare and pretty much any other commercialized VM solution.  Free is better
(even if it requires some elbow grease to put together).

Thanks!,

--J

________________________________________
From: xen-users-bounces@lists.xensource.com
[xen-users-bounces@lists.xensource.com] On Behalf Of Fajar A. Nugraha
[fajar@fajar.net]
Sent: Sunday, August 09, 2009 11:19 PM
To: Joshua Kinard
Cc: xen-users@lists.xensource.com
Subject: Re: [Xen-users] Bootable LiveDVD w/ Xen that boots Windows Image?

On Fri, Aug 7, 2009 at 11:05 PM, Joshua
Kinard<joshua.kinard@sdc-world.com> wrote:
> The tricky part is going to be the Windows install.  I figure I can install
> it on a read-write virtual disk, and later package that image up for
booting
> off of the DVD.  The only curiosity is going to be how well Windows takes
> it, because it scibbles all over the place (Registry mostly) when it boots.
It''d be easier to use tap:qcow (or similar) so that the main image can
be read only, while the changing part is located in disk/RAM. Problem
is last time I check tap:qcow has some problems (like the fact that it
doesn''t work with PV domU or Windows+GPLPV)
> There''s a specific application I need to be available when this
whole mess
> boots up.  I won''t use said app very often (1-2 times/month), but
I don''t
> want to keep an entire Windows install around for it.  I''m
thinking it''d be
> easier to somehow get this onto a bootable medium, and since Windows
wasn''t
> designed to boot off of CD and DVD media, I figured that this would be the
> next best idea.
>
> Or if people have better ideas, I am all ears...
Why not simply :
- buy a large enough external USB HDD (70G will do)
- install Ubuntu (or your favorite distro) on it
- install Xen (method varies, depending on distro)
- install Windows HVM domU

So instead of a bootable DVD you have a bootable USB HDD. This way you
can use domU as you usually do without having to worry about tap:qcow
woes.

If you want it to be even more generic, use Virtualbox instead of Xen.
That way you can have Windows guest even on non-VT/SVM hardware
(netbooks come to mind).

--
Fajar

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Hi Joshua,
 Take a look at this web pages, maybe that can help you:

http://wiki.xensource.com/xenwiki/COWHowTo
http://lxr.xensource.com/lxr/source/tools/blktap

 I''m sorry but I never have used the qcow with Xen... If you need any
help
to get you Windows domU running within the Xen Live CD, will be a pleasure
try to help you.

Cheers!
Thiago

2009/8/11 Joshua Kinard <joshua.kinard@sdc-world.com>
>  Hey Thiago,
>
> I actually had come across that and was looking at it.  I think it might do
> most or part of what I''m looking to setup (the Linux LiveCD part),
as I''m
> using Debian Leny for the Linux side.
>
> I''ll have to check out this qcow2 file image setup some more. 
Fajar
> mentioned in a mail after this one that feature may still have problems.
> Are there any documents someplace detailing the qcow2 setup?
>
> Speed isn''t really going to be a factor for me, even though
it''ll likely be
> dog-slow anyways (CD-ROM read speed + squashfs decompression +
Windows''
> sasquatch-sized footprint + specialized application we need to run).  But
if
> it works, that''ll be awesome.  At least I''ll get a coffee
break while the
> whole mess would boot up....
>
> Thanks!,
>
> --J
>
>  ------------------------------
> *From:* Thiago Camargo Martins Cordeiro [thiagocmartinsc@gmail.com]
> *Sent:* Saturday, August 08, 2009 2:31 AM
> *To:* Joshua Kinard
> *Cc:* xen-users@lists.xensource.com
> *Subject:* Re: [Xen-users] Bootable LiveDVD w/ Xen that boots Windows
> Image?
>
>  Hi Joshua!
>   The new Xen Live CD v2.0 does this, but with a Ubuntu domU instead of
> Windows domU...
>
>   Take a look at the Xen Live CD wiki page, so you can learn how to buid
> your own Xen Live CD with your Windows Guest installed...
>
>   Xen Live CD: http://wiki.xensource.com/xenwiki/LiveCD
>
>   The domU guests has a read-write root file system powered by SquashFS,
> so you must use something like qcow2 file image, I don''t know... I
hope that
> the Xen Live CD can help you!
>
>  Cheers!
> Thiago
>
> 2009/8/7 Joshua Kinard <joshua.kinard@sdc-world.com>
>
>>  Hey all,
>>
>> Need to know if ths sounds like a crazy idea or not.  Not really messed
>> with Xen a whole lot, but I have played around with other
virtualization
>> technologies, so I''m not 100% if I can do what I''m
after.
>>
>> Basic gist is, I want to create a small Xen dom0 install, primarily
using
>> Debian Lenny 5.0.2, and from it, boot a Windows XP domU instance, but
do all
>> of this from a single bootable DVD disc. I think I''ve got the
basics down
>> behind setting up a bootable Linux DVD, as I''ve built netboots
before and
>> found some some scripts which might help.
>>
>> The tricky part is going to be the Windows install.  I figure I can
>> install it on a read-write virtual disk, and later package that image
up for
>> booting off of the DVD.  The only curiosity is going to be how well
Windows
>> takes it, because it scibbles all over the place (Registry mostly) when
it
>> boots.  There''s a specific application I need to be available
when this
>> whole mess boots up.  I won''t use said app very often (1-2
times/month), but
>> I don''t want to keep an entire Windows install around for it. 
I''m thinking
>> it''d be easier to somehow get this onto a bootable medium, and
since Windows
>> wasn''t designed to boot off of CD and DVD media, I figured
that this would
>> be the next best idea.
>>
>> Or if people have better ideas, I am all ears...
>>
>> Thanks!,
>>
>>  Joshua Kinard
>>
>> _______________________________________________
>> Xen-users mailing list
>> Xen-users@lists.xensource.com
>> http://lists.xensource.com/xen-users
>>
>
>
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xensource.com
> http://lists.xensource.com/xen-users
>

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On Tue, Aug 11, 2009 at 9:24 PM, Joshua
Kinard<joshua.kinard@sdc-world.com> wrote:
> Hi Fajar,
>
> Is tap:qcow2 something that has to be done when installing the Windows
domU, or is that configured when building the LiveCD?
More of Xen config than Windows. From Windows perspective, its normal
install as usual. From Xen perspective, first you create a base image,
install everything you want on it. Then you crate a new qcow from that
base image, and then change the domU config to use it. If everything
works, you should be able to automate the process (using custom
scripts) so that (for example) the qcow image is located in RAM, and
automatically created on every boot. That way on every boot you''d have
the same state of Windows as that on base image.
> I''ll google around for some information, but if you''ve
run into any pitfalls, quirks, or specific configs that work, I''d be
curious to read about them.
Last time I check I ran into a problem similar to this
http://bugzilla.xensource.com/bugzilla/show_bug.cgi?id=1430

It''s probably fixed now, but to tell the truth I lost interest in qcow.
For production purposes zfs'' zvol can provide much better throughput,
simpler to set up and manage, and can save even more space compared to
qcow. Of course you''ll need (open)solaris (for dom0 or iscsi server)
to use zfs :D
>
> I can''t use the USB option, as our specialized application is
mostly a system checking/validation tool, and so has to be pretty static to make
sure that as we check various systems, nothing changes on the scanning system.
 The systems it does scan are all on stand-alone networks, and we want to avoid
the tool from uneccesarily modifying them (I know it won''t, but
I''m the techie -- the ones requiring this don''t understand it
at that level).
Actually for your purpose I''d say using USB + virtualbox is a perfect
choice. The base image can be created as a snapshot, then after every
scan you can revert to that snapshot. Function-wise, this is the same
as using qcow with the qcow image located in RAM.
>
> We have other methods available if this idea doesn''t pan
out....it''ll just mean additional hardware and Windows installs to
maintain.  But if this works out, it''ll save us some headaches.
 VirtualBox is also out because of PUEL licensing getting in the way.  Ditto for
VMWare and pretty much any other commercialized VM solution.  Free is better
(even if it requires some elbow grease to put together).
Virtualbox has a GPL version too :D You might have to build it
manually. I think opensuse has virtualbox-ose prebuilt binary.

Regards,

Fajar

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Hi Fajar,

Yup, read more about the qcow stuff after digging around on Google some.  Even
came across some of your messages on a thread or two about it.  I''ll
probably give this a shot and see what happens, though I think I''m
going to try combining this with the UnionFS (aufs specifically) stuff. 
I''ve read that might work better, since I''ll be able to keep
the squashfs''ed WinXP qcow2 image on the DVD itself and not have to
offload it into RAM, but somehow create a r/w qcow2 image in tmpfs, probably
ntfs format it, and allow aufs to overlay that somehow.  Although, I''m
not sure in what order to do that just yet.  Once I figure it out, scripting
will be the easy part.

I also heard about the OpenSolaris approach, but I admit that I''ve only
tinkered with OpenSolaris a little bit.  I have an experiment at home unrelated
to this project which will teach me a lot, though...  Also not sure how much
different the processes are for generating a Xen LiveCD from an OpenSolaris
install would be using the Xen scripts, let alone configuring the ZFS bits and
plugging it all together.  I grok Linux a lot easier for now.

As far as VirtualBox goes, I''ve played around with the OSE edition
(that''s all Debian distributes in Lenny), and the problems with it are
that A) It''s a bit aged (~1.6-something in Lenny), and B) I believe I
need the Guest Additions in order for WinXP to properly utilize the network
driver presented to it by the Vbox virtual layer, and that''s not free. 
I also lack a Windows development environment where I could compile that on my
own.  I also don''t know of a method to tackle the split need of storing
the core OS image on a read-only medium and a writable overlay into tmpfs using
VBox disk images...not sure if it''d appreciate that or not.

Hence, Xen.....which so far looks like it has all the tools I need.  I just have
to figure out how to jam that square peg into the round hole :)

Thanks for the tips!  (PS, I''m using Outlook OWA unfortunately, hence,
no quotation blocks, per my usual method on mailing lists).

--J

________________________________________
From: Fajar A. Nugraha [fajar@fajar.net]
Sent: Tuesday, August 11, 2009 10:06 PM
To: Joshua Kinard
Cc: xen-users@lists.xensource.com
Subject: Re: [Xen-users] Bootable LiveDVD w/ Xen that boots Windows Image?

On Tue, Aug 11, 2009 at 9:24 PM, Joshua
Kinard<joshua.kinard@sdc-world.com> wrote:
> Hi Fajar,
>
> Is tap:qcow2 something that has to be done when installing the Windows
domU, or is that configured when building the LiveCD?
More of Xen config than Windows. From Windows perspective, its normal
install as usual. From Xen perspective, first you create a base image,
install everything you want on it. Then you crate a new qcow from that
base image, and then change the domU config to use it. If everything
works, you should be able to automate the process (using custom
scripts) so that (for example) the qcow image is located in RAM, and
automatically created on every boot. That way on every boot you''d have
the same state of Windows as that on base image.
> I''ll google around for some information, but if you''ve
run into any pitfalls, quirks, or specific configs that work, I''d be
curious to read about them.
Last time I check I ran into a problem similar to this
http://bugzilla.xensource.com/bugzilla/show_bug.cgi?id=1430

It''s probably fixed now, but to tell the truth I lost interest in qcow.
For production purposes zfs'' zvol can provide much better throughput,
simpler to set up and manage, and can save even more space compared to
qcow. Of course you''ll need (open)solaris (for dom0 or iscsi server)
to use zfs :D
>
> I can''t use the USB option, as our specialized application is
mostly a system checking/validation tool, and so has to be pretty static to make
sure that as we check various systems, nothing changes on the scanning system. 
The systems it does scan are all on stand-alone networks, and we want to avoid
the tool from uneccesarily modifying them (I know it won''t, but
I''m the techie -- the ones requiring this don''t understand it
at that level).
Actually for your purpose I''d say using USB + virtualbox is a perfect
choice. The base image can be created as a snapshot, then after every
scan you can revert to that snapshot. Function-wise, this is the same
as using qcow with the qcow image located in RAM.
>
> We have other methods available if this idea doesn''t pan
out....it''ll just mean additional hardware and Windows installs to
maintain.  But if this works out, it''ll save us some headaches. 
VirtualBox is also out because of PUEL licensing getting in the way.  Ditto for
VMWare and pretty much any other commercialized VM solution.  Free is better
(even if it requires some elbow grease to put together).
Virtualbox has a GPL version too :D You might have to build it
manually. I think opensuse has virtualbox-ose prebuilt binary.

Regards,

Fajar
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On Wed, Aug 12, 2009 at 8:41 PM, Joshua
Kinard<joshua.kinard@sdc-world.com> wrote:
> Hi Fajar,
>
> Yup, read more about the qcow stuff after digging around on Google some.
 Even came across some of your messages on a thread or two about it.
 I''ll probably give this a shot and see what happens, though I think
I''m going to try combining this with the UnionFS (aufs specifically)
stuff.  I''ve read that might work better, since I''ll be able
to keep the squashfs''ed WinXP qcow2 image on the DVD itself and not
have to offload it into RAM, but somehow create a r/w qcow2 image in tmpfs,
probably ntfs format it, and allow aufs to overlay that somehow.  Although,
I''m not sure in what order to do that just yet.  Once I figure it out,
scripting will be the easy part.
Am I right in assuming you want to use aufs on Windows? AFAIK it
doesn''t run on Windows.

The other approach with aufs probaly won''t work either. You CAN use
aufs to merge the squashfs content with ramdisk to create a "writable"
root fs for the dom0 (not for Windows), but the way aufs works when
there''s a change in a file (e.g. the WInXP image file) it will copy
the WHOLE file to r/w location, and change it there. That would be
unusable as the image would be several GB in size.

So my best suggestion is you probably can use aufs for the live CD
part, but you can''t use it for the domU image. You''d have to
split it
into a base image which can be read only on DVD, and a qcow
incremental image (is this the right term?) which can be on RAM.
> As far as VirtualBox goes, I''ve played around with the OSE edition
(that''s all Debian distributes in Lenny), and the problems with it are
that A) It''s a bit aged (~1.6-something in Lenny), and B) I believe I
need the Guest Additions in order for WinXP to properly utilize the network
driver presented to it by the Vbox virtual layer, and that''s not free.
Yeah, I forgot about the guest addition :)

BTW, what kind of WIndows application are you going to use? Some
applications (like Firefox, vnc, etc.) CAN be made to run on windows
"live" CD using BARTPE. If it works you''d save the overhead
of dom0
and Xen. See http://www.nu2.nu/pebuilder/

-- 
Fajar

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Hmm, I didn''t think of it that way.

The way I read up on UnionFS and aufs'' functionality, was they could
essentially merge two files virtually, i.e., the kernel module would be able to
look at the operation coming in and route it to the proper descriptor (i.e.,
read() --> /livecd/windows/winxp.img OR write() --> /tmp/winxp.img, with
/tmp being in tmpfs).  I guess it''s not as granular as that it seems. 
Would be a neat trick, but I imagine it''d be complex as anything for a
kernel module to have to keep track of which files have variants loaded in the
writeable union area.

I have yet to try and do a squashfs on the filesystem...Right now, I have the
Windows domU sitting right at 3.5GB, and I''ve used the sysinternals
''sdelete'' tool to fill all remaining free space with zero
data, so that should allow squashfs to compress it down really far.  But I know
that squashfs is read-only, too, and inflated, only a loaded x64 server would be
able to hold a 3.5GB image in memory.  Kinda scratches that idea off of the
table.  Maybe I''ll have to go the OpenSolaris way after all?


As for the application, it''s a complex network security scanner, made
by eEye Digital Security, called "Retina".  We just don''t
want to setup and baby sit Windows installations on our Unix networks strictly
for this one app, so I figured if I can get it to run off of a CD, we can just
park some diskless hardware in a closet and pull it out whenever we need to do
network testing and such.  I''ve already tried BartPE and ReatogoxPE,
and while the latter lets me generate a plugin from the installed application,
that version doesn''t work because of the age-old nemesis, the Windows
Registry.  I''m not too keen on trying to track down every little key
this program will modify just to record it into the BartPE stuff, as it all has
to be encoded into INI files for the plugins that pebuilder uses.

So I figured trying to virtualize the whole mess would be easier.  Tried Wine
first, but the program''s installer makes some advanced use of MSI
functions, and one particular call, WixSchedFirewallUpdate or something,
isn''t implemented in Wine, so it fails (Wix appears to be an OSS
project, too, but their documentation didn''t help any).  Next, I looked
at VMWare Server (free), but the license gets in the way, then VirtualBox, and
that license is in the way as well.  Checked Wikipedia''s list of other
Virtualization solutions, and none of them seem "free enough", save
Xen and KVM, and Xen sounded like the better one to tackle.

That of course leads into the problem of Window''s habit of scribbling
everywhere when it boots.  No idea how BartPE works around that...It does appear
they offload a bit of stuff into ram for read/write, so it''s possible
that they offload the registry hives somehow, but I haven''t dug deep
enough to find out how, and implementing that inside of a VM might be difficult,
as I imagine I''d have to create images in tmpfs from Linux, then get
Xen to share them to the Windows image as another physical drive for it to use,
and then direct Windows to write all changes to that drive instead of C:.

If I get this entire concept to work, I''ll probably go on to prove that
black is white....  :)

Thanks!,

--J

________________________________________
From: Fajar A. Nugraha [fajar@fajar.net]
Sent: Thursday, August 13, 2009 1:49 AM
To: Joshua Kinard
Cc: xen-users@lists.xensource.com
Subject: Re: [Xen-users] Bootable LiveDVD w/ Xen that boots Windows Image?

On Wed, Aug 12, 2009 at 8:41 PM, Joshua
Kinard<joshua.kinard@sdc-world.com> wrote:
> Hi Fajar,
>
> Yup, read more about the qcow stuff after digging around on Google some. 
Even came across some of your messages on a thread or two about it. 
I''ll probably give this a shot and see what happens, though I think
I''m going to try combining this with the UnionFS (aufs specifically)
stuff.  I''ve read that might work better, since I''ll be able
to keep the squashfs''ed WinXP qcow2 image on the DVD itself and not
have to offload it into RAM, but somehow create a r/w qcow2 image in tmpfs,
probably ntfs format it, and allow aufs to overlay that somehow.  Although,
I''m not sure in what order to do that just yet.  Once I figure it out,
scripting will be the easy part.
Am I right in assuming you want to use aufs on Windows? AFAIK it
doesn''t run on Windows.

The other approach with aufs probaly won''t work either. You CAN use
aufs to merge the squashfs content with ramdisk to create a "writable"
root fs for the dom0 (not for Windows), but the way aufs works when
there''s a change in a file (e.g. the WInXP image file) it will copy
the WHOLE file to r/w location, and change it there. That would be
unusable as the image would be several GB in size.

So my best suggestion is you probably can use aufs for the live CD
part, but you can''t use it for the domU image. You''d have to
split it
into a base image which can be read only on DVD, and a qcow
incremental image (is this the right term?) which can be on RAM.
> As far as VirtualBox goes, I''ve played around with the OSE edition
(that''s all Debian distributes in Lenny), and the problems with it are
that A) It''s a bit aged (~1.6-something in Lenny), and B) I believe I
need the Guest Additions in order for WinXP to properly utilize the network
driver presented to it by the Vbox virtual layer, and that''s not free.
Yeah, I forgot about the guest addition :)

BTW, what kind of WIndows application are you going to use? Some
applications (like Firefox, vnc, etc.) CAN be made to run on windows
"live" CD using BARTPE. If it works you''d save the overhead
of dom0
and Xen. See http://www.nu2.nu/pebuilder/

--
Fajar
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Joshua Kinard wrote:
> Hey all,
>  
> Need to know if ths sounds like a crazy idea or not.  Not really messed 
> with Xen a whole lot, but I have played around with other virtualization 
> technologies, so I''m not 100% if I can do what I''m after.
>  
> Basic gist is, I want to create a small Xen dom0 install, primarily 
> using Debian Lenny 5.0.2, and from it, boot a Windows XP domU instance, 
> but do all of this from a single bootable DVD disc. I think I''ve
got the
> basics down behind setting up a bootable Linux DVD, as I''ve built 
> netboots before and found some some scripts which might help.
>  
> The tricky part is going to be the Windows install.  I figure I can 
> install it on a read-write virtual disk, and later package that image up 
> for booting off of the DVD.  The only curiosity is going to be how well 
> Windows takes it, because it scibbles all over the place (Registry 
> mostly) when it boots.  There''s a specific application I need to
be
> available when this whole mess boots up.  I won''t use said app
very
> often (1-2 times/month), but I don''t want to keep an entire
Windows
> install around for it.  I''m thinking it''d be easier to
somehow get this
> onto a bootable medium, and since Windows wasn''t designed to boot
off of
> CD and DVD media, I figured that this would be the next best idea.
>  
> Or if people have better ideas, I am all ears...

This should be possible. As posted earlier there already live CDs 
booting Xen and a dom0. For Windows you could use a PE image then and 
boot it from an iso image within dom0''s filesystem. Not sure how much 
space a recent Windows PE image takes but I still have some BartPE 
CDs(!) here, so it shouldn''t be too hard to get this all onto a single 
DVD image.


That being said I would still recommend a USB drive as DVD/CD are slow 
and booting an iso from within an (compreseed) squashfs won''t make 
things faster. Also think about security updates, you don''t want to
boot
with an unpatched Windows thats a couple of month old...


Best regards,
    Christian


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On Fri, Aug 14, 2009 at 1:37 AM, Joshua
Kinard<joshua.kinard@sdc-world.com> wrote:
> Hmm, I didn''t think of it that way.
>
> The way I read up on UnionFS and aufs'' functionality, was they
could essentially merge two files virtually,
Merge two directories together, to be exact. Not merge files.
> i.e., the kernel module would be able to look at the operation coming in
and route it to the proper descriptor (i.e., read() -->
/livecd/windows/winxp.img OR write() --> /tmp/winxp.img, with /tmp being in
tmpfs).  I guess it''s not as granular as that it seems.  Would be a
neat trick, but I imagine it''d be complex as anything for a kernel
module to have to keep track of which files have variants loaded in the
writeable union area.
What you describe is essentially block-level copy-on-write, which is
what qcow or zfs does. Aufs/unionfs does this per file.
> As for the application, it''s a complex network security scanner,
made by eEye Digital Security, called "Retina".  We just
don''t want to setup and baby sit Windows installations on our Unix
networks strictly for this one app, so I figured if I can get it to run off of a
CD, we can just park some diskless hardware in a closet and pull it out whenever
we need to do network testing and such.
If it were me I''d simply setup a Windows domU on any server with
enough disk and RAM, make a "template" of the "good"
installation
(preferably zfs or LVM snapshot), start it only when it''s needed, shut
it down afterwards, and (if necessary) rollback to the "good"
template. That is assuming that all host/network tested reachable from
my datacenter (either with vlans or routing). No need to add the
complexity of a live DVD/USB.

If portability is a requirement, and you''re absolutely sure
you''ll
always have VT-capable host handy, then using live USB is much
perefered than DVD due to performance and complexity reasons. But hey,
that''s just me :D

Let us know if you find a solution that works.

-- 
Fajar

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Hmm, block-level copy-on-write?  If qcow supports this out-of-the-box, then I
technically don''t need any kind of UnionFS?  Just someway to specify to
Xen a read-only image with a qcow2 image in a writable zone?

Going to dredge through some EU software patent I dug up from Microsoft. 
Supposed to document the /MININT switch to the NT kernel that supposedly makes
Windows write all registry changes to volatile memory instead of to the registry
hives.  It''s mostly used in the WinPE environment, but it looks like
BartPE leverages it too.  Seems I need more to it, though, than just modifying
boot.ini to pass it, as once I started to boot Windows from a read-only drive
mount (after discovering Xen/qemu don''t properly honor the mode bit in
the Xen config for r or r/w, and even file-level permissions), Windows crashed
with UNMOUNTABLE_BOOT_VOLUME as its error.  But it highlights the capability is
there....just not easy.

USB is out, unfortunately, too.  For security reasons, we ban USB/Firewire
drives here, with the exception of CD Burners.  Ditto on memory cards (SD, MMC,
etc..).  I''ve got very narrow operating parameters to work in, which is
why this is proving to be quite a fun challenge.

Thanks!,

--J

________________________________________
From: Fajar A. Nugraha [fajar@fajar.net]
Sent: Friday, August 14, 2009 4:53 AM
To: Joshua Kinard
Cc: xen-users@lists.xensource.com
Subject: Re: [Xen-users] Bootable LiveDVD w/ Xen that boots Windows Image?

On Fri, Aug 14, 2009 at 1:37 AM, Joshua
Kinard<joshua.kinard@sdc-world.com> wrote:
> Hmm, I didn''t think of it that way.
>
> The way I read up on UnionFS and aufs'' functionality, was they
could essentially merge two files virtually,
Merge two directories together, to be exact. Not merge files.
> i.e., the kernel module would be able to look at the operation coming in
and route it to the proper descriptor (i.e., read() -->
/livecd/windows/winxp.img OR write() --> /tmp/winxp.img, with /tmp being in
tmpfs).  I guess it''s not as granular as that it seems.  Would be a
neat trick, but I imagine it''d be complex as anything for a kernel
module to have to keep track of which files have variants loaded in the
writeable union area.
What you describe is essentially block-level copy-on-write, which is
what qcow or zfs does. Aufs/unionfs does this per file.
> As for the application, it''s a complex network security scanner,
made by eEye Digital Security, called "Retina".  We just
don''t want to setup and baby sit Windows installations on our Unix
networks strictly for this one app, so I figured if I can get it to run off of a
CD, we can just park some diskless hardware in a closet and pull it out whenever
we need to do network testing and such.
If it were me I''d simply setup a Windows domU on any server with
enough disk and RAM, make a "template" of the "good"
installation
(preferably zfs or LVM snapshot), start it only when it''s needed, shut
it down afterwards, and (if necessary) rollback to the "good"
template. That is assuming that all host/network tested reachable from
my datacenter (either with vlans or routing). No need to add the
complexity of a live DVD/USB.

If portability is a requirement, and you''re absolutely sure
you''ll
always have VT-capable host handy, then using live USB is much
perefered than DVD due to performance and complexity reasons. But hey,
that''s just me :D

Let us know if you find a solution that works.

--
Fajar
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Think I cracked it.  Don''t even need to deal with qemu''s qcow
stuff (so far).  Still more tests pending (including using read-only drive
mounts).  But assume the following steps in a dom0:

- mkdir /windows
- cd /windows
- dd if=/dev/zero of=xp.img bs=1024k count=1 seek=7168
- touch xp.cfg
- nano xp.cfg
- <edit xp.cfg, setup as desired>
- xm create xp.cfg
- <install WinXP, patch, delete garbage, defrag, zero-out free space, then
shutdown>
- xm list (verify domU is actually dead)

Now after this, xp.img will contain a baseline WinXP install.  In my specific
case, I compacted everything down to ~2GB total installed (inside the domU that
is) by using straight qemu-emulated devices (not the GPLPV stuff), no page file
or hibernation, stripping nearly every misc utility, and a bunch of other stuff,
but NO NTFS compression (for now, at least).

Next:

- dd if=/dev/zero of=xp_scratch.img bs=1024k count=1 seek=2048
- losetup /dev/loop0 xp.img
- losetup /dev/loop1 xp_scratch.img
- echo 0 $(blockdev --getsize /dev/loop0) snapshot /dev/loop0 /dev/loop1 n 64 |
dmsetup create xpcow

This maps our baseline WinXP image (xp.img) to /dev/loop0 and a scratch area to
/dev/loop1 (I don''t even know if this is needed just yet).  Needed
because dmsetup only works with block devices.  Using dmsetup, a snapshot is
created, so that all writes to loop0 get redirected to loop1, and creates a
device, /dev/mapper/xpcow.

Edit xp.cfg to use phy:/dev/mapper/xpcow,hda,w, and boot Windows:
- xm create xp.cfg

In Windows, create a dummy file on the desktop or three, then shutdown. 
Re-start the domU and back at the desktop, we see that the newly-created files
still exist, so shutdown again.  Now:

- dmsetup remove /dev/mapper/xpcow
- losetup -d /dev/loop1
- dd if=/dev/zero of=xp_scratch2.img bs=1024k count=1 seek=2048
- losetup /dev/loop1 xp_scratch2.img
- echo 0 $(blockdev --getsize /dev/loop0) snapshot /dev/loop0 /dev/loop1 n 64 |
dmsetup create xpcow
- xm create xp.cfg

Back at the desktop, the newly-created files are now gone!

I did it this way as a manual test, because I''m veryifying a few
things:
a. Simulating the effect of creating the scratch image in a /tmpfs mount, while
the baseline image will be on a physical, read-only medium.  By removing the
mapped device and changing out the scratch file, I was seeing whether the XP
image would indeed revert to the baseline state, to simulate a reboot in which
/tmpfs is cleared.  Supposedly, the ''n'' parameter to dmsetup
does this anyways, but I wanted to be doubly sure.

b. Need to scriptify the whole mess, since the baseline image will already be
pre-configured, the generation of the scratch image, setup of the loop devices,
and the device-mapping will be on-the-fly when some future CD Image I put
together boots up.

c. Avoids all the problems I see on Google about using blktap with qcow with
gplpv and such.  I''m probably not grasping some of it, but I think
those references are aiming for different uses.  So far, this setup mostly
relies wholly on the Linux dom0 to piece everything together, leaving the WinXP
domU utterly oblivious to what is going on.  And given Windows'' nature,
that''s exactly what we want.

Performance isn''t an issue for me in this, so I''m sure on a
CD, this will be slower than running Oracle on an 8086 (pretend for a moment
that this is actually possible), but if it works, and WinXP doesn''t
complain, then all is good.

Thoughts?
________________________________________
From: xen-users-bounces@lists.xensource.com
[xen-users-bounces@lists.xensource.com] On Behalf Of Joshua Kinard
[joshua.kinard@sdc-world.com]
Sent: Monday, August 17, 2009 9:50 AM
To: xen-users@lists.xensource.com
Subject: RE: [Xen-users] Bootable LiveDVD w/ Xen that boots Windows Image?

Hmm, block-level copy-on-write?  If qcow supports this out-of-the-box, then I
technically don''t need any kind of UnionFS?  Just someway to specify to
Xen a read-only image with a qcow2 image in a writable zone?

Going to dredge through some EU software patent I dug up from Microsoft. 
Supposed to document the /MININT switch to the NT kernel that supposedly makes
Windows write all registry changes to volatile memory instead of to the registry
hives.  It''s mostly used in the WinPE environment, but it looks like
BartPE leverages it too.  Seems I need more to it, though, than just modifying
boot.ini to pass it, as once I started to boot Windows from a read-only drive
mount (after discovering Xen/qemu don''t properly honor the mode bit in
the Xen config for r or r/w, and even file-level permissions), Windows crashed
with UNMOUNTABLE_BOOT_VOLUME as its error.  But it highlights the capability is
there....just not easy.

USB is out, unfortunately, too.  For security reasons, we ban USB/Firewire
drives here, with the exception of CD Burners.  Ditto on memory cards (SD, MMC,
etc..).  I''ve got very narrow operating parameters to work in, which is
why this is proving to be quite a fun challenge.

Thanks!,

--J

________________________________________
From: Fajar A. Nugraha [fajar@fajar.net]
Sent: Friday, August 14, 2009 4:53 AM
To: Joshua Kinard
Cc: xen-users@lists.xensource.com
Subject: Re: [Xen-users] Bootable LiveDVD w/ Xen that boots Windows Image?

On Fri, Aug 14, 2009 at 1:37 AM, Joshua
Kinard<joshua.kinard@sdc-world.com> wrote:
> Hmm, I didn''t think of it that way.
>
> The way I read up on UnionFS and aufs'' functionality, was they
could essentially merge two files virtually,
Merge two directories together, to be exact. Not merge files.
> i.e., the kernel module would be able to look at the operation coming in
and route it to the proper descriptor (i.e., read() -->
/livecd/windows/winxp.img OR write() --> /tmp/winxp.img, with /tmp being in
tmpfs).  I guess it''s not as granular as that it seems.  Would be a
neat trick, but I imagine it''d be complex as anything for a kernel
module to have to keep track of which files have variants loaded in the
writeable union area.
What you describe is essentially block-level copy-on-write, which is
what qcow or zfs does. Aufs/unionfs does this per file.
> As for the application, it''s a complex network security scanner,
made by eEye Digital Security, called "Retina".  We just
don''t want to setup and baby sit Windows installations on our Unix
networks strictly for this one app, so I figured if I can get it to run off of a
CD, we can just park some diskless hardware in a closet and pull it out whenever
we need to do network testing and such.
If it were me I''d simply setup a Windows domU on any server with
enough disk and RAM, make a "template" of the "good"
installation
(preferably zfs or LVM snapshot), start it only when it''s needed, shut
it down afterwards, and (if necessary) rollback to the "good"
template. That is assuming that all host/network tested reachable from
my datacenter (either with vlans or routing). No need to add the
complexity of a live DVD/USB.

If portability is a requirement, and you''re absolutely sure
you''ll
always have VT-capable host handy, then using live USB is much
perefered than DVD due to performance and complexity reasons. But hey,
that''s just me :D

Let us know if you find a solution that works.

--
Fajar
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Almost forgot, I got the idea and command examples for this from these two
sites:

Cowloop comparison vs. device-mapper
http://www.atcomputing.nl/Tools/cowloop/devicemapper.html

Right to your own device:
http://linuxgazette.net/114/kapil.html


________________________________________
From: xen-users-bounces@lists.xensource.com
[xen-users-bounces@lists.xensource.com] On Behalf Of Joshua Kinard
[joshua.kinard@sdc-world.com]
Sent: Monday, August 17, 2009 9:50 AM
To: xen-users@lists.xensource.com
Subject: RE: [Xen-users] Bootable LiveDVD w/ Xen that boots Windows Image?

Hmm, block-level copy-on-write?  If qcow supports this out-of-the-box, then I
technically don''t need any kind of UnionFS?  Just someway to specify to
Xen a read-only image with a qcow2 image in a writable zone?

Going to dredge through some EU software patent I dug up from Microsoft. 
Supposed to document the /MININT switch to the NT kernel that supposedly makes
Windows write all registry changes to volatile memory instead of to the registry
hives.  It''s mostly used in the WinPE environment, but it looks like
BartPE leverages it too.  Seems I need more to it, though, than just modifying
boot.ini to pass it, as once I started to boot Windows from a read-only drive
mount (after discovering Xen/qemu don''t properly honor the mode bit in
the Xen config for r or r/w, and even file-level permissions), Windows crashed
with UNMOUNTABLE_BOOT_VOLUME as its error.  But it highlights the capability is
there....just not easy.

USB is out, unfortunately, too.  For security reasons, we ban USB/Firewire
drives here, with the exception of CD Burners.  Ditto on memory cards (SD, MMC,
etc..).  I''ve got very narrow operating parameters to work in, which is
why this is proving to be quite a fun challenge.

Thanks!,

--J

________________________________________
From: Fajar A. Nugraha [fajar@fajar.net]
Sent: Friday, August 14, 2009 4:53 AM
To: Joshua Kinard
Cc: xen-users@lists.xensource.com
Subject: Re: [Xen-users] Bootable LiveDVD w/ Xen that boots Windows Image?

On Fri, Aug 14, 2009 at 1:37 AM, Joshua
Kinard<joshua.kinard@sdc-world.com> wrote:
> Hmm, I didn''t think of it that way.
>
> The way I read up on UnionFS and aufs'' functionality, was they
could essentially merge two files virtually,
Merge two directories together, to be exact. Not merge files.
> i.e., the kernel module would be able to look at the operation coming in
and route it to the proper descriptor (i.e., read() -->
/livecd/windows/winxp.img OR write() --> /tmp/winxp.img, with /tmp being in
tmpfs).  I guess it''s not as granular as that it seems.  Would be a
neat trick, but I imagine it''d be complex as anything for a kernel
module to have to keep track of which files have variants loaded in the
writeable union area.
What you describe is essentially block-level copy-on-write, which is
what qcow or zfs does. Aufs/unionfs does this per file.
> As for the application, it''s a complex network security scanner,
made by eEye Digital Security, called "Retina".  We just
don''t want to setup and baby sit Windows installations on our Unix
networks strictly for this one app, so I figured if I can get it to run off of a
CD, we can just park some diskless hardware in a closet and pull it out whenever
we need to do network testing and such.
If it were me I''d simply setup a Windows domU on any server with
enough disk and RAM, make a "template" of the "good"
installation
(preferably zfs or LVM snapshot), start it only when it''s needed, shut
it down afterwards, and (if necessary) rollback to the "good"
template. That is assuming that all host/network tested reachable from
my datacenter (either with vlans or routing). No need to add the
complexity of a live DVD/USB.

If portability is a requirement, and you''re absolutely sure
you''ll
always have VT-capable host handy, then using live USB is much
perefered than DVD due to performance and complexity reasons. But hey,
that''s just me :D

Let us know if you find a solution that works.

--
Fajar
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

This procedure will be in the next release of the Xen Live CD!!

qcow2 and LVM CoW snapshot for saving storage space while providing fast
domU creation from RW snapshots.... good for Eucalyptus, I guess!

2009/8/18 Joshua Kinard <joshua.kinard@sdc-world.com>
> Think I cracked it.  Don''t even need to deal with qemu''s
qcow stuff (so
> far).  Still more tests pending (including using read-only drive mounts).
>  But assume the following steps in a dom0:
>
> - mkdir /windows
> - cd /windows
> - dd if=/dev/zero of=xp.img bs=1024k count=1 seek=7168
> - touch xp.cfg
> - nano xp.cfg
> - <edit xp.cfg, setup as desired>
> - xm create xp.cfg
> - <install WinXP, patch, delete garbage, defrag, zero-out free space,
then
> shutdown>
> - xm list (verify domU is actually dead)
>
> Now after this, xp.img will contain a baseline WinXP install.  In my
> specific case, I compacted everything down to ~2GB total installed (inside
> the domU that is) by using straight qemu-emulated devices (not the GPLPV
> stuff), no page file or hibernation, stripping nearly every misc utility,
> and a bunch of other stuff, but NO NTFS compression (for now, at least).
>
> Next:
>
> - dd if=/dev/zero of=xp_scratch.img bs=1024k count=1 seek=2048
> - losetup /dev/loop0 xp.img
> - losetup /dev/loop1 xp_scratch.img
> - echo 0 $(blockdev --getsize /dev/loop0) snapshot /dev/loop0 /dev/loop1 n
> 64 | dmsetup create xpcow
>
> This maps our baseline WinXP image (xp.img) to /dev/loop0 and a scratch
> area to /dev/loop1 (I don''t even know if this is needed just yet).
Needed
> because dmsetup only works with block devices.  Using dmsetup, a snapshot
is
> created, so that all writes to loop0 get redirected to loop1, and creates a
> device, /dev/mapper/xpcow.
>
> Edit xp.cfg to use phy:/dev/mapper/xpcow,hda,w, and boot Windows:
> - xm create xp.cfg
>
> In Windows, create a dummy file on the desktop or three, then shutdown.
>  Re-start the domU and back at the desktop, we see that the newly-created
> files still exist, so shutdown again.  Now:
>
> - dmsetup remove /dev/mapper/xpcow
> - losetup -d /dev/loop1
> - dd if=/dev/zero of=xp_scratch2.img bs=1024k count=1 seek=2048
> - losetup /dev/loop1 xp_scratch2.img
> - echo 0 $(blockdev --getsize /dev/loop0) snapshot /dev/loop0 /dev/loop1 n
> 64 | dmsetup create xpcow
> - xm create xp.cfg
>
> Back at the desktop, the newly-created files are now gone!
>
> I did it this way as a manual test, because I''m veryifying a few
things:
> a. Simulating the effect of creating the scratch image in a /tmpfs mount,
> while the baseline image will be on a physical, read-only medium.  By
> removing the mapped device and changing out the scratch file, I was seeing
> whether the XP image would indeed revert to the baseline state, to simulate
> a reboot in which /tmpfs is cleared.  Supposedly, the ''n''
parameter to
> dmsetup does this anyways, but I wanted to be doubly sure.
>
> b. Need to scriptify the whole mess, since the baseline image will already
> be pre-configured, the generation of the scratch image, setup of the loop
> devices, and the device-mapping will be on-the-fly when some future CD
Image
> I put together boots up.
>
> c. Avoids all the problems I see on Google about using blktap with qcow
> with gplpv and such.  I''m probably not grasping some of it, but I
think
> those references are aiming for different uses.  So far, this setup mostly
> relies wholly on the Linux dom0 to piece everything together, leaving the
> WinXP domU utterly oblivious to what is going on.  And given
Windows''
> nature, that''s exactly what we want.
>
> Performance isn''t an issue for me in this, so I''m sure on
a CD, this will
> be slower than running Oracle on an 8086 (pretend for a moment that this is
> actually possible), but if it works, and WinXP doesn''t complain,
then all is
> good.
>
> Thoughts?
> ________________________________________
> From: xen-users-bounces@lists.xensource.com [
> xen-users-bounces@lists.xensource.com] On Behalf Of Joshua Kinard [
> joshua.kinard@sdc-world.com]
> Sent: Monday, August 17, 2009 9:50 AM
> To: xen-users@lists.xensource.com
> Subject: RE: [Xen-users] Bootable LiveDVD w/ Xen that boots Windows Image?
>
> Hmm, block-level copy-on-write?  If qcow supports this out-of-the-box, then
> I technically don''t need any kind of UnionFS?  Just someway to
specify to
> Xen a read-only image with a qcow2 image in a writable zone?
>
> Going to dredge through some EU software patent I dug up from Microsoft.
>  Supposed to document the /MININT switch to the NT kernel that supposedly
> makes Windows write all registry changes to volatile memory instead of to
> the registry hives.  It''s mostly used in the WinPE environment,
but it looks
> like BartPE leverages it too.  Seems I need more to it, though, than just
> modifying boot.ini to pass it, as once I started to boot Windows from a
> read-only drive mount (after discovering Xen/qemu don''t properly
honor the
> mode bit in the Xen config for r or r/w, and even file-level permissions),
> Windows crashed with UNMOUNTABLE_BOOT_VOLUME as its error.  But it
> highlights the capability is there....just not easy.
>
> USB is out, unfortunately, too.  For security reasons, we ban USB/Firewire
> drives here, with the exception of CD Burners.  Ditto on memory cards (SD,
> MMC, etc..).  I''ve got very narrow operating parameters to work
in, which is
> why this is proving to be quite a fun challenge.
>
> Thanks!,
>
> --J
>
> ________________________________________
> From: Fajar A. Nugraha [fajar@fajar.net]
> Sent: Friday, August 14, 2009 4:53 AM
> To: Joshua Kinard
> Cc: xen-users@lists.xensource.com
> Subject: Re: [Xen-users] Bootable LiveDVD w/ Xen that boots Windows Image?
>
> On Fri, Aug 14, 2009 at 1:37 AM, Joshua
> Kinard<joshua.kinard@sdc-world.com> wrote:
> > Hmm, I didn''t think of it that way.
> >
> > The way I read up on UnionFS and aufs'' functionality, was
they could
> essentially merge two files virtually,
>
> Merge two directories together, to be exact. Not merge files.
>
> > i.e., the kernel module would be able to look at the operation coming
in
> and route it to the proper descriptor (i.e., read() -->
> /livecd/windows/winxp.img OR write() --> /tmp/winxp.img, with /tmp being
in
> tmpfs).  I guess it''s not as granular as that it seems.  Would be
a neat
> trick, but I imagine it''d be complex as anything for a kernel
module to have
> to keep track of which files have variants loaded in the writeable union
> area.
>
> What you describe is essentially block-level copy-on-write, which is
> what qcow or zfs does. Aufs/unionfs does this per file.
>
> > As for the application, it''s a complex network security
scanner, made by
> eEye Digital Security, called "Retina".  We just don''t
want to setup and
> baby sit Windows installations on our Unix networks strictly for this one
> app, so I figured if I can get it to run off of a CD, we can just park some
> diskless hardware in a closet and pull it out whenever we need to do
network
> testing and such.
>
> If it were me I''d simply setup a Windows domU on any server with
> enough disk and RAM, make a "template" of the "good"
installation
> (preferably zfs or LVM snapshot), start it only when it''s needed,
shut
> it down afterwards, and (if necessary) rollback to the "good"
> template. That is assuming that all host/network tested reachable from
> my datacenter (either with vlans or routing). No need to add the
> complexity of a live DVD/USB.
>
> If portability is a requirement, and you''re absolutely sure
you''ll
> always have VT-capable host handy, then using live USB is much
> perefered than DVD due to performance and complexity reasons. But hey,
> that''s just me :D
>
> Let us know if you find a solution that works.
>
> --
> Fajar
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xensource.com
> http://lists.xensource.com/xen-users
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xensource.com
> http://lists.xensource.com/xen-users
>

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On Tue, Aug 18, 2009 at 11:13 PM, Joshua
Kinard<joshua.kinard@sdc-world.com> wrote:
> Think I cracked it.  Don''t even need to deal with qemu''s
qcow stuff (so far).
> - echo 0 $(blockdev --getsize /dev/loop0) snapshot /dev/loop0 /dev/loop1 n
64 | dmsetup create xpcow
>
> This maps our baseline WinXP image (xp.img) to /dev/loop0 and a scratch
area to /dev/loop1 (I don''t even know if this is needed just yet).
 Needed because dmsetup only works with block devices.  Using dmsetup, a
snapshot is created, so that all writes to loop0 get redirected to loop1, and
creates a device, /dev/mapper/xpcow.
Good one! I know about LVM snapshot, but didn''t know you can do that
with dmsetup. Thanks for the info.

-- 
Fajar

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Hmm, I don''t think qcow format is going to work with that method.  At
least when I tried it, the bochs bios didn''t like what it was looking
at.  But it''s hard to tell if that was bochs, or ntldr giving me a fit.

I do know that RAW images work fine, though, and I just tested it with squashfs
by running mksquashfs on the base image, and mounting that via loop2 to my base/
subfolder.  Then re-created the scratch area, ran dmsetup, and booted.  Loaded
up great, did some changes, shutdown, unloaded the mapper device, unloaded the
scratch loop, unmounted my tmpfs, and re-created that segment again, and Windows
came back in pristine condition.  So that pretty much means my next battle is
Debian''s package manager not liking my local repository (I
can''t connect my Xen system to the internet, so I have to use such a
setup).  As that''s how the Xen LiveCD scripts are doing things, and
assuming everything can be fetched online.

Some things I have noticed, though:
- After detaching a loopback device, there seems to be a delay in when the
kernel actually notices that it''s detached.  If one does losetup -d
<loop> followed by losetup <loop> <new img>, it seems like in
a few cases, the kernel still thought it was looking at the older loop
attachment.  I ran into this when using loopback to look at the disk at the disk
level, and then passing -o 32256 to losetup to look strictly at the
first/primary NTFS partition (while trying to clone it to a smaller disk image,
which I finally gave up on doing).

- Haven''t found the cause just yet, but dmsetup sometimes
doesn''t like looking at loopback devices, and possibly per the first
item, refuses to create the mapped device, citing a resource is busy.  Reboot
fixes this, and I suspect both of these are caused by all the experimenting
I''m doing with switching between the various loopbacks, I probably
confuse the kernel.  From a scripted standpoint, it should be a non-issue,
though.

- Only tested this entire idea/setup against Windows XP.  It seems to be fine,
and that generally means any other OS as a domU should work fine as well
(because NTFS is by far the pickiest, most temperamental FS known).  You get way
more options with Linux/Other UNIXes as far as tricking them into thinking
they''re in a r/w world, but this can be extended to encapsulate quite a
few domU''s.  However, the only limitation I can see is the number of
loopback interfaces.  I use three to setup the Windows domU: /dev/loop2 for the
squashfs mount, /dev/loop0 for the windows image inside the squashfs mount, and
/dev/loop1 for the scratch area.  I think more loop devices can be easily
created, but I''ve not experimented that far just yet...

That''s all for now...I''ve got to install my network scanner
into this Windows image and re-clean it and run these tests all over again.

Cheers!

--J
________________________________
From: Thiago Camargo Martins Cordeiro [thiagocmartinsc@gmail.com]
Sent: Tuesday, August 18, 2009 11:29 PM
To: Joshua Kinard
Cc: xen-users@lists.xensource.com
Subject: Re: [Xen-users] Bootable LiveDVD w/ Xen that boots Windows Image?

This procedure will be in the next release of the Xen Live CD!!

qcow2 and LVM CoW snapshot for saving storage space while providing fast domU
creation from RW snapshots.... good for Eucalyptus, I guess!

2009/8/18 Joshua Kinard
<joshua.kinard@sdc-world.com<mailto:joshua.kinard@sdc-world.com>>
Think I cracked it.  Don''t even need to deal with qemu''s qcow
stuff (so far).  Still more tests pending (including using read-only drive
mounts).  But assume the following steps in a dom0:

- mkdir /windows
- cd /windows
- dd if=/dev/zero of=xp.img bs=1024k count=1 seek=7168
- touch xp.cfg
- nano xp.cfg
- <edit xp.cfg, setup as desired>
- xm create xp.cfg
- <install WinXP, patch, delete garbage, defrag, zero-out free space, then
shutdown>
- xm list (verify domU is actually dead)

Now after this, xp.img will contain a baseline WinXP install.  In my specific
case, I compacted everything down to ~2GB total installed (inside the domU that
is) by using straight qemu-emulated devices (not the GPLPV stuff), no page file
or hibernation, stripping nearly every misc utility, and a bunch of other stuff,
but NO NTFS compression (for now, at least).

Next:

- dd if=/dev/zero of=xp_scratch.img bs=1024k count=1 seek=2048
- losetup /dev/loop0 xp.img
- losetup /dev/loop1 xp_scratch.img
- echo 0 $(blockdev --getsize /dev/loop0) snapshot /dev/loop0 /dev/loop1 n 64 |
dmsetup create xpcow

This maps our baseline WinXP image (xp.img) to /dev/loop0 and a scratch area to
/dev/loop1 (I don''t even know if this is needed just yet).  Needed
because dmsetup only works with block devices.  Using dmsetup, a snapshot is
created, so that all writes to loop0 get redirected to loop1, and creates a
device, /dev/mapper/xpcow.

Edit xp.cfg to use phy:/dev/mapper/xpcow,hda,w, and boot Windows:
- xm create xp.cfg

In Windows, create a dummy file on the desktop or three, then shutdown. 
Re-start the domU and back at the desktop, we see that the newly-created files
still exist, so shutdown again.  Now:

- dmsetup remove /dev/mapper/xpcow
- losetup -d /dev/loop1
- dd if=/dev/zero of=xp_scratch2.img bs=1024k count=1 seek=2048
- losetup /dev/loop1 xp_scratch2.img
- echo 0 $(blockdev --getsize /dev/loop0) snapshot /dev/loop0 /dev/loop1 n 64 |
dmsetup create xpcow
- xm create xp.cfg

Back at the desktop, the newly-created files are now gone!

I did it this way as a manual test, because I''m veryifying a few
things:
a. Simulating the effect of creating the scratch image in a /tmpfs mount, while
the baseline image will be on a physical, read-only medium.  By removing the
mapped device and changing out the scratch file, I was seeing whether the XP
image would indeed revert to the baseline state, to simulate a reboot in which
/tmpfs is cleared.  Supposedly, the ''n'' parameter to dmsetup
does this anyways, but I wanted to be doubly sure.

b. Need to scriptify the whole mess, since the baseline image will already be
pre-configured, the generation of the scratch image, setup of the loop devices,
and the device-mapping will be on-the-fly when some future CD Image I put
together boots up.

c. Avoids all the problems I see on Google about using blktap with qcow with
gplpv and such.  I''m probably not grasping some of it, but I think
those references are aiming for different uses.  So far, this setup mostly
relies wholly on the Linux dom0 to piece everything together, leaving the WinXP
domU utterly oblivious to what is going on.  And given Windows'' nature,
that''s exactly what we want.

Performance isn''t an issue for me in this, so I''m sure on a
CD, this will be slower than running Oracle on an 8086 (pretend for a moment
that this is actually possible), but if it works, and WinXP doesn''t
complain, then all is good.

Thoughts?
________________________________________
From:
xen-users-bounces@lists.xensource.com<mailto:xen-users-bounces@lists.xensource.com>
[xen-users-bounces@lists.xensource.com<mailto:xen-users-bounces@lists.xensource.com>]
On Behalf Of Joshua Kinard
[joshua.kinard@sdc-world.com<mailto:joshua.kinard@sdc-world.com>]
Sent: Monday, August 17, 2009 9:50 AM
To: xen-users@lists.xensource.com<mailto:xen-users@lists.xensource.com>
Subject: RE: [Xen-users] Bootable LiveDVD w/ Xen that boots Windows Image?

Hmm, block-level copy-on-write?  If qcow supports this out-of-the-box, then I
technically don''t need any kind of UnionFS?  Just someway to specify to
Xen a read-only image with a qcow2 image in a writable zone?

Going to dredge through some EU software patent I dug up from Microsoft. 
Supposed to document the /MININT switch to the NT kernel that supposedly makes
Windows write all registry changes to volatile memory instead of to the registry
hives.  It''s mostly used in the WinPE environment, but it looks like
BartPE leverages it too.  Seems I need more to it, though, than just modifying
boot.ini to pass it, as once I started to boot Windows from a read-only drive
mount (after discovering Xen/qemu don''t properly honor the mode bit in
the Xen config for r or r/w, and even file-level permissions), Windows crashed
with UNMOUNTABLE_BOOT_VOLUME as its error.  But it highlights the capability is
there....just not easy.

USB is out, unfortunately, too.  For security reasons, we ban USB/Firewire
drives here, with the exception of CD Burners.  Ditto on memory cards (SD, MMC,
etc..).  I''ve got very narrow operating parameters to work in, which is
why this is proving to be quite a fun challenge.

Thanks!,

--J

________________________________________
From: Fajar A. Nugraha [fajar@fajar.net<mailto:fajar@fajar.net>]
Sent: Friday, August 14, 2009 4:53 AM
To: Joshua Kinard
Cc: xen-users@lists.xensource.com<mailto:xen-users@lists.xensource.com>
Subject: Re: [Xen-users] Bootable LiveDVD w/ Xen that boots Windows Image?

On Fri, Aug 14, 2009 at 1:37 AM, Joshua
Kinard<joshua.kinard@sdc-world.com<mailto:joshua.kinard@sdc-world.com>>
wrote:
> Hmm, I didn''t think of it that way.
>
> The way I read up on UnionFS and aufs'' functionality, was they
could essentially merge two files virtually,
Merge two directories together, to be exact. Not merge files.
> i.e., the kernel module would be able to look at the operation coming in
and route it to the proper descriptor (i.e., read() -->
/livecd/windows/winxp.img OR write() --> /tmp/winxp.img, with /tmp being in
tmpfs).  I guess it''s not as granular as that it seems.  Would be a
neat trick, but I imagine it''d be complex as anything for a kernel
module to have to keep track of which files have variants loaded in the
writeable union area.
What you describe is essentially block-level copy-on-write, which is
what qcow or zfs does. Aufs/unionfs does this per file.
> As for the application, it''s a complex network security scanner,
made by eEye Digital Security, called "Retina".  We just
don''t want to setup and baby sit Windows installations on our Unix
networks strictly for this one app, so I figured if I can get it to run off of a
CD, we can just park some diskless hardware in a closet and pull it out whenever
we need to do network testing and such.
If it were me I''d simply setup a Windows domU on any server with
enough disk and RAM, make a "template" of the "good"
installation
(preferably zfs or LVM snapshot), start it only when it''s needed, shut
it down afterwards, and (if necessary) rollback to the "good"
template. That is assuming that all host/network tested reachable from
my datacenter (either with vlans or routing). No need to add the
complexity of a live DVD/USB.

If portability is a requirement, and you''re absolutely sure
you''ll
always have VT-capable host handy, then using live USB is much
perefered than DVD due to performance and complexity reasons. But hey,
that''s just me :D

Let us know if you find a solution that works.

--
Fajar
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com<mailto:Xen-users@lists.xensource.com>
http://lists.xensource.com/xen-users
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com<mailto:Xen-users@lists.xensource.com>
http://lists.xensource.com/xen-users



_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Two questions, probably the same answer. I am just trying to  
understand why you are trying to do this, and what practical use this  
has. Maybe I will have a use for this in the future.

What is the goal in doing this?
Are usb thumb drives not an option?

Lewis

On Aug 19, 2009, at 7:56 AM, Joshua Kinard wrote:
> Hmm, I don''t think qcow format is going to work with that method.
> At least when I tried it, the bochs bios didn''t like what it was  
> looking at.  But it''s hard to tell if that was bochs, or ntldr  
> giving me a fit.
>
> I do know that RAW images work fine, though, and I just tested it  
> with squashfs by running mksquashfs on the base image, and mounting  
> that via loop2 to my base/ subfolder.  Then re-created the scratch  
> area, ran dmsetup, and booted.  Loaded up great, did some changes,  
> shutdown, unloaded the mapper device, unloaded the scratch loop,  
> unmounted my tmpfs, and re-created that segment again, and Windows  
> came back in pristine condition.  So that pretty much means my next  
> battle is Debian''s package manager not liking my local repository
(I
> can''t connect my Xen system to the internet, so I have to use such
a
> setup).  As that''s how the Xen LiveCD scripts are doing things,
and
> assuming everything can be fetched online.
>
> Some things I have noticed, though:
> - After detaching a loopback device, there seems to be a delay in  
> when the kernel actually notices that it''s detached.  If one does
> losetup -d <loop> followed by losetup <loop> <new img>,
it seems
> like in a few cases, the kernel still thought it was looking at the  
> older loop attachment.  I ran into this when using loopback to look  
> at the disk at the disk level, and then passing -o 32256 to losetup  
> to look strictly at the first/primary NTFS partition (while trying  
> to clone it to a smaller disk image, which I finally gave up on  
> doing).
>
> - Haven''t found the cause just yet, but dmsetup sometimes
doesn''t
> like looking at loopback devices, and possibly per the first item,  
> refuses to create the mapped device, citing a resource is busy.   
> Reboot fixes this, and I suspect both of these are caused by all the  
> experimenting I''m doing with switching between the various  
> loopbacks, I probably confuse the kernel.  From a scripted  
> standpoint, it should be a non-issue, though.
>
> - Only tested this entire idea/setup against Windows XP.  It seems  
> to be fine, and that generally means any other OS as a domU should  
> work fine as well (because NTFS is by far the pickiest, most  
> temperamental FS known).  You get way more options with Linux/Other  
> UNIXes as far as tricking them into thinking they''re in a r/w
world,
> but this can be extended to encapsulate quite a few domU''s.   
> However, the only limitation I can see is the number of loopback  
> interfaces.  I use three to setup the Windows domU: /dev/loop2 for  
> the squashfs mount, /dev/loop0 for the windows image inside the  
> squashfs mount, and /dev/loop1 for the scratch area.  I think more  
> loop devices can be easily created, but I''ve not experimented that
> far just yet...
>
> That''s all for now...I''ve got to install my network
scanner into
> this Windows image and re-clean it and run these tests all over again.
>
> Cheers!
>
> --J
> From: Thiago Camargo Martins Cordeiro [thiagocmartinsc@gmail.com]
> Sent: Tuesday, August 18, 2009 11:29 PM
> To: Joshua Kinard
> Cc: xen-users@lists.xensource.com
> Subject: Re: [Xen-users] Bootable LiveDVD w/ Xen that boots Windows  
> Image?
>
> This procedure will be in the next release of the Xen Live CD!!
>
> qcow2 and LVM CoW snapshot for saving storage space while providing  
> fast domU creation from RW snapshots.... good for Eucalyptus, I guess!
>
> 2009/8/18 Joshua Kinard <joshua.kinard@sdc-world.com>
> Think I cracked it.  Don''t even need to deal with qemu''s
qcow stuff
> (so far).  Still more tests pending (including using read-only drive  
> mounts).  But assume the following steps in a dom0:
>
> - mkdir /windows
> - cd /windows
> - dd if=/dev/zero of=xp.img bs=1024k count=1 seek=7168
> - touch xp.cfg
> - nano xp.cfg
> - <edit xp.cfg, setup as desired>
> - xm create xp.cfg
> - <install WinXP, patch, delete garbage, defrag, zero-out free  
> space, then shutdown>
> - xm list (verify domU is actually dead)
>
> Now after this, xp.img will contain a baseline WinXP install.  In my  
> specific case, I compacted everything down to ~2GB total installed  
> (inside the domU that is) by using straight qemu-emulated devices  
> (not the GPLPV stuff), no page file or hibernation, stripping nearly  
> every misc utility, and a bunch of other stuff, but NO NTFS  
> compression (for now, at least).
>
> Next:
>
> - dd if=/dev/zero of=xp_scratch.img bs=1024k count=1 seek=2048
> - losetup /dev/loop0 xp.img
> - losetup /dev/loop1 xp_scratch.img
> - echo 0 $(blockdev --getsize /dev/loop0) snapshot /dev/loop0 /dev/ 
> loop1 n 64 | dmsetup create xpcow
>
> This maps our baseline WinXP image (xp.img) to /dev/loop0 and a  
> scratch area to /dev/loop1 (I don''t even know if this is needed
just
> yet).  Needed because dmsetup only works with block devices.  Using  
> dmsetup, a snapshot is created, so that all writes to loop0 get  
> redirected to loop1, and creates a device, /dev/mapper/xpcow.
>
> Edit xp.cfg to use phy:/dev/mapper/xpcow,hda,w, and boot Windows:
> - xm create xp.cfg
>
> In Windows, create a dummy file on the desktop or three, then  
> shutdown.  Re-start the domU and back at the desktop, we see that  
> the newly-created files still exist, so shutdown again.  Now:
>
> - dmsetup remove /dev/mapper/xpcow
> - losetup -d /dev/loop1
> - dd if=/dev/zero of=xp_scratch2.img bs=1024k count=1 seek=2048
> - losetup /dev/loop1 xp_scratch2.img
> - echo 0 $(blockdev --getsize /dev/loop0) snapshot /dev/loop0 /dev/ 
> loop1 n 64 | dmsetup create xpcow
> - xm create xp.cfg
>
> Back at the desktop, the newly-created files are now gone!
>
> I did it this way as a manual test, because I''m veryifying a few  
> things:
> a. Simulating the effect of creating the scratch image in a /tmpfs  
> mount, while the baseline image will be on a physical, read-only  
> medium.  By removing the mapped device and changing out the scratch  
> file, I was seeing whether the XP image would indeed revert to the  
> baseline state, to simulate a reboot in which /tmpfs is cleared.   
> Supposedly, the ''n'' parameter to dmsetup does this
anyways, but I
> wanted to be doubly sure.
>
> b. Need to scriptify the whole mess, since the baseline image will  
> already be pre-configured, the generation of the scratch image,  
> setup of the loop devices, and the device-mapping will be on-the-fly  
> when some future CD Image I put together boots up.
>
> c. Avoids all the problems I see on Google about using blktap with  
> qcow with gplpv and such.  I''m probably not grasping some of it,
but
> I think those references are aiming for different uses.  So far,  
> this setup mostly relies wholly on the Linux dom0 to piece  
> everything together, leaving the WinXP domU utterly oblivious to  
> what is going on.  And given Windows'' nature, that''s
exactly what we
> want.
>
> Performance isn''t an issue for me in this, so I''m sure on
a CD, this
> will be slower than running Oracle on an 8086 (pretend for a moment  
> that this is actually possible), but if it works, and WinXP
doesn''t
> complain, then all is good.
>
> Thoughts?
> ________________________________________
> From: xen-users-bounces@lists.xensource.com
[xen-users-bounces@lists.xensource.com
> ] On Behalf Of Joshua Kinard [joshua.kinard@sdc-world.com]
> Sent: Monday, August 17, 2009 9:50 AM
> To: xen-users@lists.xensource.com
> Subject: RE: [Xen-users] Bootable LiveDVD w/ Xen that boots Windows  
> Image?
>
> Hmm, block-level copy-on-write?  If qcow supports this out-of-the- 
> box, then I technically don''t need any kind of UnionFS?  Just  
> someway to specify to Xen a read-only image with a qcow2 image in a  
> writable zone?
>
> Going to dredge through some EU software patent I dug up from  
> Microsoft.  Supposed to document the /MININT switch to the NT kernel  
> that supposedly makes Windows write all registry changes to volatile  
> memory instead of to the registry hives.  It''s mostly used in the
> WinPE environment, but it looks like BartPE leverages it too.  Seems  
> I need more to it, though, than just modifying boot.ini to pass it,  
> as once I started to boot Windows from a read-only drive mount  
> (after discovering Xen/qemu don''t properly honor the mode bit in
the
> Xen config for r or r/w, and even file-level permissions), Windows  
> crashed with UNMOUNTABLE_BOOT_VOLUME as its error.  But it  
> highlights the capability is there....just not easy.
>
> USB is out, unfortunately, too.  For security reasons, we ban USB/ 
> Firewire drives here, with the exception of CD Burners.  Ditto on  
> memory cards (SD, MMC, etc..).  I''ve got very narrow operating  
> parameters to work in, which is why this is proving to be quite a  
> fun challenge.
>
> Thanks!,
>
> --J
>
> ________________________________________
> From: Fajar A. Nugraha [fajar@fajar.net]
> Sent: Friday, August 14, 2009 4:53 AM
> To: Joshua Kinard
> Cc: xen-users@lists.xensource.com
> Subject: Re: [Xen-users] Bootable LiveDVD w/ Xen that boots Windows  
> Image?
>
> On Fri, Aug 14, 2009 at 1:37 AM, Joshua
> Kinard<joshua.kinard@sdc-world.com> wrote:
> > Hmm, I didn''t think of it that way.
> >
> > The way I read up on UnionFS and aufs'' functionality, was
they
> could essentially merge two files virtually,
>
> Merge two directories together, to be exact. Not merge files.
>
> > i.e., the kernel module would be able to look at the operation  
> coming in and route it to the proper descriptor (i.e., read() --> / 
> livecd/windows/winxp.img OR write() --> /tmp/winxp.img, with /tmp  
> being in tmpfs).  I guess it''s not as granular as that it seems.
> Would be a neat trick, but I imagine it''d be complex as anything
for
> a kernel module to have to keep track of which files have variants  
> loaded in the writeable union area.
>
> What you describe is essentially block-level copy-on-write, which is
> what qcow or zfs does. Aufs/unionfs does this per file.
>
> > As for the application, it''s a complex network security
scanner,
> made by eEye Digital Security, called "Retina".  We just
don''t want
> to setup and baby sit Windows installations on our Unix networks  
> strictly for this one app, so I figured if I can get it to run off  
> of a CD, we can just park some diskless hardware in a closet and  
> pull it out whenever we need to do network testing and such.
>
> If it were me I''d simply setup a Windows domU on any server with
> enough disk and RAM, make a "template" of the "good"
installation
> (preferably zfs or LVM snapshot), start it only when it''s needed,
shut
> it down afterwards, and (if necessary) rollback to the "good"
> template. That is assuming that all host/network tested reachable from
> my datacenter (either with vlans or routing). No need to add the
> complexity of a live DVD/USB.
>
> If portability is a requirement, and you''re absolutely sure
you''ll
> always have VT-capable host handy, then using live USB is much
> perefered than DVD due to performance and complexity reasons. But hey,
> that''s just me :D
>
> Let us know if you find a solution that works.
>
> --
> Fajar
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xensource.com
> http://lists.xensource.com/xen-users
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xensource.com
> http://lists.xensource.com/xen-users
>
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xensource.com
> http://lists.xensource.com/xen-users

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

try a make KERNCONF=XEN menuconfig

then you should see some xen options under the CPU option.

then isssue a make KERNCONF=XEN to make your kernel.

----- Original Message ----- 
From: "John Lewis" <jlewis@concentric.com>
To: "Joshua Kinard" <joshua.kinard@sdc-world.com>
Cc: <xen-users@lists.xensource.com>
Sent: Wednesday, August 19, 2009 1:24 PM
Subject: Re: [Xen-users] Bootable LiveDVD w/ Xen that boots Windows Image?

> Two questions, probably the same answer. I am just trying to  understand 
> why you are trying to do this, and what practical use this  has. Maybe I 
> will have a use for this in the future.
>
> What is the goal in doing this?
> Are usb thumb drives not an option?
>
> Lewis
>
> On Aug 19, 2009, at 7:56 AM, Joshua Kinard wrote:
>
>> Hmm, I don''t think qcow format is going to work with that
method.   At
>> least when I tried it, the bochs bios didn''t like what it was 
looking
>> at.  But it''s hard to tell if that was bochs, or ntldr  giving
me a fit.
>>
>> I do know that RAW images work fine, though, and I just tested it  with
>> squashfs by running mksquashfs on the base image, and mounting  that
via
>> loop2 to my base/ subfolder.  Then re-created the scratch  area, ran 
>> dmsetup, and booted.  Loaded up great, did some changes,  shutdown, 
>> unloaded the mapper device, unloaded the scratch loop,  unmounted my 
>> tmpfs, and re-created that segment again, and Windows  came back in 
>> pristine condition.  So that pretty much means my next  battle is 
>> Debian''s package manager not liking my local repository (I 
can''t connect
>> my Xen system to the internet, so I have to use such a  setup).  As 
>> that''s how the Xen LiveCD scripts are doing things, and 
assuming
>> everything can be fetched online.
>>
>> Some things I have noticed, though:
>> - After detaching a loopback device, there seems to be a delay in  when
>> the kernel actually notices that it''s detached.  If one does 
losetup -d
>> <loop> followed by losetup <loop> <new img>, it seems
like in a few
>> cases, the kernel still thought it was looking at the  older loop 
>> attachment.  I ran into this when using loopback to look  at the disk
at
>> the disk level, and then passing -o 32256 to losetup  to look strictly
at
>> the first/primary NTFS partition (while trying  to clone it to a
smaller
>> disk image, which I finally gave up on  doing).
>>
>> - Haven''t found the cause just yet, but dmsetup sometimes
doesn''t  like
>> looking at loopback devices, and possibly per the first item,  refuses
to
>> create the mapped device, citing a resource is busy.   Reboot fixes
this,
>> and I suspect both of these are caused by all the  experimenting
I''m
>> doing with switching between the various  loopbacks, I probably confuse
>> the kernel.  From a scripted  standpoint, it should be a non-issue, 
>> though.
>>
>> - Only tested this entire idea/setup against Windows XP.  It seems  to
be
>> fine, and that generally means any other OS as a domU should  work fine
>> as well (because NTFS is by far the pickiest, most  temperamental FS 
>> known).  You get way more options with Linux/Other  UNIXes as far as 
>> tricking them into thinking they''re in a r/w world,  but this
can be
>> extended to encapsulate quite a few domU''s.   However, the
only
>> limitation I can see is the number of loopback  interfaces.  I use
three
>> to setup the Windows domU: /dev/loop2 for  the squashfs mount,
/dev/loop0
>> for the windows image inside the  squashfs mount, and /dev/loop1 for
the
>> scratch area.  I think more  loop devices can be easily created, but
I''ve
>> not experimented that  far just yet...
>>
>> That''s all for now...I''ve got to install my network
scanner into  this
>> Windows image and re-clean it and run these tests all over again.
>>
>> Cheers!
>>
>> --J
>> From: Thiago Camargo Martins Cordeiro [thiagocmartinsc@gmail.com]
>> Sent: Tuesday, August 18, 2009 11:29 PM
>> To: Joshua Kinard
>> Cc: xen-users@lists.xensource.com
>> Subject: Re: [Xen-users] Bootable LiveDVD w/ Xen that boots Windows 
>> Image?
>>
>> This procedure will be in the next release of the Xen Live CD!!
>>
>> qcow2 and LVM CoW snapshot for saving storage space while providing 
fast
>> domU creation from RW snapshots.... good for Eucalyptus, I guess!
>>
>> 2009/8/18 Joshua Kinard <joshua.kinard@sdc-world.com>
>> Think I cracked it.  Don''t even need to deal with
qemu''s qcow stuff  (so
>> far).  Still more tests pending (including using read-only drive 
>> mounts).  But assume the following steps in a dom0:
>>
>> - mkdir /windows
>> - cd /windows
>> - dd if=/dev/zero of=xp.img bs=1024k count=1 seek=7168
>> - touch xp.cfg
>> - nano xp.cfg
>> - <edit xp.cfg, setup as desired>
>> - xm create xp.cfg
>> - <install WinXP, patch, delete garbage, defrag, zero-out free 
space,
>> then shutdown>
>> - xm list (verify domU is actually dead)
>>
>> Now after this, xp.img will contain a baseline WinXP install.  In my 
>> specific case, I compacted everything down to ~2GB total installed 
>> (inside the domU that is) by using straight qemu-emulated devices  (not
>> the GPLPV stuff), no page file or hibernation, stripping nearly  every 
>> misc utility, and a bunch of other stuff, but NO NTFS  compression (for
>> now, at least).
>>
>> Next:
>>
>> - dd if=/dev/zero of=xp_scratch.img bs=1024k count=1 seek=2048
>> - losetup /dev/loop0 xp.img
>> - losetup /dev/loop1 xp_scratch.img
>> - echo 0 $(blockdev --getsize /dev/loop0) snapshot /dev/loop0 /dev/
loop1
>> n 64 | dmsetup create xpcow
>>
>> This maps our baseline WinXP image (xp.img) to /dev/loop0 and a 
scratch
>> area to /dev/loop1 (I don''t even know if this is needed just 
yet).
>> Needed because dmsetup only works with block devices.  Using  dmsetup,
a
>> snapshot is created, so that all writes to loop0 get  redirected to 
>> loop1, and creates a device, /dev/mapper/xpcow.
>>
>> Edit xp.cfg to use phy:/dev/mapper/xpcow,hda,w, and boot Windows:
>> - xm create xp.cfg
>>
>> In Windows, create a dummy file on the desktop or three, then 
shutdown.
>> Re-start the domU and back at the desktop, we see that  the
newly-created
>> files still exist, so shutdown again.  Now:
>>
>> - dmsetup remove /dev/mapper/xpcow
>> - losetup -d /dev/loop1
>> - dd if=/dev/zero of=xp_scratch2.img bs=1024k count=1 seek=2048
>> - losetup /dev/loop1 xp_scratch2.img
>> - echo 0 $(blockdev --getsize /dev/loop0) snapshot /dev/loop0 /dev/
loop1
>> n 64 | dmsetup create xpcow
>> - xm create xp.cfg
>>
>> Back at the desktop, the newly-created files are now gone!
>>
>> I did it this way as a manual test, because I''m veryifying a
few  things:
>> a. Simulating the effect of creating the scratch image in a /tmpfs 
>> mount, while the baseline image will be on a physical, read-only 
medium.
>> By removing the mapped device and changing out the scratch  file, I was
>> seeing whether the XP image would indeed revert to the  baseline state,
>> to simulate a reboot in which /tmpfs is cleared.   Supposedly, the
''n''
>> parameter to dmsetup does this anyways, but I  wanted to be doubly
sure.
>>
>> b. Need to scriptify the whole mess, since the baseline image will 
>> already be pre-configured, the generation of the scratch image,  setup
of
>> the loop devices, and the device-mapping will be on-the-fly  when some 
>> future CD Image I put together boots up.
>>
>> c. Avoids all the problems I see on Google about using blktap with 
qcow
>> with gplpv and such.  I''m probably not grasping some of it,
but  I think
>> those references are aiming for different uses.  So far,  this setup 
>> mostly relies wholly on the Linux dom0 to piece  everything together, 
>> leaving the WinXP domU utterly oblivious to  what is going on.  And
given
>> Windows'' nature, that''s exactly what we  want.
>>
>> Performance isn''t an issue for me in this, so I''m
sure on a CD, this
>> will be slower than running Oracle on an 8086 (pretend for a moment 
that
>> this is actually possible), but if it works, and WinXP doesn''t
complain,
>> then all is good.
>>
>> Thoughts?
>> ________________________________________
>> From: xen-users-bounces@lists.xensource.com 
>> [xen-users-bounces@lists.xensource.com ] On Behalf Of Joshua Kinard 
>> [joshua.kinard@sdc-world.com]
>> Sent: Monday, August 17, 2009 9:50 AM
>> To: xen-users@lists.xensource.com
>> Subject: RE: [Xen-users] Bootable LiveDVD w/ Xen that boots Windows 
>> Image?
>>
>> Hmm, block-level copy-on-write?  If qcow supports this out-of-the- box,
>> then I technically don''t need any kind of UnionFS?  Just 
someway to
>> specify to Xen a read-only image with a qcow2 image in a  writable
zone?
>>
>> Going to dredge through some EU software patent I dug up from 
Microsoft.
>> Supposed to document the /MININT switch to the NT kernel  that
supposedly
>> makes Windows write all registry changes to volatile  memory instead of
>> to the registry hives.  It''s mostly used in the  WinPE
environment, but
>> it looks like BartPE leverages it too.  Seems  I need more to it,
though,
>> than just modifying boot.ini to pass it,  as once I started to boot 
>> Windows from a read-only drive mount  (after discovering Xen/qemu
don''t
>> properly honor the mode bit in the  Xen config for r or r/w, and even 
>> file-level permissions), Windows  crashed with UNMOUNTABLE_BOOT_VOLUME
as
>> its error.  But it  highlights the capability is there....just not
easy.
>>
>> USB is out, unfortunately, too.  For security reasons, we ban USB/ 
>> Firewire drives here, with the exception of CD Burners.  Ditto on 
memory
>> cards (SD, MMC, etc..).  I''ve got very narrow operating 
parameters to
>> work in, which is why this is proving to be quite a  fun challenge.
>>
>> Thanks!,
>>
>> --J
>>
>> ________________________________________
>> From: Fajar A. Nugraha [fajar@fajar.net]
>> Sent: Friday, August 14, 2009 4:53 AM
>> To: Joshua Kinard
>> Cc: xen-users@lists.xensource.com
>> Subject: Re: [Xen-users] Bootable LiveDVD w/ Xen that boots Windows 
>> Image?
>>
>> On Fri, Aug 14, 2009 at 1:37 AM, Joshua
>> Kinard<joshua.kinard@sdc-world.com> wrote:
>> > Hmm, I didn''t think of it that way.
>> >
>> > The way I read up on UnionFS and aufs'' functionality, was
they
>> could essentially merge two files virtually,
>>
>> Merge two directories together, to be exact. Not merge files.
>>
>> > i.e., the kernel module would be able to look at the operation
>> coming in and route it to the proper descriptor (i.e., read() --> / 
>> livecd/windows/winxp.img OR write() --> /tmp/winxp.img, with /tmp 
being
>> in tmpfs).  I guess it''s not as granular as that it seems.  
Would be a
>> neat trick, but I imagine it''d be complex as anything for  a
kernel
>> module to have to keep track of which files have variants  loaded in
the
>> writeable union area.
>>
>> What you describe is essentially block-level copy-on-write, which is
>> what qcow or zfs does. Aufs/unionfs does this per file.
>>
>> > As for the application, it''s a complex network security
scanner,
>> made by eEye Digital Security, called "Retina".  We just
don''t want  to
>> setup and baby sit Windows installations on our Unix networks  strictly
>> for this one app, so I figured if I can get it to run off  of a CD, we 
>> can just park some diskless hardware in a closet and  pull it out 
>> whenever we need to do network testing and such.
>>
>> If it were me I''d simply setup a Windows domU on any server
with
>> enough disk and RAM, make a "template" of the
"good" installation
>> (preferably zfs or LVM snapshot), start it only when it''s
needed, shut
>> it down afterwards, and (if necessary) rollback to the "good"
>> template. That is assuming that all host/network tested reachable from
>> my datacenter (either with vlans or routing). No need to add the
>> complexity of a live DVD/USB.
>>
>> If portability is a requirement, and you''re absolutely sure
you''ll
>> always have VT-capable host handy, then using live USB is much
>> perefered than DVD due to performance and complexity reasons. But hey,
>> that''s just me :D
>>
>> Let us know if you find a solution that works.
>>
>> --
>> Fajar
>> _______________________________________________
>> Xen-users mailing list
>> Xen-users@lists.xensource.com
>> http://lists.xensource.com/xen-users
>> _______________________________________________
>> Xen-users mailing list
>> Xen-users@lists.xensource.com
>> http://lists.xensource.com/xen-users
>>
>> _______________________________________________
>> Xen-users mailing list
>> Xen-users@lists.xensource.com
>> http://lists.xensource.com/xen-users
>
>
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xensource.com
> http://lists.xensource.com/xen-users
> 

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Pretty familiar with kernel building, actually (done a bit of work on SGI MIPS
platforms running Linux).  If you''re refering to my Debian issue,
that''s more just figuring out how to get their package manager to
"trust" my local repository so I can tweak the Xen LiveCD building
scripts to pull packages locally to build stuff.  Not too much of a hassle, just
a matter of some tinkering around.

Thanks though!,

--J

________________________________________
From: Robbie garrett [rgarrett@hostourweb.com]
Sent: Thursday, August 20, 2009 8:07 AM
To: John Lewis; Joshua Kinard
Cc: xen-users@lists.xensource.com
Subject: Re: [Xen-users] Bootable LiveDVD w/ Xen that boots Windows Image?

try a make KERNCONF=XEN menuconfig

then you should see some xen options under the CPU option.

then isssue a make KERNCONF=XEN to make your kernel.

----- Original Message -----
From: "John Lewis" <jlewis@concentric.com>
To: "Joshua Kinard" <joshua.kinard@sdc-world.com>
Cc: <xen-users@lists.xensource.com>
Sent: Wednesday, August 19, 2009 1:24 PM
Subject: Re: [Xen-users] Bootable LiveDVD w/ Xen that boots Windows Image?

> Two questions, probably the same answer. I am just trying to  understand
> why you are trying to do this, and what practical use this  has. Maybe I
> will have a use for this in the future.
>
> What is the goal in doing this?
> Are usb thumb drives not an option?
>
> Lewis
>
> On Aug 19, 2009, at 7:56 AM, Joshua Kinard wrote:
>
>> Hmm, I don''t think qcow format is going to work with that
method.   At
>> least when I tried it, the bochs bios didn''t like what it was 
looking
>> at.  But it''s hard to tell if that was bochs, or ntldr  giving
me a fit.
>>
>> I do know that RAW images work fine, though, and I just tested it  with
>> squashfs by running mksquashfs on the base image, and mounting  that
via
>> loop2 to my base/ subfolder.  Then re-created the scratch  area, ran
>> dmsetup, and booted.  Loaded up great, did some changes,  shutdown,
>> unloaded the mapper device, unloaded the scratch loop,  unmounted my
>> tmpfs, and re-created that segment again, and Windows  came back in
>> pristine condition.  So that pretty much means my next  battle is
>> Debian''s package manager not liking my local repository (I 
can''t connect
>> my Xen system to the internet, so I have to use such a  setup).  As
>> that''s how the Xen LiveCD scripts are doing things, and 
assuming
>> everything can be fetched online.
>>
>> Some things I have noticed, though:
>> - After detaching a loopback device, there seems to be a delay in  when
>> the kernel actually notices that it''s detached.  If one does 
losetup -d
>> <loop> followed by losetup <loop> <new img>, it seems
like in a few
>> cases, the kernel still thought it was looking at the  older loop
>> attachment.  I ran into this when using loopback to look  at the disk
at
>> the disk level, and then passing -o 32256 to losetup  to look strictly
at
>> the first/primary NTFS partition (while trying  to clone it to a
smaller
>> disk image, which I finally gave up on  doing).
>>
>> - Haven''t found the cause just yet, but dmsetup sometimes
doesn''t  like
>> looking at loopback devices, and possibly per the first item,  refuses
to
>> create the mapped device, citing a resource is busy.   Reboot fixes
this,
>> and I suspect both of these are caused by all the  experimenting
I''m
>> doing with switching between the various  loopbacks, I probably confuse
>> the kernel.  From a scripted  standpoint, it should be a non-issue,
>> though.
>>
>> - Only tested this entire idea/setup against Windows XP.  It seems  to
be
>> fine, and that generally means any other OS as a domU should  work fine
>> as well (because NTFS is by far the pickiest, most  temperamental FS
>> known).  You get way more options with Linux/Other  UNIXes as far as
>> tricking them into thinking they''re in a r/w world,  but this
can be
>> extended to encapsulate quite a few domU''s.   However, the
only
>> limitation I can see is the number of loopback  interfaces.  I use
three
>> to setup the Windows domU: /dev/loop2 for  the squashfs mount,
/dev/loop0
>> for the windows image inside the  squashfs mount, and /dev/loop1 for
the
>> scratch area.  I think more  loop devices can be easily created, but
I''ve
>> not experimented that  far just yet...
>>
>> That''s all for now...I''ve got to install my network
scanner into  this
>> Windows image and re-clean it and run these tests all over again.
>>
>> Cheers!
>>
>> --J
>> From: Thiago Camargo Martins Cordeiro [thiagocmartinsc@gmail.com]
>> Sent: Tuesday, August 18, 2009 11:29 PM
>> To: Joshua Kinard
>> Cc: xen-users@lists.xensource.com
>> Subject: Re: [Xen-users] Bootable LiveDVD w/ Xen that boots Windows
>> Image?
>>
>> This procedure will be in the next release of the Xen Live CD!!
>>
>> qcow2 and LVM CoW snapshot for saving storage space while providing 
fast
>> domU creation from RW snapshots.... good for Eucalyptus, I guess!
>>
>> 2009/8/18 Joshua Kinard <joshua.kinard@sdc-world.com>
>> Think I cracked it.  Don''t even need to deal with
qemu''s qcow stuff  (so
>> far).  Still more tests pending (including using read-only drive
>> mounts).  But assume the following steps in a dom0:
>>
>> - mkdir /windows
>> - cd /windows
>> - dd if=/dev/zero of=xp.img bs=1024k count=1 seek=7168
>> - touch xp.cfg
>> - nano xp.cfg
>> - <edit xp.cfg, setup as desired>
>> - xm create xp.cfg
>> - <install WinXP, patch, delete garbage, defrag, zero-out free 
space,
>> then shutdown>
>> - xm list (verify domU is actually dead)
>>
>> Now after this, xp.img will contain a baseline WinXP install.  In my
>> specific case, I compacted everything down to ~2GB total installed
>> (inside the domU that is) by using straight qemu-emulated devices  (not
>> the GPLPV stuff), no page file or hibernation, stripping nearly  every
>> misc utility, and a bunch of other stuff, but NO NTFS  compression (for
>> now, at least).
>>
>> Next:
>>
>> - dd if=/dev/zero of=xp_scratch.img bs=1024k count=1 seek=2048
>> - losetup /dev/loop0 xp.img
>> - losetup /dev/loop1 xp_scratch.img
>> - echo 0 $(blockdev --getsize /dev/loop0) snapshot /dev/loop0 /dev/
loop1
>> n 64 | dmsetup create xpcow
>>
>> This maps our baseline WinXP image (xp.img) to /dev/loop0 and a 
scratch
>> area to /dev/loop1 (I don''t even know if this is needed just 
yet).
>> Needed because dmsetup only works with block devices.  Using  dmsetup,
a
>> snapshot is created, so that all writes to loop0 get  redirected to
>> loop1, and creates a device, /dev/mapper/xpcow.
>>
>> Edit xp.cfg to use phy:/dev/mapper/xpcow,hda,w, and boot Windows:
>> - xm create xp.cfg
>>
>> In Windows, create a dummy file on the desktop or three, then 
shutdown.
>> Re-start the domU and back at the desktop, we see that  the
newly-created
>> files still exist, so shutdown again.  Now:
>>
>> - dmsetup remove /dev/mapper/xpcow
>> - losetup -d /dev/loop1
>> - dd if=/dev/zero of=xp_scratch2.img bs=1024k count=1 seek=2048
>> - losetup /dev/loop1 xp_scratch2.img
>> - echo 0 $(blockdev --getsize /dev/loop0) snapshot /dev/loop0 /dev/
loop1
>> n 64 | dmsetup create xpcow
>> - xm create xp.cfg
>>
>> Back at the desktop, the newly-created files are now gone!
>>
>> I did it this way as a manual test, because I''m veryifying a
few  things:
>> a. Simulating the effect of creating the scratch image in a /tmpfs
>> mount, while the baseline image will be on a physical, read-only 
medium.
>> By removing the mapped device and changing out the scratch  file, I was
>> seeing whether the XP image would indeed revert to the  baseline state,
>> to simulate a reboot in which /tmpfs is cleared.   Supposedly, the
''n''
>> parameter to dmsetup does this anyways, but I  wanted to be doubly
sure.
>>
>> b. Need to scriptify the whole mess, since the baseline image will
>> already be pre-configured, the generation of the scratch image,  setup
of
>> the loop devices, and the device-mapping will be on-the-fly  when some
>> future CD Image I put together boots up.
>>
>> c. Avoids all the problems I see on Google about using blktap with 
qcow
>> with gplpv and such.  I''m probably not grasping some of it,
but  I think
>> those references are aiming for different uses.  So far,  this setup
>> mostly relies wholly on the Linux dom0 to piece  everything together,
>> leaving the WinXP domU utterly oblivious to  what is going on.  And
given
>> Windows'' nature, that''s exactly what we  want.
>>
>> Performance isn''t an issue for me in this, so I''m
sure on a CD, this
>> will be slower than running Oracle on an 8086 (pretend for a moment 
that
>> this is actually possible), but if it works, and WinXP doesn''t
complain,
>> then all is good.
>>
>> Thoughts?
>> ________________________________________
>> From: xen-users-bounces@lists.xensource.com
>> [xen-users-bounces@lists.xensource.com ] On Behalf Of Joshua Kinard
>> [joshua.kinard@sdc-world.com]
>> Sent: Monday, August 17, 2009 9:50 AM
>> To: xen-users@lists.xensource.com
>> Subject: RE: [Xen-users] Bootable LiveDVD w/ Xen that boots Windows
>> Image?
>>
>> Hmm, block-level copy-on-write?  If qcow supports this out-of-the- box,
>> then I technically don''t need any kind of UnionFS?  Just 
someway to
>> specify to Xen a read-only image with a qcow2 image in a  writable
zone?
>>
>> Going to dredge through some EU software patent I dug up from 
Microsoft.
>> Supposed to document the /MININT switch to the NT kernel  that
supposedly
>> makes Windows write all registry changes to volatile  memory instead of
>> to the registry hives.  It''s mostly used in the  WinPE
environment, but
>> it looks like BartPE leverages it too.  Seems  I need more to it,
though,
>> than just modifying boot.ini to pass it,  as once I started to boot
>> Windows from a read-only drive mount  (after discovering Xen/qemu
don''t
>> properly honor the mode bit in the  Xen config for r or r/w, and even
>> file-level permissions), Windows  crashed with UNMOUNTABLE_BOOT_VOLUME
as
>> its error.  But it  highlights the capability is there....just not
easy.
>>
>> USB is out, unfortunately, too.  For security reasons, we ban USB/
>> Firewire drives here, with the exception of CD Burners.  Ditto on 
memory
>> cards (SD, MMC, etc..).  I''ve got very narrow operating 
parameters to
>> work in, which is why this is proving to be quite a  fun challenge.
>>
>> Thanks!,
>>
>> --J
>>
>> ________________________________________
>> From: Fajar A. Nugraha [fajar@fajar.net]
>> Sent: Friday, August 14, 2009 4:53 AM
>> To: Joshua Kinard
>> Cc: xen-users@lists.xensource.com
>> Subject: Re: [Xen-users] Bootable LiveDVD w/ Xen that boots Windows
>> Image?
>>
>> On Fri, Aug 14, 2009 at 1:37 AM, Joshua
>> Kinard<joshua.kinard@sdc-world.com> wrote:
>> > Hmm, I didn''t think of it that way.
>> >
>> > The way I read up on UnionFS and aufs'' functionality, was
they
>> could essentially merge two files virtually,
>>
>> Merge two directories together, to be exact. Not merge files.
>>
>> > i.e., the kernel module would be able to look at the operation
>> coming in and route it to the proper descriptor (i.e., read() --> /
>> livecd/windows/winxp.img OR write() --> /tmp/winxp.img, with /tmp 
being
>> in tmpfs).  I guess it''s not as granular as that it seems.  
Would be a
>> neat trick, but I imagine it''d be complex as anything for  a
kernel
>> module to have to keep track of which files have variants  loaded in
the
>> writeable union area.
>>
>> What you describe is essentially block-level copy-on-write, which is
>> what qcow or zfs does. Aufs/unionfs does this per file.
>>
>> > As for the application, it''s a complex network security
scanner,
>> made by eEye Digital Security, called "Retina".  We just
don''t want  to
>> setup and baby sit Windows installations on our Unix networks  strictly
>> for this one app, so I figured if I can get it to run off  of a CD, we
>> can just park some diskless hardware in a closet and  pull it out
>> whenever we need to do network testing and such.
>>
>> If it were me I''d simply setup a Windows domU on any server
with
>> enough disk and RAM, make a "template" of the
"good" installation
>> (preferably zfs or LVM snapshot), start it only when it''s
needed, shut
>> it down afterwards, and (if necessary) rollback to the "good"
>> template. That is assuming that all host/network tested reachable from
>> my datacenter (either with vlans or routing). No need to add the
>> complexity of a live DVD/USB.
>>
>> If portability is a requirement, and you''re absolutely sure
you''ll
>> always have VT-capable host handy, then using live USB is much
>> perefered than DVD due to performance and complexity reasons. But hey,
>> that''s just me :D
>>
>> Let us know if you find a solution that works.
>>
>> --
>> Fajar
>> _______________________________________________
>> Xen-users mailing list
>> Xen-users@lists.xensource.com
>> http://lists.xensource.com/xen-users
>> _______________________________________________
>> Xen-users mailing list
>> Xen-users@lists.xensource.com
>> http://lists.xensource.com/xen-users
>>
>> _______________________________________________
>> Xen-users mailing list
>> Xen-users@lists.xensource.com
>> http://lists.xensource.com/xen-users
>
>
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xensource.com
> http://lists.xensource.com/xen-users
>
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Yeah, I''ve looked at the GPLPV stuff, and it looks like it''d
expose Windows to the real hardware.  While that has big performance benefits,
since this would be on a CD that could visit multiple machine configurations,
leaving the driver detection to the underlying Linux distro, and virtualizing
stuff for the Windows guest will be the more preferred route (because
don''t we all love Windows'' brain-dead driver detection,
especially in XP?).

I did try taking a look at BartPE initially, but making a plugin for the PE
environment proved futile, as there are just way too many registry entries
created by the scanning program upon install.  WinPE was another option (and
it''s free now as well), but it won''t run .NET code at all, so
that idea was ruled out.  Wine got nowhere, and VMWare/VirtualBox have too many
license restrictions.  Although the end result will be an internal tool, some of
the knowledge I''m piecing together for this will wind up in the
community (mostly the device-mapper concepts I imagine), but we still want to be
as legit as possible internally.  It''s just good practice :)

SquashFS takes care of the RAMDisk problem.  A 7.1GB WinXP disk image,
internally consumes 2.7GB after all the XP patches are applied (per MBSA 2.1),
and my network scanner program installed with its latest audit updates.  But
that gets compressed down to 1.3GB with SquashFS 3.1.  I might try to kludge 4.x
onto my Debian setup for their performance/compression enhancements, but
we''ll see.  On a 4.7GB DVD, that still leaves me quite a bit of room
for the Linux root, and being Debian based, that will be really small,
especially with its own SquashFS glue and the appropriate boot scripts in the
initramfs.

The plus is I''ve learned a ton about Xen and virtualization, previously
having only been a VMWare guy.  So I''ve got other ideas
that''ll bubble up in my head on how to leverage all of this down the
road, too!

Cheers,

--J

________________________________________
From: John Lewis [jlewis@concentric.com]
Sent: Wednesday, August 19, 2009 6:46 PM
To: Joshua Kinard
Subject: Re: [Xen-users] Bootable LiveDVD w/ Xen that boots Windows Image?

Thanks for the info! Sounds like you have an interesting solution in
the works to your challenge. Its a shame that microsoft doesn''t have a
live dvd distro for this type of need. I would suggest putting the
entire sparse image of windows into sort sort of ramdisk. I am more of
a domu/windows/pv driver guy so I am not sure how you would go about
doing this with the live distro you are choosing.

Good luck, sounds like an interesting project. I am looking forward to
hearing what the final resolution is (if it isn''t in the list already).

Lewis


On Aug 19, 2009, at 11:37 AM, Joshua Kinard wrote:
> Yup, thumb drives, SD cards, MMC, etc, aren''t an option where
I''m
> at.  I maintain several offline networks for testing software, and
> they all run various UNIXes, but no Windows setup.  Higher level
> management wants us to run vuneralbility scans on these networks
> using a Windows-based product (which I referenced earlier), but
> telling them "We don''t use Windows" won''t work.
So I was looking
> for a way to scan the networks without maintaining a Windows install
> on each one specifically for this one program.
>
> Hence, booting Windows off of a LiveCD pre-loaded with everything I
> need (and fully licensed, because we have a site license).  Unique
> problem, unique solution....just needs some more glue to put it all
> together.  I can update the one Windows install as-needed, load
> newer audit information, burn it to CD, and run my scans.
>
> Hope that clears things up!
>
> --J
> ________________________________________
> From: John Lewis [jlewis@concentric.com]
> Sent: Wednesday, August 19, 2009 1:24 PM
> To: Joshua Kinard
> Cc: xen-users@lists.xensource.com
> Subject: Re: [Xen-users] Bootable LiveDVD w/ Xen that boots Windows
> Image?
>
> Two questions, probably the same answer. I am just trying to
> understand why you are trying to do this, and what practical use this
> has. Maybe I will have a use for this in the future.
>
> What is the goal in doing this?
> Are usb thumb drives not an option?
>
> Lewis
>
> On Aug 19, 2009, at 7:56 AM, Joshua Kinard wrote:
>
>> Hmm, I don''t think qcow format is going to work with that
method.
>> At least when I tried it, the bochs bios didn''t like what it
was
>> looking at.  But it''s hard to tell if that was bochs, or ntldr
>> giving me a fit.
>>
>> I do know that RAW images work fine, though, and I just tested it
>> with squashfs by running mksquashfs on the base image, and mounting
>> that via loop2 to my base/ subfolder.  Then re-created the scratch
>> area, ran dmsetup, and booted.  Loaded up great, did some changes,
>> shutdown, unloaded the mapper device, unloaded the scratch loop,
>> unmounted my tmpfs, and re-created that segment again, and Windows
>> came back in pristine condition.  So that pretty much means my next
>> battle is Debian''s package manager not liking my local
repository (I
>> can''t connect my Xen system to the internet, so I have to use
such a
>> setup).  As that''s how the Xen LiveCD scripts are doing
things, and
>> assuming everything can be fetched online.
>>
>> Some things I have noticed, though:
>> - After detaching a loopback device, there seems to be a delay in
>> when the kernel actually notices that it''s detached.  If one
does
>> losetup -d <loop> followed by losetup <loop> <new
img>, it seems
>> like in a few cases, the kernel still thought it was looking at the
>> older loop attachment.  I ran into this when using loopback to look
>> at the disk at the disk level, and then passing -o 32256 to losetup
>> to look strictly at the first/primary NTFS partition (while trying
>> to clone it to a smaller disk image, which I finally gave up on
>> doing).
>>
>> - Haven''t found the cause just yet, but dmsetup sometimes
doesn''t
>> like looking at loopback devices, and possibly per the first item,
>> refuses to create the mapped device, citing a resource is busy.
>> Reboot fixes this, and I suspect both of these are caused by all the
>> experimenting I''m doing with switching between the various
>> loopbacks, I probably confuse the kernel.  From a scripted
>> standpoint, it should be a non-issue, though.
>>
>> - Only tested this entire idea/setup against Windows XP.  It seems
>> to be fine, and that generally means any other OS as a domU should
>> work fine as well (because NTFS is by far the pickiest, most
>> temperamental FS known).  You get way more options with Linux/Other
>> UNIXes as far as tricking them into thinking they''re in a r/w
world,
>> but this can be extended to encapsulate quite a few domU''s.
>> However, the only limitation I can see is the number of loopback
>> interfaces.  I use three to setup the Windows domU: /dev/loop2 for
>> the squashfs mount, /dev/loop0 for the windows image inside the
>> squashfs mount, and /dev/loop1 for the scratch area.  I think more
>> loop devices can be easily created, but I''ve not experimented
that
>> far just yet...
>>
>> That''s all for now...I''ve got to install my network
scanner into
>> this Windows image and re-clean it and run these tests all over
>> again.
>>
>> Cheers!
>>
>> --J
>> From: Thiago Camargo Martins Cordeiro [thiagocmartinsc@gmail.com]
>> Sent: Tuesday, August 18, 2009 11:29 PM
>> To: Joshua Kinard
>> Cc: xen-users@lists.xensource.com
>> Subject: Re: [Xen-users] Bootable LiveDVD w/ Xen that boots Windows
>> Image?
>>
>> This procedure will be in the next release of the Xen Live CD!!
>>
>> qcow2 and LVM CoW snapshot for saving storage space while providing
>> fast domU creation from RW snapshots.... good for Eucalyptus, I
>> guess!
>>
>> 2009/8/18 Joshua Kinard <joshua.kinard@sdc-world.com>
>> Think I cracked it.  Don''t even need to deal with
qemu''s qcow stuff
>> (so far).  Still more tests pending (including using read-only drive
>> mounts).  But assume the following steps in a dom0:
>>
>> - mkdir /windows
>> - cd /windows
>> - dd if=/dev/zero of=xp.img bs=1024k count=1 seek=7168
>> - touch xp.cfg
>> - nano xp.cfg
>> - <edit xp.cfg, setup as desired>
>> - xm create xp.cfg
>> - <install WinXP, patch, delete garbage, defrag, zero-out free
>> space, then shutdown>
>> - xm list (verify domU is actually dead)
>>
>> Now after this, xp.img will contain a baseline WinXP install.  In my
>> specific case, I compacted everything down to ~2GB total installed
>> (inside the domU that is) by using straight qemu-emulated devices
>> (not the GPLPV stuff), no page file or hibernation, stripping nearly
>> every misc utility, and a bunch of other stuff, but NO NTFS
>> compression (for now, at least).
>>
>> Next:
>>
>> - dd if=/dev/zero of=xp_scratch.img bs=1024k count=1 seek=2048
>> - losetup /dev/loop0 xp.img
>> - losetup /dev/loop1 xp_scratch.img
>> - echo 0 $(blockdev --getsize /dev/loop0) snapshot /dev/loop0 /dev/
>> loop1 n 64 | dmsetup create xpcow
>>
>> This maps our baseline WinXP image (xp.img) to /dev/loop0 and a
>> scratch area to /dev/loop1 (I don''t even know if this is
needed just
>> yet).  Needed because dmsetup only works with block devices.  Using
>> dmsetup, a snapshot is created, so that all writes to loop0 get
>> redirected to loop1, and creates a device, /dev/mapper/xpcow.
>>
>> Edit xp.cfg to use phy:/dev/mapper/xpcow,hda,w, and boot Windows:
>> - xm create xp.cfg
>>
>> In Windows, create a dummy file on the desktop or three, then
>> shutdown.  Re-start the domU and back at the desktop, we see that
>> the newly-created files still exist, so shutdown again.  Now:
>>
>> - dmsetup remove /dev/mapper/xpcow
>> - losetup -d /dev/loop1
>> - dd if=/dev/zero of=xp_scratch2.img bs=1024k count=1 seek=2048
>> - losetup /dev/loop1 xp_scratch2.img
>> - echo 0 $(blockdev --getsize /dev/loop0) snapshot /dev/loop0 /dev/
>> loop1 n 64 | dmsetup create xpcow
>> - xm create xp.cfg
>>
>> Back at the desktop, the newly-created files are now gone!
>>
>> I did it this way as a manual test, because I''m veryifying a
few
>> things:
>> a. Simulating the effect of creating the scratch image in a /tmpfs
>> mount, while the baseline image will be on a physical, read-only
>> medium.  By removing the mapped device and changing out the scratch
>> file, I was seeing whether the XP image would indeed revert to the
>> baseline state, to simulate a reboot in which /tmpfs is cleared.
>> Supposedly, the ''n'' parameter to dmsetup does this
anyways, but I
>> wanted to be doubly sure.
>>
>> b. Need to scriptify the whole mess, since the baseline image will
>> already be pre-configured, the generation of the scratch image,
>> setup of the loop devices, and the device-mapping will be on-the-fly
>> when some future CD Image I put together boots up.
>>
>> c. Avoids all the problems I see on Google about using blktap with
>> qcow with gplpv and such.  I''m probably not grasping some of
it, but
>> I think those references are aiming for different uses.  So far,
>> this setup mostly relies wholly on the Linux dom0 to piece
>> everything together, leaving the WinXP domU utterly oblivious to
>> what is going on.  And given Windows'' nature, that''s
exactly what we
>> want.
>>
>> Performance isn''t an issue for me in this, so I''m
sure on a CD, this
>> will be slower than running Oracle on an 8086 (pretend for a moment
>> that this is actually possible), but if it works, and WinXP
doesn''t
>> complain, then all is good.
>>
>> Thoughts?
>> ________________________________________
>> From: xen-users-bounces@lists.xensource.com
[xen-users-bounces@lists.xensource.com
>> ] On Behalf Of Joshua Kinard [joshua.kinard@sdc-world.com]
>> Sent: Monday, August 17, 2009 9:50 AM
>> To: xen-users@lists.xensource.com
>> Subject: RE: [Xen-users] Bootable LiveDVD w/ Xen that boots Windows
>> Image?
>>
>> Hmm, block-level copy-on-write?  If qcow supports this out-of-the-
>> box, then I technically don''t need any kind of UnionFS?  Just
>> someway to specify to Xen a read-only image with a qcow2 image in a
>> writable zone?
>>
>> Going to dredge through some EU software patent I dug up from
>> Microsoft.  Supposed to document the /MININT switch to the NT kernel
>> that supposedly makes Windows write all registry changes to volatile
>> memory instead of to the registry hives.  It''s mostly used in
the
>> WinPE environment, but it looks like BartPE leverages it too.  Seems
>> I need more to it, though, than just modifying boot.ini to pass it,
>> as once I started to boot Windows from a read-only drive mount
>> (after discovering Xen/qemu don''t properly honor the mode bit
in the
>> Xen config for r or r/w, and even file-level permissions), Windows
>> crashed with UNMOUNTABLE_BOOT_VOLUME as its error.  But it
>> highlights the capability is there....just not easy.
>>
>> USB is out, unfortunately, too.  For security reasons, we ban USB/
>> Firewire drives here, with the exception of CD Burners.  Ditto on
>> memory cards (SD, MMC, etc..).  I''ve got very narrow operating
>> parameters to work in, which is why this is proving to be quite a
>> fun challenge.
>>
>> Thanks!,
>>
>> --J
>>
>> ________________________________________
>> From: Fajar A. Nugraha [fajar@fajar.net]
>> Sent: Friday, August 14, 2009 4:53 AM
>> To: Joshua Kinard
>> Cc: xen-users@lists.xensource.com
>> Subject: Re: [Xen-users] Bootable LiveDVD w/ Xen that boots Windows
>> Image?
>>
>> On Fri, Aug 14, 2009 at 1:37 AM, Joshua
>> Kinard<joshua.kinard@sdc-world.com> wrote:
>>> Hmm, I didn''t think of it that way.
>>>
>>> The way I read up on UnionFS and aufs'' functionality, was
they
>> could essentially merge two files virtually,
>>
>> Merge two directories together, to be exact. Not merge files.
>>
>>> i.e., the kernel module would be able to look at the operation
>> coming in and route it to the proper descriptor (i.e., read() --> /
>> livecd/windows/winxp.img OR write() --> /tmp/winxp.img, with /tmp
>> being in tmpfs).  I guess it''s not as granular as that it
seems.
>> Would be a neat trick, but I imagine it''d be complex as
anything for
>> a kernel module to have to keep track of which files have variants
>> loaded in the writeable union area.
>>
>> What you describe is essentially block-level copy-on-write, which is
>> what qcow or zfs does. Aufs/unionfs does this per file.
>>
>>> As for the application, it''s a complex network security
scanner,
>> made by eEye Digital Security, called "Retina".  We just
don''t want
>> to setup and baby sit Windows installations on our Unix networks
>> strictly for this one app, so I figured if I can get it to run off
>> of a CD, we can just park some diskless hardware in a closet and
>> pull it out whenever we need to do network testing and such.
>>
>> If it were me I''d simply setup a Windows domU on any server
with
>> enough disk and RAM, make a "template" of the
"good" installation
>> (preferably zfs or LVM snapshot), start it only when it''s
needed,
>> shut
>> it down afterwards, and (if necessary) rollback to the "good"
>> template. That is assuming that all host/network tested reachable
>> from
>> my datacenter (either with vlans or routing). No need to add the
>> complexity of a live DVD/USB.
>>
>> If portability is a requirement, and you''re absolutely sure
you''ll
>> always have VT-capable host handy, then using live USB is much
>> perefered than DVD due to performance and complexity reasons. But
>> hey,
>> that''s just me :D
>>
>> Let us know if you find a solution that works.
>>
>> --
>> Fajar
>> _______________________________________________
>> Xen-users mailing list
>> Xen-users@lists.xensource.com
>> http://lists.xensource.com/xen-users
>> _______________________________________________
>> Xen-users mailing list
>> Xen-users@lists.xensource.com
>> http://lists.xensource.com/xen-users
>>
>> _______________________________________________
>> Xen-users mailing list
>> Xen-users@lists.xensource.com
>> http://lists.xensource.com/xen-users
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

please excuse it.


I responded to the wrong message.

----- Original Message ----- 
From: "Joshua Kinard" <joshua.kinard@sdc-world.com>
To: <xen-users@lists.xensource.com>
Sent: Thursday, August 20, 2009 9:31 AM
Subject: RE: [Xen-users] Bootable LiveDVD w/ Xen that boots Windows Image?


Pretty familiar with kernel building, actually (done a bit of work on SGI 
MIPS platforms running Linux).  If you''re refering to my Debian issue, 
that''s more just figuring out how to get their package manager to
"trust" my
local repository so I can tweak the Xen LiveCD building scripts to pull 
packages locally to build stuff.  Not too much of a hassle, just a matter of 
some tinkering around.

Thanks though!,

--J

________________________________________
From: Robbie garrett [rgarrett@hostourweb.com]
Sent: Thursday, August 20, 2009 8:07 AM
To: John Lewis; Joshua Kinard
Cc: xen-users@lists.xensource.com
Subject: Re: [Xen-users] Bootable LiveDVD w/ Xen that boots Windows Image?

try a make KERNCONF=XEN menuconfig

then you should see some xen options under the CPU option.

then isssue a make KERNCONF=XEN to make your kernel.

----- Original Message -----
From: "John Lewis" <jlewis@concentric.com>
To: "Joshua Kinard" <joshua.kinard@sdc-world.com>
Cc: <xen-users@lists.xensource.com>
Sent: Wednesday, August 19, 2009 1:24 PM
Subject: Re: [Xen-users] Bootable LiveDVD w/ Xen that boots Windows Image?

> Two questions, probably the same answer. I am just trying to  understand
> why you are trying to do this, and what practical use this  has. Maybe I
> will have a use for this in the future.
>
> What is the goal in doing this?
> Are usb thumb drives not an option?
>
> Lewis
>
> On Aug 19, 2009, at 7:56 AM, Joshua Kinard wrote:
>
>> Hmm, I don''t think qcow format is going to work with that
method.   At
>> least when I tried it, the bochs bios didn''t like what it was 
looking
>> at.  But it''s hard to tell if that was bochs, or ntldr  giving
me a fit.
>>
>> I do know that RAW images work fine, though, and I just tested it  with
>> squashfs by running mksquashfs on the base image, and mounting  that
via
>> loop2 to my base/ subfolder.  Then re-created the scratch  area, ran
>> dmsetup, and booted.  Loaded up great, did some changes,  shutdown,
>> unloaded the mapper device, unloaded the scratch loop,  unmounted my
>> tmpfs, and re-created that segment again, and Windows  came back in
>> pristine condition.  So that pretty much means my next  battle is
>> Debian''s package manager not liking my local repository (I 
can''t connect
>> my Xen system to the internet, so I have to use such a  setup).  As
>> that''s how the Xen LiveCD scripts are doing things, and 
assuming
>> everything can be fetched online.
>>
>> Some things I have noticed, though:
>> - After detaching a loopback device, there seems to be a delay in  when
>> the kernel actually notices that it''s detached.  If one does 
losetup -d
>> <loop> followed by losetup <loop> <new img>, it seems
like in a few
>> cases, the kernel still thought it was looking at the  older loop
>> attachment.  I ran into this when using loopback to look  at the disk
at
>> the disk level, and then passing -o 32256 to losetup  to look strictly
at
>> the first/primary NTFS partition (while trying  to clone it to a
smaller
>> disk image, which I finally gave up on  doing).
>>
>> - Haven''t found the cause just yet, but dmsetup sometimes
doesn''t  like
>> looking at loopback devices, and possibly per the first item,  refuses
to
>> create the mapped device, citing a resource is busy.   Reboot fixes
this,
>> and I suspect both of these are caused by all the  experimenting
I''m
>> doing with switching between the various  loopbacks, I probably confuse
>> the kernel.  From a scripted  standpoint, it should be a non-issue,
>> though.
>>
>> - Only tested this entire idea/setup against Windows XP.  It seems  to
be
>> fine, and that generally means any other OS as a domU should  work fine
>> as well (because NTFS is by far the pickiest, most  temperamental FS
>> known).  You get way more options with Linux/Other  UNIXes as far as
>> tricking them into thinking they''re in a r/w world,  but this
can be
>> extended to encapsulate quite a few domU''s.   However, the
only
>> limitation I can see is the number of loopback  interfaces.  I use
three
>> to setup the Windows domU: /dev/loop2 for  the squashfs mount,
/dev/loop0
>> for the windows image inside the  squashfs mount, and /dev/loop1 for
the
>> scratch area.  I think more  loop devices can be easily created, but
I''ve
>> not experimented that  far just yet...
>>
>> That''s all for now...I''ve got to install my network
scanner into  this
>> Windows image and re-clean it and run these tests all over again.
>>
>> Cheers!
>>
>> --J
>> From: Thiago Camargo Martins Cordeiro [thiagocmartinsc@gmail.com]
>> Sent: Tuesday, August 18, 2009 11:29 PM
>> To: Joshua Kinard
>> Cc: xen-users@lists.xensource.com
>> Subject: Re: [Xen-users] Bootable LiveDVD w/ Xen that boots Windows
>> Image?
>>
>> This procedure will be in the next release of the Xen Live CD!!
>>
>> qcow2 and LVM CoW snapshot for saving storage space while providing 
fast
>> domU creation from RW snapshots.... good for Eucalyptus, I guess!
>>
>> 2009/8/18 Joshua Kinard <joshua.kinard@sdc-world.com>
>> Think I cracked it.  Don''t even need to deal with
qemu''s qcow stuff  (so
>> far).  Still more tests pending (including using read-only drive
>> mounts).  But assume the following steps in a dom0:
>>
>> - mkdir /windows
>> - cd /windows
>> - dd if=/dev/zero of=xp.img bs=1024k count=1 seek=7168
>> - touch xp.cfg
>> - nano xp.cfg
>> - <edit xp.cfg, setup as desired>
>> - xm create xp.cfg
>> - <install WinXP, patch, delete garbage, defrag, zero-out free 
space,
>> then shutdown>
>> - xm list (verify domU is actually dead)
>>
>> Now after this, xp.img will contain a baseline WinXP install.  In my
>> specific case, I compacted everything down to ~2GB total installed
>> (inside the domU that is) by using straight qemu-emulated devices  (not
>> the GPLPV stuff), no page file or hibernation, stripping nearly  every
>> misc utility, and a bunch of other stuff, but NO NTFS  compression (for
>> now, at least).
>>
>> Next:
>>
>> - dd if=/dev/zero of=xp_scratch.img bs=1024k count=1 seek=2048
>> - losetup /dev/loop0 xp.img
>> - losetup /dev/loop1 xp_scratch.img
>> - echo 0 $(blockdev --getsize /dev/loop0) snapshot /dev/loop0 /dev/
loop1
>> n 64 | dmsetup create xpcow
>>
>> This maps our baseline WinXP image (xp.img) to /dev/loop0 and a 
scratch
>> area to /dev/loop1 (I don''t even know if this is needed just 
yet).
>> Needed because dmsetup only works with block devices.  Using  dmsetup,
a
>> snapshot is created, so that all writes to loop0 get  redirected to
>> loop1, and creates a device, /dev/mapper/xpcow.
>>
>> Edit xp.cfg to use phy:/dev/mapper/xpcow,hda,w, and boot Windows:
>> - xm create xp.cfg
>>
>> In Windows, create a dummy file on the desktop or three, then 
shutdown.
>> Re-start the domU and back at the desktop, we see that  the
newly-created
>> files still exist, so shutdown again.  Now:
>>
>> - dmsetup remove /dev/mapper/xpcow
>> - losetup -d /dev/loop1
>> - dd if=/dev/zero of=xp_scratch2.img bs=1024k count=1 seek=2048
>> - losetup /dev/loop1 xp_scratch2.img
>> - echo 0 $(blockdev --getsize /dev/loop0) snapshot /dev/loop0 /dev/
loop1
>> n 64 | dmsetup create xpcow
>> - xm create xp.cfg
>>
>> Back at the desktop, the newly-created files are now gone!
>>
>> I did it this way as a manual test, because I''m veryifying a
few  things:
>> a. Simulating the effect of creating the scratch image in a /tmpfs
>> mount, while the baseline image will be on a physical, read-only 
medium.
>> By removing the mapped device and changing out the scratch  file, I was
>> seeing whether the XP image would indeed revert to the  baseline state,
>> to simulate a reboot in which /tmpfs is cleared.   Supposedly, the
''n''
>> parameter to dmsetup does this anyways, but I  wanted to be doubly
sure.
>>
>> b. Need to scriptify the whole mess, since the baseline image will
>> already be pre-configured, the generation of the scratch image,  setup
of
>> the loop devices, and the device-mapping will be on-the-fly  when some
>> future CD Image I put together boots up.
>>
>> c. Avoids all the problems I see on Google about using blktap with 
qcow
>> with gplpv and such.  I''m probably not grasping some of it,
but  I think
>> those references are aiming for different uses.  So far,  this setup
>> mostly relies wholly on the Linux dom0 to piece  everything together,
>> leaving the WinXP domU utterly oblivious to  what is going on.  And
given
>> Windows'' nature, that''s exactly what we  want.
>>
>> Performance isn''t an issue for me in this, so I''m
sure on a CD, this
>> will be slower than running Oracle on an 8086 (pretend for a moment 
that
>> this is actually possible), but if it works, and WinXP doesn''t
complain,
>> then all is good.
>>
>> Thoughts?
>> ________________________________________
>> From: xen-users-bounces@lists.xensource.com
>> [xen-users-bounces@lists.xensource.com ] On Behalf Of Joshua Kinard
>> [joshua.kinard@sdc-world.com]
>> Sent: Monday, August 17, 2009 9:50 AM
>> To: xen-users@lists.xensource.com
>> Subject: RE: [Xen-users] Bootable LiveDVD w/ Xen that boots Windows
>> Image?
>>
>> Hmm, block-level copy-on-write?  If qcow supports this out-of-the- box,
>> then I technically don''t need any kind of UnionFS?  Just 
someway to
>> specify to Xen a read-only image with a qcow2 image in a  writable
zone?
>>
>> Going to dredge through some EU software patent I dug up from 
Microsoft.
>> Supposed to document the /MININT switch to the NT kernel  that
supposedly
>> makes Windows write all registry changes to volatile  memory instead of
>> to the registry hives.  It''s mostly used in the  WinPE
environment, but
>> it looks like BartPE leverages it too.  Seems  I need more to it,
though,
>> than just modifying boot.ini to pass it,  as once I started to boot
>> Windows from a read-only drive mount  (after discovering Xen/qemu
don''t
>> properly honor the mode bit in the  Xen config for r or r/w, and even
>> file-level permissions), Windows  crashed with UNMOUNTABLE_BOOT_VOLUME
as
>> its error.  But it  highlights the capability is there....just not
easy.
>>
>> USB is out, unfortunately, too.  For security reasons, we ban USB/
>> Firewire drives here, with the exception of CD Burners.  Ditto on 
memory
>> cards (SD, MMC, etc..).  I''ve got very narrow operating 
parameters to
>> work in, which is why this is proving to be quite a  fun challenge.
>>
>> Thanks!,
>>
>> --J
>>
>> ________________________________________
>> From: Fajar A. Nugraha [fajar@fajar.net]
>> Sent: Friday, August 14, 2009 4:53 AM
>> To: Joshua Kinard
>> Cc: xen-users@lists.xensource.com
>> Subject: Re: [Xen-users] Bootable LiveDVD w/ Xen that boots Windows
>> Image?
>>
>> On Fri, Aug 14, 2009 at 1:37 AM, Joshua
>> Kinard<joshua.kinard@sdc-world.com> wrote:
>> > Hmm, I didn''t think of it that way.
>> >
>> > The way I read up on UnionFS and aufs'' functionality, was
they
>> could essentially merge two files virtually,
>>
>> Merge two directories together, to be exact. Not merge files.
>>
>> > i.e., the kernel module would be able to look at the operation
>> coming in and route it to the proper descriptor (i.e., read() --> /
>> livecd/windows/winxp.img OR write() --> /tmp/winxp.img, with /tmp 
being
>> in tmpfs).  I guess it''s not as granular as that it seems.  
Would be a
>> neat trick, but I imagine it''d be complex as anything for  a
kernel
>> module to have to keep track of which files have variants  loaded in
the
>> writeable union area.
>>
>> What you describe is essentially block-level copy-on-write, which is
>> what qcow or zfs does. Aufs/unionfs does this per file.
>>
>> > As for the application, it''s a complex network security
scanner,
>> made by eEye Digital Security, called "Retina".  We just
don''t want  to
>> setup and baby sit Windows installations on our Unix networks  strictly
>> for this one app, so I figured if I can get it to run off  of a CD, we
>> can just park some diskless hardware in a closet and  pull it out
>> whenever we need to do network testing and such.
>>
>> If it were me I''d simply setup a Windows domU on any server
with
>> enough disk and RAM, make a "template" of the
"good" installation
>> (preferably zfs or LVM snapshot), start it only when it''s
needed, shut
>> it down afterwards, and (if necessary) rollback to the "good"
>> template. That is assuming that all host/network tested reachable from
>> my datacenter (either with vlans or routing). No need to add the
>> complexity of a live DVD/USB.
>>
>> If portability is a requirement, and you''re absolutely sure
you''ll
>> always have VT-capable host handy, then using live USB is much
>> perefered than DVD due to performance and complexity reasons. But hey,
>> that''s just me :D
>>
>> Let us know if you find a solution that works.
>>
>> --
>> Fajar
>> _______________________________________________
>> Xen-users mailing list
>> Xen-users@lists.xensource.com
>> http://lists.xensource.com/xen-users
>> _______________________________________________
>> Xen-users mailing list
>> Xen-users@lists.xensource.com
>> http://lists.xensource.com/xen-users
>>
>> _______________________________________________
>> Xen-users mailing list
>> Xen-users@lists.xensource.com
>> http://lists.xensource.com/xen-users
>
>
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xensource.com
> http://lists.xensource.com/xen-users
>
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

I think this is more of  Debian bug, or a glitch in my apt mirror that
I''ve setup, but has anyone run into a problem with debootstrap bugging
out while attempting to unpack the "zlib1g-udeb" package? 
I''ve tried the xenlivecd-0.8.2 scripts, debootstrap by itself, and
live-magic.  All of them die at this point.

Source of the problem appears to be zlib1g-udeb attempting to overwrite a
library already installed by the standard zlib1g package.  I''ve watched
a libc-udeb variant get by fine, so I''m not sure if this is an issue
specifically with this package, debootstrap itself, or such.  Wanted to see if
anyone else has run into it using the xenlivecd scripts before I moved onto
upstream.

Thanks!,

--J
________________________________________
From: xen-users-bounces@lists.xensource.com
[xen-users-bounces@lists.xensource.com] On Behalf Of Joshua Kinard
[joshua.kinard@sdc-world.com]
Sent: Thursday, August 20, 2009 9:47 AM
To: xen-users@lists.xensource.com
Subject: RE: [Xen-users] Bootable LiveDVD w/ Xen that boots Windows Image?

Yeah, I''ve looked at the GPLPV stuff, and it looks like it''d
expose Windows to the real hardware.  While that has big performance benefits,
since this would be on a CD that could visit multiple machine configurations,
leaving the driver detection to the underlying Linux distro, and virtualizing
stuff for the Windows guest will be the more preferred route (because
don''t we all love Windows'' brain-dead driver detection,
especially in XP?).

I did try taking a look at BartPE initially, but making a plugin for the PE
environment proved futile, as there are just way too many registry entries
created by the scanning program upon install.  WinPE was another option (and
it''s free now as well), but it won''t run .NET code at all, so
that idea was ruled out.  Wine got nowhere, and VMWare/VirtualBox have too many
license restrictions.  Although the end result will be an internal tool, some of
the knowledge I''m piecing together for this will wind up in the
community (mostly the device-mapper concepts I imagine), but we still want to be
as legit as possible internally.  It''s just good practice :)

SquashFS takes care of the RAMDisk problem.  A 7.1GB WinXP disk image,
internally consumes 2.7GB after all the XP patches are applied (per MBSA 2.1),
and my network scanner program installed with its latest audit updates.  But
that gets compressed down to 1.3GB with SquashFS 3.1.  I might try to kludge 4.x
onto my Debian setup for their performance/compression enhancements, but
we''ll see.  On a 4.7GB DVD, that still leaves me quite a bit of room
for the Linux root, and being Debian based, that will be really small,
especially with its own SquashFS glue and the appropriate boot scripts in the
initramfs.

The plus is I''ve learned a ton about Xen and virtualization, previously
having only been a VMWare guy.  So I''ve got other ideas
that''ll bubble up in my head on how to leverage all of this down the
road, too!

Cheers,

--J

________________________________________
From: John Lewis [jlewis@concentric.com]
Sent: Wednesday, August 19, 2009 6:46 PM
To: Joshua Kinard
Subject: Re: [Xen-users] Bootable LiveDVD w/ Xen that boots Windows Image?

Thanks for the info! Sounds like you have an interesting solution in
the works to your challenge. Its a shame that microsoft doesn''t have a
live dvd distro for this type of need. I would suggest putting the
entire sparse image of windows into sort sort of ramdisk. I am more of
a domu/windows/pv driver guy so I am not sure how you would go about
doing this with the live distro you are choosing.

Good luck, sounds like an interesting project. I am looking forward to
hearing what the final resolution is (if it isn''t in the list already).

Lewis


On Aug 19, 2009, at 11:37 AM, Joshua Kinard wrote:
> Yup, thumb drives, SD cards, MMC, etc, aren''t an option where
I''m
> at.  I maintain several offline networks for testing software, and
> they all run various UNIXes, but no Windows setup.  Higher level
> management wants us to run vuneralbility scans on these networks
> using a Windows-based product (which I referenced earlier), but
> telling them "We don''t use Windows" won''t work.
So I was looking
> for a way to scan the networks without maintaining a Windows install
> on each one specifically for this one program.
>
> Hence, booting Windows off of a LiveCD pre-loaded with everything I
> need (and fully licensed, because we have a site license).  Unique
> problem, unique solution....just needs some more glue to put it all
> together.  I can update the one Windows install as-needed, load
> newer audit information, burn it to CD, and run my scans.
>
> Hope that clears things up!
>
> --J
> ________________________________________
> From: John Lewis [jlewis@concentric.com]
> Sent: Wednesday, August 19, 2009 1:24 PM
> To: Joshua Kinard
> Cc: xen-users@lists.xensource.com
> Subject: Re: [Xen-users] Bootable LiveDVD w/ Xen that boots Windows
> Image?
>
> Two questions, probably the same answer. I am just trying to
> understand why you are trying to do this, and what practical use this
> has. Maybe I will have a use for this in the future.
>
> What is the goal in doing this?
> Are usb thumb drives not an option?
>
> Lewis
>
> On Aug 19, 2009, at 7:56 AM, Joshua Kinard wrote:
>
>> Hmm, I don''t think qcow format is going to work with that
method.
>> At least when I tried it, the bochs bios didn''t like what it
was
>> looking at.  But it''s hard to tell if that was bochs, or ntldr
>> giving me a fit.
>>
>> I do know that RAW images work fine, though, and I just tested it
>> with squashfs by running mksquashfs on the base image, and mounting
>> that via loop2 to my base/ subfolder.  Then re-created the scratch
>> area, ran dmsetup, and booted.  Loaded up great, did some changes,
>> shutdown, unloaded the mapper device, unloaded the scratch loop,
>> unmounted my tmpfs, and re-created that segment again, and Windows
>> came back in pristine condition.  So that pretty much means my next
>> battle is Debian''s package manager not liking my local
repository (I
>> can''t connect my Xen system to the internet, so I have to use
such a
>> setup).  As that''s how the Xen LiveCD scripts are doing
things, and
>> assuming everything can be fetched online.
>>
>> Some things I have noticed, though:
>> - After detaching a loopback device, there seems to be a delay in
>> when the kernel actually notices that it''s detached.  If one
does
>> losetup -d <loop> followed by losetup <loop> <new
img>, it seems
>> like in a few cases, the kernel still thought it was looking at the
>> older loop attachment.  I ran into this when using loopback to look
>> at the disk at the disk level, and then passing -o 32256 to losetup
>> to look strictly at the first/primary NTFS partition (while trying
>> to clone it to a smaller disk image, which I finally gave up on
>> doing).
>>
>> - Haven''t found the cause just yet, but dmsetup sometimes
doesn''t
>> like looking at loopback devices, and possibly per the first item,
>> refuses to create the mapped device, citing a resource is busy.
>> Reboot fixes this, and I suspect both of these are caused by all the
>> experimenting I''m doing with switching between the various
>> loopbacks, I probably confuse the kernel.  From a scripted
>> standpoint, it should be a non-issue, though.
>>
>> - Only tested this entire idea/setup against Windows XP.  It seems
>> to be fine, and that generally means any other OS as a domU should
>> work fine as well (because NTFS is by far the pickiest, most
>> temperamental FS known).  You get way more options with Linux/Other
>> UNIXes as far as tricking them into thinking they''re in a r/w
world,
>> but this can be extended to encapsulate quite a few domU''s.
>> However, the only limitation I can see is the number of loopback
>> interfaces.  I use three to setup the Windows domU: /dev/loop2 for
>> the squashfs mount, /dev/loop0 for the windows image inside the
>> squashfs mount, and /dev/loop1 for the scratch area.  I think more
>> loop devices can be easily created, but I''ve not experimented
that
>> far just yet...
>>
>> That''s all for now...I''ve got to install my network
scanner into
>> this Windows image and re-clean it and run these tests all over
>> again.
>>
>> Cheers!
>>
>> --J
>> From: Thiago Camargo Martins Cordeiro [thiagocmartinsc@gmail.com]
>> Sent: Tuesday, August 18, 2009 11:29 PM
>> To: Joshua Kinard
>> Cc: xen-users@lists.xensource.com
>> Subject: Re: [Xen-users] Bootable LiveDVD w/ Xen that boots Windows
>> Image?
>>
>> This procedure will be in the next release of the Xen Live CD!!
>>
>> qcow2 and LVM CoW snapshot for saving storage space while providing
>> fast domU creation from RW snapshots.... good for Eucalyptus, I
>> guess!
>>
>> 2009/8/18 Joshua Kinard <joshua.kinard@sdc-world.com>
>> Think I cracked it.  Don''t even need to deal with
qemu''s qcow stuff
>> (so far).  Still more tests pending (including using read-only drive
>> mounts).  But assume the following steps in a dom0:
>>
>> - mkdir /windows
>> - cd /windows
>> - dd if=/dev/zero of=xp.img bs=1024k count=1 seek=7168
>> - touch xp.cfg
>> - nano xp.cfg
>> - <edit xp.cfg, setup as desired>
>> - xm create xp.cfg
>> - <install WinXP, patch, delete garbage, defrag, zero-out free
>> space, then shutdown>
>> - xm list (verify domU is actually dead)
>>
>> Now after this, xp.img will contain a baseline WinXP install.  In my
>> specific case, I compacted everything down to ~2GB total installed
>> (inside the domU that is) by using straight qemu-emulated devices
>> (not the GPLPV stuff), no page file or hibernation, stripping nearly
>> every misc utility, and a bunch of other stuff, but NO NTFS
>> compression (for now, at least).
>>
>> Next:
>>
>> - dd if=/dev/zero of=xp_scratch.img bs=1024k count=1 seek=2048
>> - losetup /dev/loop0 xp.img
>> - losetup /dev/loop1 xp_scratch.img
>> - echo 0 $(blockdev --getsize /dev/loop0) snapshot /dev/loop0 /dev/
>> loop1 n 64 | dmsetup create xpcow
>>
>> This maps our baseline WinXP image (xp.img) to /dev/loop0 and a
>> scratch area to /dev/loop1 (I don''t even know if this is
needed just
>> yet).  Needed because dmsetup only works with block devices.  Using
>> dmsetup, a snapshot is created, so that all writes to loop0 get
>> redirected to loop1, and creates a device, /dev/mapper/xpcow.
>>
>> Edit xp.cfg to use phy:/dev/mapper/xpcow,hda,w, and boot Windows:
>> - xm create xp.cfg
>>
>> In Windows, create a dummy file on the desktop or three, then
>> shutdown.  Re-start the domU and back at the desktop, we see that
>> the newly-created files still exist, so shutdown again.  Now:
>>
>> - dmsetup remove /dev/mapper/xpcow
>> - losetup -d /dev/loop1
>> - dd if=/dev/zero of=xp_scratch2.img bs=1024k count=1 seek=2048
>> - losetup /dev/loop1 xp_scratch2.img
>> - echo 0 $(blockdev --getsize /dev/loop0) snapshot /dev/loop0 /dev/
>> loop1 n 64 | dmsetup create xpcow
>> - xm create xp.cfg
>>
>> Back at the desktop, the newly-created files are now gone!
>>
>> I did it this way as a manual test, because I''m veryifying a
few
>> things:
>> a. Simulating the effect of creating the scratch image in a /tmpfs
>> mount, while the baseline image will be on a physical, read-only
>> medium.  By removing the mapped device and changing out the scratch
>> file, I was seeing whether the XP image would indeed revert to the
>> baseline state, to simulate a reboot in which /tmpfs is cleared.
>> Supposedly, the ''n'' parameter to dmsetup does this
anyways, but I
>> wanted to be doubly sure.
>>
>> b. Need to scriptify the whole mess, since the baseline image will
>> already be pre-configured, the generation of the scratch image,
>> setup of the loop devices, and the device-mapping will be on-the-fly
>> when some future CD Image I put together boots up.
>>
>> c. Avoids all the problems I see on Google about using blktap with
>> qcow with gplpv and such.  I''m probably not grasping some of
it, but
>> I think those references are aiming for different uses.  So far,
>> this setup mostly relies wholly on the Linux dom0 to piece
>> everything together, leaving the WinXP domU utterly oblivious to
>> what is going on.  And given Windows'' nature, that''s
exactly what we
>> want.
>>
>> Performance isn''t an issue for me in this, so I''m
sure on a CD, this
>> will be slower than running Oracle on an 8086 (pretend for a moment
>> that this is actually possible), but if it works, and WinXP
doesn''t
>> complain, then all is good.
>>
>> Thoughts?
>> ________________________________________
>> From: xen-users-bounces@lists.xensource.com
[xen-users-bounces@lists.xensource.com
>> ] On Behalf Of Joshua Kinard [joshua.kinard@sdc-world.com]
>> Sent: Monday, August 17, 2009 9:50 AM
>> To: xen-users@lists.xensource.com
>> Subject: RE: [Xen-users] Bootable LiveDVD w/ Xen that boots Windows
>> Image?
>>
>> Hmm, block-level copy-on-write?  If qcow supports this out-of-the-
>> box, then I technically don''t need any kind of UnionFS?  Just
>> someway to specify to Xen a read-only image with a qcow2 image in a
>> writable zone?
>>
>> Going to dredge through some EU software patent I dug up from
>> Microsoft.  Supposed to document the /MININT switch to the NT kernel
>> that supposedly makes Windows write all registry changes to volatile
>> memory instead of to the registry hives.  It''s mostly used in
the
>> WinPE environment, but it looks like BartPE leverages it too.  Seems
>> I need more to it, though, than just modifying boot.ini to pass it,
>> as once I started to boot Windows from a read-only drive mount
>> (after discovering Xen/qemu don''t properly honor the mode bit
in the
>> Xen config for r or r/w, and even file-level permissions), Windows
>> crashed with UNMOUNTABLE_BOOT_VOLUME as its error.  But it
>> highlights the capability is there....just not easy.
>>
>> USB is out, unfortunately, too.  For security reasons, we ban USB/
>> Firewire drives here, with the exception of CD Burners.  Ditto on
>> memory cards (SD, MMC, etc..).  I''ve got very narrow operating
>> parameters to work in, which is why this is proving to be quite a
>> fun challenge.
>>
>> Thanks!,
>>
>> --J
>>
>> ________________________________________
>> From: Fajar A. Nugraha [fajar@fajar.net]
>> Sent: Friday, August 14, 2009 4:53 AM
>> To: Joshua Kinard
>> Cc: xen-users@lists.xensource.com
>> Subject: Re: [Xen-users] Bootable LiveDVD w/ Xen that boots Windows
>> Image?
>>
>> On Fri, Aug 14, 2009 at 1:37 AM, Joshua
>> Kinard<joshua.kinard@sdc-world.com> wrote:
>>> Hmm, I didn''t think of it that way.
>>>
>>> The way I read up on UnionFS and aufs'' functionality, was
they
>> could essentially merge two files virtually,
>>
>> Merge two directories together, to be exact. Not merge files.
>>
>>> i.e., the kernel module would be able to look at the operation
>> coming in and route it to the proper descriptor (i.e., read() --> /
>> livecd/windows/winxp.img OR write() --> /tmp/winxp.img, with /tmp
>> being in tmpfs).  I guess it''s not as granular as that it
seems.
>> Would be a neat trick, but I imagine it''d be complex as
anything for
>> a kernel module to have to keep track of which files have variants
>> loaded in the writeable union area.
>>
>> What you describe is essentially block-level copy-on-write, which is
>> what qcow or zfs does. Aufs/unionfs does this per file.
>>
>>> As for the application, it''s a complex network security
scanner,
>> made by eEye Digital Security, called "Retina".  We just
don''t want
>> to setup and baby sit Windows installations on our Unix networks
>> strictly for this one app, so I figured if I can get it to run off
>> of a CD, we can just park some diskless hardware in a closet and
>> pull it out whenever we need to do network testing and such.
>>
>> If it were me I''d simply setup a Windows domU on any server
with
>> enough disk and RAM, make a "template" of the
"good" installation
>> (preferably zfs or LVM snapshot), start it only when it''s
needed,
>> shut
>> it down afterwards, and (if necessary) rollback to the "good"
>> template. That is assuming that all host/network tested reachable
>> from
>> my datacenter (either with vlans or routing). No need to add the
>> complexity of a live DVD/USB.
>>
>> If portability is a requirement, and you''re absolutely sure
you''ll
>> always have VT-capable host handy, then using live USB is much
>> perefered than DVD due to performance and complexity reasons. But
>> hey,
>> that''s just me :D
>>
>> Let us know if you find a solution that works.
>>
>> --
>> Fajar
>> _______________________________________________
>> Xen-users mailing list
>> Xen-users@lists.xensource.com
>> http://lists.xensource.com/xen-users
>> _______________________________________________
>> Xen-users mailing list
>> Xen-users@lists.xensource.com
>> http://lists.xensource.com/xen-users
>>
>> _______________________________________________
>> Xen-users mailing list
>> Xen-users@lists.xensource.com
>> http://lists.xensource.com/xen-users
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Regarding your comment on GPLPV, it does not expose any real
hardware to the HVM domU; it exposes and provides drivers for the PV devices
that are always used by PV domains (instead of the fully virtualized devices
typically used by HVM domains).  I don''t believe the PV devices change
when
the hardware on a machine changes (just the fully virtualized devices
don''t
change), so the GPLPV drivers should provide performance enhancement without
adding any new potential (non-driver related) issues once they are working
on a LiveDVD (because nothing should change in terms of what your guest is
seeing once you have everything on RO storage).
	Dustin

-----Original Message-----
From: xen-users-bounces@lists.xensource.com
[mailto:xen-users-bounces@lists.xensource.com] On Behalf Of Joshua Kinard
Sent: Thursday, August 20, 2009 09:48
To: xen-users@lists.xensource.com
Subject: RE: [Xen-users] Bootable LiveDVD w/ Xen that boots Windows Image?

Yeah, I''ve looked at the GPLPV stuff, and it looks like it''d
expose Windows
to the real hardware.  While that has big performance benefits, since this
would be on a CD that could visit multiple machine configurations, leaving
the driver detection to the underlying Linux distro, and virtualizing stuff
for the Windows guest will be the more preferred route (because don''t
we all
love Windows'' brain-dead driver detection, especially in XP?).

I did try taking a look at BartPE initially, but making a plugin for the PE
environment proved futile, as there are just way too many registry entries
created by the scanning program upon install.  WinPE was another option (and
it''s free now as well), but it won''t run .NET code at all, so
that idea was
ruled out.  Wine got nowhere, and VMWare/VirtualBox have too many license
restrictions.  Although the end result will be an internal tool, some of the
knowledge I''m piecing together for this will wind up in the community
(mostly the device-mapper concepts I imagine), but we still want to be as
legit as possible internally.  It''s just good practice :)

SquashFS takes care of the RAMDisk problem.  A 7.1GB WinXP disk image,
internally consumes 2.7GB after all the XP patches are applied (per MBSA
2.1), and my network scanner program installed with its latest audit
updates.  But that gets compressed down to 1.3GB with SquashFS 3.1.  I might
try to kludge 4.x onto my Debian setup for their performance/compression
enhancements, but we''ll see.  On a 4.7GB DVD, that still leaves me
quite a
bit of room for the Linux root, and being Debian based, that will be really
small, especially with its own SquashFS glue and the appropriate boot
scripts in the initramfs.

The plus is I''ve learned a ton about Xen and virtualization, previously
having only been a VMWare guy.  So I''ve got other ideas
that''ll bubble up in
my head on how to leverage all of this down the road, too!

Cheers,

--J

________________________________________
From: John Lewis [jlewis@concentric.com]
Sent: Wednesday, August 19, 2009 6:46 PM
To: Joshua Kinard
Subject: Re: [Xen-users] Bootable LiveDVD w/ Xen that boots Windows Image?

Thanks for the info! Sounds like you have an interesting solution in
the works to your challenge. Its a shame that microsoft doesn''t have a
live dvd distro for this type of need. I would suggest putting the
entire sparse image of windows into sort sort of ramdisk. I am more of
a domu/windows/pv driver guy so I am not sure how you would go about
doing this with the live distro you are choosing.

Good luck, sounds like an interesting project. I am looking forward to
hearing what the final resolution is (if it isn''t in the list already).

Lewis


On Aug 19, 2009, at 11:37 AM, Joshua Kinard wrote:
> Yup, thumb drives, SD cards, MMC, etc, aren''t an option where
I''m
> at.  I maintain several offline networks for testing software, and
> they all run various UNIXes, but no Windows setup.  Higher level
> management wants us to run vuneralbility scans on these networks
> using a Windows-based product (which I referenced earlier), but
> telling them "We don''t use Windows" won''t work.
So I was looking
> for a way to scan the networks without maintaining a Windows install
> on each one specifically for this one program.
>
> Hence, booting Windows off of a LiveCD pre-loaded with everything I
> need (and fully licensed, because we have a site license).  Unique
> problem, unique solution....just needs some more glue to put it all
> together.  I can update the one Windows install as-needed, load
> newer audit information, burn it to CD, and run my scans.
>
> Hope that clears things up!
>
> --J
> ________________________________________
> From: John Lewis [jlewis@concentric.com]
> Sent: Wednesday, August 19, 2009 1:24 PM
> To: Joshua Kinard
> Cc: xen-users@lists.xensource.com
> Subject: Re: [Xen-users] Bootable LiveDVD w/ Xen that boots Windows
> Image?
>
> Two questions, probably the same answer. I am just trying to
> understand why you are trying to do this, and what practical use this
> has. Maybe I will have a use for this in the future.
>
> What is the goal in doing this?
> Are usb thumb drives not an option?
>
> Lewis
>
> On Aug 19, 2009, at 7:56 AM, Joshua Kinard wrote:
>
>> Hmm, I don''t think qcow format is going to work with that
method.
>> At least when I tried it, the bochs bios didn''t like what it
was
>> looking at.  But it''s hard to tell if that was bochs, or ntldr
>> giving me a fit.
>>
>> I do know that RAW images work fine, though, and I just tested it
>> with squashfs by running mksquashfs on the base image, and mounting
>> that via loop2 to my base/ subfolder.  Then re-created the scratch
>> area, ran dmsetup, and booted.  Loaded up great, did some changes,
>> shutdown, unloaded the mapper device, unloaded the scratch loop,
>> unmounted my tmpfs, and re-created that segment again, and Windows
>> came back in pristine condition.  So that pretty much means my next
>> battle is Debian''s package manager not liking my local
repository (I
>> can''t connect my Xen system to the internet, so I have to use
such a
>> setup).  As that''s how the Xen LiveCD scripts are doing
things, and
>> assuming everything can be fetched online.
>>
>> Some things I have noticed, though:
>> - After detaching a loopback device, there seems to be a delay in
>> when the kernel actually notices that it''s detached.  If one
does
>> losetup -d <loop> followed by losetup <loop> <new
img>, it seems
>> like in a few cases, the kernel still thought it was looking at the
>> older loop attachment.  I ran into this when using loopback to look
>> at the disk at the disk level, and then passing -o 32256 to losetup
>> to look strictly at the first/primary NTFS partition (while trying
>> to clone it to a smaller disk image, which I finally gave up on
>> doing).
>>
>> - Haven''t found the cause just yet, but dmsetup sometimes
doesn''t
>> like looking at loopback devices, and possibly per the first item,
>> refuses to create the mapped device, citing a resource is busy.
>> Reboot fixes this, and I suspect both of these are caused by all the
>> experimenting I''m doing with switching between the various
>> loopbacks, I probably confuse the kernel.  From a scripted
>> standpoint, it should be a non-issue, though.
>>
>> - Only tested this entire idea/setup against Windows XP.  It seems
>> to be fine, and that generally means any other OS as a domU should
>> work fine as well (because NTFS is by far the pickiest, most
>> temperamental FS known).  You get way more options with Linux/Other
>> UNIXes as far as tricking them into thinking they''re in a r/w
world,
>> but this can be extended to encapsulate quite a few domU''s.
>> However, the only limitation I can see is the number of loopback
>> interfaces.  I use three to setup the Windows domU: /dev/loop2 for
>> the squashfs mount, /dev/loop0 for the windows image inside the
>> squashfs mount, and /dev/loop1 for the scratch area.  I think more
>> loop devices can be easily created, but I''ve not experimented
that
>> far just yet...
>>
>> That''s all for now...I''ve got to install my network
scanner into
>> this Windows image and re-clean it and run these tests all over
>> again.
>>
>> Cheers!
>>
>> --J
>> From: Thiago Camargo Martins Cordeiro [thiagocmartinsc@gmail.com]
>> Sent: Tuesday, August 18, 2009 11:29 PM
>> To: Joshua Kinard
>> Cc: xen-users@lists.xensource.com
>> Subject: Re: [Xen-users] Bootable LiveDVD w/ Xen that boots Windows
>> Image?
>>
>> This procedure will be in the next release of the Xen Live CD!!
>>
>> qcow2 and LVM CoW snapshot for saving storage space while providing
>> fast domU creation from RW snapshots.... good for Eucalyptus, I
>> guess!
>>
>> 2009/8/18 Joshua Kinard <joshua.kinard@sdc-world.com>
>> Think I cracked it.  Don''t even need to deal with
qemu''s qcow stuff
>> (so far).  Still more tests pending (including using read-only drive
>> mounts).  But assume the following steps in a dom0:
>>
>> - mkdir /windows
>> - cd /windows
>> - dd if=/dev/zero of=xp.img bs=1024k count=1 seek=7168
>> - touch xp.cfg
>> - nano xp.cfg
>> - <edit xp.cfg, setup as desired>
>> - xm create xp.cfg
>> - <install WinXP, patch, delete garbage, defrag, zero-out free
>> space, then shutdown>
>> - xm list (verify domU is actually dead)
>>
>> Now after this, xp.img will contain a baseline WinXP install.  In my
>> specific case, I compacted everything down to ~2GB total installed
>> (inside the domU that is) by using straight qemu-emulated devices
>> (not the GPLPV stuff), no page file or hibernation, stripping nearly
>> every misc utility, and a bunch of other stuff, but NO NTFS
>> compression (for now, at least).
>>
>> Next:
>>
>> - dd if=/dev/zero of=xp_scratch.img bs=1024k count=1 seek=2048
>> - losetup /dev/loop0 xp.img
>> - losetup /dev/loop1 xp_scratch.img
>> - echo 0 $(blockdev --getsize /dev/loop0) snapshot /dev/loop0 /dev/
>> loop1 n 64 | dmsetup create xpcow
>>
>> This maps our baseline WinXP image (xp.img) to /dev/loop0 and a
>> scratch area to /dev/loop1 (I don''t even know if this is
needed just
>> yet).  Needed because dmsetup only works with block devices.  Using
>> dmsetup, a snapshot is created, so that all writes to loop0 get
>> redirected to loop1, and creates a device, /dev/mapper/xpcow.
>>
>> Edit xp.cfg to use phy:/dev/mapper/xpcow,hda,w, and boot Windows:
>> - xm create xp.cfg
>>
>> In Windows, create a dummy file on the desktop or three, then
>> shutdown.  Re-start the domU and back at the desktop, we see that
>> the newly-created files still exist, so shutdown again.  Now:
>>
>> - dmsetup remove /dev/mapper/xpcow
>> - losetup -d /dev/loop1
>> - dd if=/dev/zero of=xp_scratch2.img bs=1024k count=1 seek=2048
>> - losetup /dev/loop1 xp_scratch2.img
>> - echo 0 $(blockdev --getsize /dev/loop0) snapshot /dev/loop0 /dev/
>> loop1 n 64 | dmsetup create xpcow
>> - xm create xp.cfg
>>
>> Back at the desktop, the newly-created files are now gone!
>>
>> I did it this way as a manual test, because I''m veryifying a
few
>> things:
>> a. Simulating the effect of creating the scratch image in a /tmpfs
>> mount, while the baseline image will be on a physical, read-only
>> medium.  By removing the mapped device and changing out the scratch
>> file, I was seeing whether the XP image would indeed revert to the
>> baseline state, to simulate a reboot in which /tmpfs is cleared.
>> Supposedly, the ''n'' parameter to dmsetup does this
anyways, but I
>> wanted to be doubly sure.
>>
>> b. Need to scriptify the whole mess, since the baseline image will
>> already be pre-configured, the generation of the scratch image,
>> setup of the loop devices, and the device-mapping will be on-the-fly
>> when some future CD Image I put together boots up.
>>
>> c. Avoids all the problems I see on Google about using blktap with
>> qcow with gplpv and such.  I''m probably not grasping some of
it, but
>> I think those references are aiming for different uses.  So far,
>> this setup mostly relies wholly on the Linux dom0 to piece
>> everything together, leaving the WinXP domU utterly oblivious to
>> what is going on.  And given Windows'' nature, that''s
exactly what we
>> want.
>>
>> Performance isn''t an issue for me in this, so I''m
sure on a CD, this
>> will be slower than running Oracle on an 8086 (pretend for a moment
>> that this is actually possible), but if it works, and WinXP
doesn''t
>> complain, then all is good.
>>
>> Thoughts?
>> ________________________________________
>> From: xen-users-bounces@lists.xensource.com
[xen-users-bounces@lists.xensource.com
>> ] On Behalf Of Joshua Kinard [joshua.kinard@sdc-world.com]
>> Sent: Monday, August 17, 2009 9:50 AM
>> To: xen-users@lists.xensource.com
>> Subject: RE: [Xen-users] Bootable LiveDVD w/ Xen that boots Windows
>> Image?
>>
>> Hmm, block-level copy-on-write?  If qcow supports this out-of-the-
>> box, then I technically don''t need any kind of UnionFS?  Just
>> someway to specify to Xen a read-only image with a qcow2 image in a
>> writable zone?
>>
>> Going to dredge through some EU software patent I dug up from
>> Microsoft.  Supposed to document the /MININT switch to the NT kernel
>> that supposedly makes Windows write all registry changes to volatile
>> memory instead of to the registry hives.  It''s mostly used in
the
>> WinPE environment, but it looks like BartPE leverages it too.  Seems
>> I need more to it, though, than just modifying boot.ini to pass it,
>> as once I started to boot Windows from a read-only drive mount
>> (after discovering Xen/qemu don''t properly honor the mode bit
in the
>> Xen config for r or r/w, and even file-level permissions), Windows
>> crashed with UNMOUNTABLE_BOOT_VOLUME as its error.  But it
>> highlights the capability is there....just not easy.
>>
>> USB is out, unfortunately, too.  For security reasons, we ban USB/
>> Firewire drives here, with the exception of CD Burners.  Ditto on
>> memory cards (SD, MMC, etc..).  I''ve got very narrow operating
>> parameters to work in, which is why this is proving to be quite a
>> fun challenge.
>>
>> Thanks!,
>>
>> --J
>>
>> ________________________________________
>> From: Fajar A. Nugraha [fajar@fajar.net]
>> Sent: Friday, August 14, 2009 4:53 AM
>> To: Joshua Kinard
>> Cc: xen-users@lists.xensource.com
>> Subject: Re: [Xen-users] Bootable LiveDVD w/ Xen that boots Windows
>> Image?
>>
>> On Fri, Aug 14, 2009 at 1:37 AM, Joshua
>> Kinard<joshua.kinard@sdc-world.com> wrote:
>>> Hmm, I didn''t think of it that way.
>>>
>>> The way I read up on UnionFS and aufs'' functionality, was
they
>> could essentially merge two files virtually,
>>
>> Merge two directories together, to be exact. Not merge files.
>>
>>> i.e., the kernel module would be able to look at the operation
>> coming in and route it to the proper descriptor (i.e., read() --> /
>> livecd/windows/winxp.img OR write() --> /tmp/winxp.img, with /tmp
>> being in tmpfs).  I guess it''s not as granular as that it
seems.
>> Would be a neat trick, but I imagine it''d be complex as
anything for
>> a kernel module to have to keep track of which files have variants
>> loaded in the writeable union area.
>>
>> What you describe is essentially block-level copy-on-write, which is
>> what qcow or zfs does. Aufs/unionfs does this per file.
>>
>>> As for the application, it''s a complex network security
scanner,
>> made by eEye Digital Security, called "Retina".  We just
don''t want
>> to setup and baby sit Windows installations on our Unix networks
>> strictly for this one app, so I figured if I can get it to run off
>> of a CD, we can just park some diskless hardware in a closet and
>> pull it out whenever we need to do network testing and such.
>>
>> If it were me I''d simply setup a Windows domU on any server
with
>> enough disk and RAM, make a "template" of the
"good" installation
>> (preferably zfs or LVM snapshot), start it only when it''s
needed,
>> shut
>> it down afterwards, and (if necessary) rollback to the "good"
>> template. That is assuming that all host/network tested reachable
>> from
>> my datacenter (either with vlans or routing). No need to add the
>> complexity of a live DVD/USB.
>>
>> If portability is a requirement, and you''re absolutely sure
you''ll
>> always have VT-capable host handy, then using live USB is much
>> perefered than DVD due to performance and complexity reasons. But
>> hey,
>> that''s just me :D
>>
>> Let us know if you find a solution that works.
>>
>> --
>> Fajar
>> _______________________________________________
>> Xen-users mailing list
>> Xen-users@lists.xensource.com
>> http://lists.xensource.com/xen-users
>> _______________________________________________
>> Xen-users mailing list
>> Xen-users@lists.xensource.com
>> http://lists.xensource.com/xen-users
>>
>> _______________________________________________
>> Xen-users mailing list
>> Xen-users@lists.xensource.com
>> http://lists.xensource.com/xen-users
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users



_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Handy to know, thanks!

I''ll look at giving them a shot and see what happens.

--J

________________________________________
From: Dustin Henning [Dustin.Henning@prd-inc.com]
Sent: Monday, August 24, 2009 1:04 PM
To: Joshua Kinard; xen-users@lists.xensource.com
Subject: RE: [Xen-users] Bootable LiveDVD w/ Xen that boots Windows Image?

        Regarding your comment on GPLPV, it does not expose any real
hardware to the HVM domU; it exposes and provides drivers for the PV devices
that are always used by PV domains (instead of the fully virtualized devices
typically used by HVM domains).  I don''t believe the PV devices change
when
the hardware on a machine changes (just the fully virtualized devices
don''t
change), so the GPLPV drivers should provide performance enhancement without
adding any new potential (non-driver related) issues once they are working
on a LiveDVD (because nothing should change in terms of what your guest is
seeing once you have everything on RO storage).
        Dustin

-----Original Message-----
From: xen-users-bounces@lists.xensource.com
[mailto:xen-users-bounces@lists.xensource.com] On Behalf Of Joshua Kinard
Sent: Thursday, August 20, 2009 09:48
To: xen-users@lists.xensource.com
Subject: RE: [Xen-users] Bootable LiveDVD w/ Xen that boots Windows Image?

Yeah, I''ve looked at the GPLPV stuff, and it looks like it''d
expose Windows
to the real hardware.  While that has big performance benefits, since this
would be on a CD that could visit multiple machine configurations, leaving
the driver detection to the underlying Linux distro, and virtualizing stuff
for the Windows guest will be the more preferred route (because don''t
we all
love Windows'' brain-dead driver detection, especially in XP?).

I did try taking a look at BartPE initially, but making a plugin for the PE
environment proved futile, as there are just way too many registry entries
created by the scanning program upon install.  WinPE was another option (and
it''s free now as well), but it won''t run .NET code at all, so
that idea was
ruled out.  Wine got nowhere, and VMWare/VirtualBox have too many license
restrictions.  Although the end result will be an internal tool, some of the
knowledge I''m piecing together for this will wind up in the community
(mostly the device-mapper concepts I imagine), but we still want to be as
legit as possible internally.  It''s just good practice :)

SquashFS takes care of the RAMDisk problem.  A 7.1GB WinXP disk image,
internally consumes 2.7GB after all the XP patches are applied (per MBSA
2.1), and my network scanner program installed with its latest audit
updates.  But that gets compressed down to 1.3GB with SquashFS 3.1.  I might
try to kludge 4.x onto my Debian setup for their performance/compression
enhancements, but we''ll see.  On a 4.7GB DVD, that still leaves me
quite a
bit of room for the Linux root, and being Debian based, that will be really
small, especially with its own SquashFS glue and the appropriate boot
scripts in the initramfs.

The plus is I''ve learned a ton about Xen and virtualization, previously
having only been a VMWare guy.  So I''ve got other ideas
that''ll bubble up in
my head on how to leverage all of this down the road, too!

Cheers,

--J

________________________________________
From: John Lewis [jlewis@concentric.com]
Sent: Wednesday, August 19, 2009 6:46 PM
To: Joshua Kinard
Subject: Re: [Xen-users] Bootable LiveDVD w/ Xen that boots Windows Image?

Thanks for the info! Sounds like you have an interesting solution in
the works to your challenge. Its a shame that microsoft doesn''t have a
live dvd distro for this type of need. I would suggest putting the
entire sparse image of windows into sort sort of ramdisk. I am more of
a domu/windows/pv driver guy so I am not sure how you would go about
doing this with the live distro you are choosing.

Good luck, sounds like an interesting project. I am looking forward to
hearing what the final resolution is (if it isn''t in the list already).

Lewis


On Aug 19, 2009, at 11:37 AM, Joshua Kinard wrote:
> Yup, thumb drives, SD cards, MMC, etc, aren''t an option where
I''m
> at.  I maintain several offline networks for testing software, and
> they all run various UNIXes, but no Windows setup.  Higher level
> management wants us to run vuneralbility scans on these networks
> using a Windows-based product (which I referenced earlier), but
> telling them "We don''t use Windows" won''t work.
So I was looking
> for a way to scan the networks without maintaining a Windows install
> on each one specifically for this one program.
>
> Hence, booting Windows off of a LiveCD pre-loaded with everything I
> need (and fully licensed, because we have a site license).  Unique
> problem, unique solution....just needs some more glue to put it all
> together.  I can update the one Windows install as-needed, load
> newer audit information, burn it to CD, and run my scans.
>
> Hope that clears things up!
>
> --J
> ________________________________________
> From: John Lewis [jlewis@concentric.com]
> Sent: Wednesday, August 19, 2009 1:24 PM
> To: Joshua Kinard
> Cc: xen-users@lists.xensource.com
> Subject: Re: [Xen-users] Bootable LiveDVD w/ Xen that boots Windows
> Image?
>
> Two questions, probably the same answer. I am just trying to
> understand why you are trying to do this, and what practical use this
> has. Maybe I will have a use for this in the future.
>
> What is the goal in doing this?
> Are usb thumb drives not an option?
>
> Lewis
>
> On Aug 19, 2009, at 7:56 AM, Joshua Kinard wrote:
>
>> Hmm, I don''t think qcow format is going to work with that
method.
>> At least when I tried it, the bochs bios didn''t like what it
was
>> looking at.  But it''s hard to tell if that was bochs, or ntldr
>> giving me a fit.
>>
>> I do know that RAW images work fine, though, and I just tested it
>> with squashfs by running mksquashfs on the base image, and mounting
>> that via loop2 to my base/ subfolder.  Then re-created the scratch
>> area, ran dmsetup, and booted.  Loaded up great, did some changes,
>> shutdown, unloaded the mapper device, unloaded the scratch loop,
>> unmounted my tmpfs, and re-created that segment again, and Windows
>> came back in pristine condition.  So that pretty much means my next
>> battle is Debian''s package manager not liking my local
repository (I
>> can''t connect my Xen system to the internet, so I have to use
such a
>> setup).  As that''s how the Xen LiveCD scripts are doing
things, and
>> assuming everything can be fetched online.
>>
>> Some things I have noticed, though:
>> - After detaching a loopback device, there seems to be a delay in
>> when the kernel actually notices that it''s detached.  If one
does
>> losetup -d <loop> followed by losetup <loop> <new
img>, it seems
>> like in a few cases, the kernel still thought it was looking at the
>> older loop attachment.  I ran into this when using loopback to look
>> at the disk at the disk level, and then passing -o 32256 to losetup
>> to look strictly at the first/primary NTFS partition (while trying
>> to clone it to a smaller disk image, which I finally gave up on
>> doing).
>>
>> - Haven''t found the cause just yet, but dmsetup sometimes
doesn''t
>> like looking at loopback devices, and possibly per the first item,
>> refuses to create the mapped device, citing a resource is busy.
>> Reboot fixes this, and I suspect both of these are caused by all the
>> experimenting I''m doing with switching between the various
>> loopbacks, I probably confuse the kernel.  From a scripted
>> standpoint, it should be a non-issue, though.
>>
>> - Only tested this entire idea/setup against Windows XP.  It seems
>> to be fine, and that generally means any other OS as a domU should
>> work fine as well (because NTFS is by far the pickiest, most
>> temperamental FS known).  You get way more options with Linux/Other
>> UNIXes as far as tricking them into thinking they''re in a r/w
world,
>> but this can be extended to encapsulate quite a few domU''s.
>> However, the only limitation I can see is the number of loopback
>> interfaces.  I use three to setup the Windows domU: /dev/loop2 for
>> the squashfs mount, /dev/loop0 for the windows image inside the
>> squashfs mount, and /dev/loop1 for the scratch area.  I think more
>> loop devices can be easily created, but I''ve not experimented
that
>> far just yet...
>>
>> That''s all for now...I''ve got to install my network
scanner into
>> this Windows image and re-clean it and run these tests all over
>> again.
>>
>> Cheers!
>>
>> --J
>> From: Thiago Camargo Martins Cordeiro [thiagocmartinsc@gmail.com]
>> Sent: Tuesday, August 18, 2009 11:29 PM
>> To: Joshua Kinard
>> Cc: xen-users@lists.xensource.com
>> Subject: Re: [Xen-users] Bootable LiveDVD w/ Xen that boots Windows
>> Image?
>>
>> This procedure will be in the next release of the Xen Live CD!!
>>
>> qcow2 and LVM CoW snapshot for saving storage space while providing
>> fast domU creation from RW snapshots.... good for Eucalyptus, I
>> guess!
>>
>> 2009/8/18 Joshua Kinard <joshua.kinard@sdc-world.com>
>> Think I cracked it.  Don''t even need to deal with
qemu''s qcow stuff
>> (so far).  Still more tests pending (including using read-only drive
>> mounts).  But assume the following steps in a dom0:
>>
>> - mkdir /windows
>> - cd /windows
>> - dd if=/dev/zero of=xp.img bs=1024k count=1 seek=7168
>> - touch xp.cfg
>> - nano xp.cfg
>> - <edit xp.cfg, setup as desired>
>> - xm create xp.cfg
>> - <install WinXP, patch, delete garbage, defrag, zero-out free
>> space, then shutdown>
>> - xm list (verify domU is actually dead)
>>
>> Now after this, xp.img will contain a baseline WinXP install.  In my
>> specific case, I compacted everything down to ~2GB total installed
>> (inside the domU that is) by using straight qemu-emulated devices
>> (not the GPLPV stuff), no page file or hibernation, stripping nearly
>> every misc utility, and a bunch of other stuff, but NO NTFS
>> compression (for now, at least).
>>
>> Next:
>>
>> - dd if=/dev/zero of=xp_scratch.img bs=1024k count=1 seek=2048
>> - losetup /dev/loop0 xp.img
>> - losetup /dev/loop1 xp_scratch.img
>> - echo 0 $(blockdev --getsize /dev/loop0) snapshot /dev/loop0 /dev/
>> loop1 n 64 | dmsetup create xpcow
>>
>> This maps our baseline WinXP image (xp.img) to /dev/loop0 and a
>> scratch area to /dev/loop1 (I don''t even know if this is
needed just
>> yet).  Needed because dmsetup only works with block devices.  Using
>> dmsetup, a snapshot is created, so that all writes to loop0 get
>> redirected to loop1, and creates a device, /dev/mapper/xpcow.
>>
>> Edit xp.cfg to use phy:/dev/mapper/xpcow,hda,w, and boot Windows:
>> - xm create xp.cfg
>>
>> In Windows, create a dummy file on the desktop or three, then
>> shutdown.  Re-start the domU and back at the desktop, we see that
>> the newly-created files still exist, so shutdown again.  Now:
>>
>> - dmsetup remove /dev/mapper/xpcow
>> - losetup -d /dev/loop1
>> - dd if=/dev/zero of=xp_scratch2.img bs=1024k count=1 seek=2048
>> - losetup /dev/loop1 xp_scratch2.img
>> - echo 0 $(blockdev --getsize /dev/loop0) snapshot /dev/loop0 /dev/
>> loop1 n 64 | dmsetup create xpcow
>> - xm create xp.cfg
>>
>> Back at the desktop, the newly-created files are now gone!
>>
>> I did it this way as a manual test, because I''m veryifying a
few
>> things:
>> a. Simulating the effect of creating the scratch image in a /tmpfs
>> mount, while the baseline image will be on a physical, read-only
>> medium.  By removing the mapped device and changing out the scratch
>> file, I was seeing whether the XP image would indeed revert to the
>> baseline state, to simulate a reboot in which /tmpfs is cleared.
>> Supposedly, the ''n'' parameter to dmsetup does this
anyways, but I
>> wanted to be doubly sure.
>>
>> b. Need to scriptify the whole mess, since the baseline image will
>> already be pre-configured, the generation of the scratch image,
>> setup of the loop devices, and the device-mapping will be on-the-fly
>> when some future CD Image I put together boots up.
>>
>> c. Avoids all the problems I see on Google about using blktap with
>> qcow with gplpv and such.  I''m probably not grasping some of
it, but
>> I think those references are aiming for different uses.  So far,
>> this setup mostly relies wholly on the Linux dom0 to piece
>> everything together, leaving the WinXP domU utterly oblivious to
>> what is going on.  And given Windows'' nature, that''s
exactly what we
>> want.
>>
>> Performance isn''t an issue for me in this, so I''m
sure on a CD, this
>> will be slower than running Oracle on an 8086 (pretend for a moment
>> that this is actually possible), but if it works, and WinXP
doesn''t
>> complain, then all is good.
>>
>> Thoughts?
>> ________________________________________
>> From: xen-users-bounces@lists.xensource.com
[xen-users-bounces@lists.xensource.com
>> ] On Behalf Of Joshua Kinard [joshua.kinard@sdc-world.com]
>> Sent: Monday, August 17, 2009 9:50 AM
>> To: xen-users@lists.xensource.com
>> Subject: RE: [Xen-users] Bootable LiveDVD w/ Xen that boots Windows
>> Image?
>>
>> Hmm, block-level copy-on-write?  If qcow supports this out-of-the-
>> box, then I technically don''t need any kind of UnionFS?  Just
>> someway to specify to Xen a read-only image with a qcow2 image in a
>> writable zone?
>>
>> Going to dredge through some EU software patent I dug up from
>> Microsoft.  Supposed to document the /MININT switch to the NT kernel
>> that supposedly makes Windows write all registry changes to volatile
>> memory instead of to the registry hives.  It''s mostly used in
the
>> WinPE environment, but it looks like BartPE leverages it too.  Seems
>> I need more to it, though, than just modifying boot.ini to pass it,
>> as once I started to boot Windows from a read-only drive mount
>> (after discovering Xen/qemu don''t properly honor the mode bit
in the
>> Xen config for r or r/w, and even file-level permissions), Windows
>> crashed with UNMOUNTABLE_BOOT_VOLUME as its error.  But it
>> highlights the capability is there....just not easy.
>>
>> USB is out, unfortunately, too.  For security reasons, we ban USB/
>> Firewire drives here, with the exception of CD Burners.  Ditto on
>> memory cards (SD, MMC, etc..).  I''ve got very narrow operating
>> parameters to work in, which is why this is proving to be quite a
>> fun challenge.
>>
>> Thanks!,
>>
>> --J
>>
>> ________________________________________
>> From: Fajar A. Nugraha [fajar@fajar.net]
>> Sent: Friday, August 14, 2009 4:53 AM
>> To: Joshua Kinard
>> Cc: xen-users@lists.xensource.com
>> Subject: Re: [Xen-users] Bootable LiveDVD w/ Xen that boots Windows
>> Image?
>>
>> On Fri, Aug 14, 2009 at 1:37 AM, Joshua
>> Kinard<joshua.kinard@sdc-world.com> wrote:
>>> Hmm, I didn''t think of it that way.
>>>
>>> The way I read up on UnionFS and aufs'' functionality, was
they
>> could essentially merge two files virtually,
>>
>> Merge two directories together, to be exact. Not merge files.
>>
>>> i.e., the kernel module would be able to look at the operation
>> coming in and route it to the proper descriptor (i.e., read() --> /
>> livecd/windows/winxp.img OR write() --> /tmp/winxp.img, with /tmp
>> being in tmpfs).  I guess it''s not as granular as that it
seems.
>> Would be a neat trick, but I imagine it''d be complex as
anything for
>> a kernel module to have to keep track of which files have variants
>> loaded in the writeable union area.
>>
>> What you describe is essentially block-level copy-on-write, which is
>> what qcow or zfs does. Aufs/unionfs does this per file.
>>
>>> As for the application, it''s a complex network security
scanner,
>> made by eEye Digital Security, called "Retina".  We just
don''t want
>> to setup and baby sit Windows installations on our Unix networks
>> strictly for this one app, so I figured if I can get it to run off
>> of a CD, we can just park some diskless hardware in a closet and
>> pull it out whenever we need to do network testing and such.
>>
>> If it were me I''d simply setup a Windows domU on any server
with
>> enough disk and RAM, make a "template" of the
"good" installation
>> (preferably zfs or LVM snapshot), start it only when it''s
needed,
>> shut
>> it down afterwards, and (if necessary) rollback to the "good"
>> template. That is assuming that all host/network tested reachable
>> from
>> my datacenter (either with vlans or routing). No need to add the
>> complexity of a live DVD/USB.
>>
>> If portability is a requirement, and you''re absolutely sure
you''ll
>> always have VT-capable host handy, then using live USB is much
>> perefered than DVD due to performance and complexity reasons. But
>> hey,
>> that''s just me :D
>>
>> Let us know if you find a solution that works.
>>
>> --
>> Fajar
>> _______________________________________________
>> Xen-users mailing list
>> Xen-users@lists.xensource.com
>> http://lists.xensource.com/xen-users
>> _______________________________________________
>> Xen-users mailing list
>> Xen-users@lists.xensource.com
>> http://lists.xensource.com/xen-users
>>
>> _______________________________________________
>> Xen-users mailing list
>> Xen-users@lists.xensource.com
>> http://lists.xensource.com/xen-users
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users