Xen devel - Oct 2013 - Is there an issue with turning off "scrubbing free RAM" on boot with Xen 4.1.3

Greetings,

Thank you in advance for your support!

Our HP Xen 4.1.3 servers have 1TB of RAM, each Xen servers take 20 
minutes to boot largely due to the "scrub free RAM" phase. If/when we 
have dom0 failures and HA kicks-in, we would like to reduce the boot 
time to make the resource quickly available, perhaps using the 
no-bootscrub attribute in grub.conf.

Could you please share your comments about turning of RAM scrubbing, 
i.e. have you seen any consequences, security issues and/or threats, red 
flags, etc...?

We have asked the same question at the commercially supported Xen 
forums, i.e. Oracle and Citrix, as well as to each aforementioned 
support team, and have not received a lick of meaningful information.

Respectfully,

Roddy

-- 
Roddy Rodstein  CEO and Founder
Mokum Solutions, Inc.
Phone:  (415) 252-9164
E-mail: roddy.rodstein@mokumsolutions.com  Web: http://mokumsolutions.com and
http://itnewscast.com
Follow me on Twitter: http://twitter.com/itnewscast
Up-to-date Oracle news by Mokum: http://itnewscast.com/
CONFIDENTIAL  "The information contained in this e-mail and any attachment
is confidential. It is intended only for the named addressee(s). If you are not
the named addressee please notify the sender immediately and do not disclose,
copy or distribute the contents to any other person other than the intended
addressee(s)"




_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

On Wed, Oct 09, 2013 at 11:24:22AM -0700, Roddy Rodstein
wrote:
>    Greetings,
> 
>    Thank you in advance for your support!
> 
>    Our HP Xen 4.1.3 servers have 1TB of RAM, each Xen servers take 20
minutes
>    to boot largely due to the "scrub free RAM" phase. If/when we
have dom0
>    failures and HA kicks-in, we would like to reduce the boot time to make
>    the resource quickly available, perhaps using the no-bootscrub attribute
>    in grub.conf.
> 
>    Could you please share your comments about turning of RAM scrubbing,
i.e.
>    have you seen any consequences, security issues and/or threats, red
flags,
>    etc...?
> 
>    We have asked the same question at the commercially supported Xen
forums,
>    i.e. Oracle and Citrix, as well as to each aforementioned support team,
>    and have not received a lick of meaningful information.
>
If that''s a custom build of Xen you can apply the patches that optimize
the boot time memory scrubbing,
they''ve been posted to xen-devel a couple of times.. 

-- Pasi
 
> 
> 
>    Respectfully,
> 
>    Roddy
> 
>  --
>  Roddy Rodstein  CEO and Founder
>  Mokum Solutions, Inc.
>  Phone:  (415) 252-9164
>  E-mail: [1]roddy.rodstein@mokumsolutions.com  Web:
[2]http://mokumsolutions.com and [3]http://itnewscast.com
>  Follow me on Twitter: [4]http://twitter.com/itnewscast
>  Up-to-date Oracle news by Mokum: [5]http://itnewscast.com/
>  CONFIDENTIAL  "The information contained in this e-mail and any
attachment is confidential. It is intended only for the named addressee(s). If
you are not the named addressee please notify the sender immediately and do not
disclose, copy or distribute the contents to any other person other than the
intended addressee(s)"
> 
> References
> 
>    Visible links
>    1. mailto:roddy.rodstein@mokumsolutions.com
>    2. http://mokumsolutions.com/
>    3. http://itnewscast.com/
>    4. http://twitter.com/itnewscast
>    5. http://itnewscast.com/
> begin:vcard
> fn:Roddy Rodstein
> n:Rodstein;Roddy
> email;internet:roddy.rodstein@mokumsolutions.com
> tel;work:4152529164
> tel;cell:4158602851
> version:2.1
> end:vcard
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

On 09/10/13 19:24, Roddy Rodstein wrote:
>
> Greetings,
>
> Thank you in advance for your support!
>
> Our HP Xen 4.1.3 servers have 1TB of RAM, each Xen servers take 20 
> minutes to boot largely due to the "scrub free RAM" phase.
If/when we
> have dom0 failures and HA kicks-in, we would like to reduce the boot 
> time to make the resource quickly available, perhaps using the 
> no-bootscrub attribute in grub.conf.
>
>
Malcolm''s patch to parallelize scrubbing was posted recently

http://lists.xen.org/archives/html/xen-devel/2013-09/msg03171.html

I don''t think it''s been committed to xen-unstable yet,

Simon



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

On Wed, 2013-10-09 at 11:24 -0700, Roddy Rodstein wrote:
> Could you please share your comments about turning of RAM scrubbing,
> i.e. have you seen any consequences, security issues and/or threats,
> red flags, etc...?
The scrub is there to protect against possibly stale data in RAM left
over from guests running during the previous boot being exposed to new
guests. If you don''t care about that threat then you don''t
need to scan
the boot RAM.

Ian.

On 09/10/13 19:24, Roddy Rodstein wrote:
>
> Greetings,
>
>  
>
> Thank you in advance for your support!
>
>  
>
> Our HP Xen 4.1.3 servers have 1TB of RAM, each Xen servers take 20
> minutes to boot largely due to the "scrub free RAM" phase.
If/when we
> have dom0 failures and HA kicks-in, we would like to reduce the boot
> time to make the resource quickly available, perhaps using the
> no-bootscrub attribute in grub.conf.
>
>  
>
> Could you please share your comments about turning of RAM scrubbing,
> i.e. have you seen any consequences, security issues and/or threats,
> red flags, etc...?
>
>  
>
> We have asked the same question at the commercially supported Xen
> forums, i.e. Oracle and Citrix, as well as to each aforementioned
> support team, and have not received a lick of meaningful information.
>
>  
>
> Respectfully,
>
> Roddy
>
In the Xen model, domains are responsible for clearing any sensitive
data they have out of memory before shutdown.

The bootscrub is a preventative measure to ensure that after a crash,
stale domain information is cleared from RAM before that RAM is reused
for a new VM.

If this is not a concern for you, then you can easily turn bootscrub off
by adding "no-bootscrub" to the Xen command line.

~Andrew


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

On Thu, Oct 10, 2013 at 10:42:14AM +0100, Andrew Cooper
wrote:
> On 09/10/13 19:24, Roddy Rodstein wrote:
[...]
> > Could you please share your comments about turning of RAM scrubbing,
> > i.e. have you seen any consequences, security issues and/or threats,
> > red flags, etc...?
[...]
> In the Xen model, domains are responsible for clearing any sensitive
> data they have out of memory before shutdown.
This isn''t strictly true. Memory is scrubbed by Xen when the domain
cannot do it for itself (i.e., when a domain is dying during
shutdown). However by default domains /are/ responsible for scrubbing
pages that are returned to Xen via a reservation adjustment (i.e.,
pages returned via the balloon driver).

--msw
> The bootscrub is a preventative measure to ensure that after a crash,
> stale domain information is cleared from RAM before that RAM is reused
> for a new VM.
> 
> If this is not a concern for you, then you can easily turn bootscrub off
> by adding "no-bootscrub" to the Xen command line.

On Sun, 2013-11-10 at 14:25 -0800, Matt Wilson wrote:
> On Thu, Oct 10, 2013 at 10:42:14AM +0100, Andrew Cooper wrote:
> > On 09/10/13 19:24, Roddy Rodstein wrote:
> 
> [...]
> 
> > > Could you please share your comments about turning of RAM
scrubbing,
> > > i.e. have you seen any consequences, security issues and/or
threats,
> > > red flags, etc...?
> 
> [...]
> 
> > In the Xen model, domains are responsible for clearing any sensitive
> > data they have out of memory before shutdown.
> 
> This isn''t strictly true. Memory is scrubbed by Xen when the
domain
> cannot do it for itself (i.e., when a domain is dying during
> shutdown).
Isn''t this only when the domain is killed by the toolstack or crashes
etc. On a graceful shutdown I thought the guest was still responsible
for clearing any memory it cared about.
>  However by default domains /are/ responsible for scrubbing
> pages that are returned to Xen via a reservation adjustment (i.e.,
> pages returned via the balloon driver).
> 
> --msw
> 
> > The bootscrub is a preventative measure to ensure that after a crash,
> > stale domain information is cleared from RAM before that RAM is reused
> > for a new VM.
> > 
> > If this is not a concern for you, then you can easily turn bootscrub
off
> > by adding "no-bootscrub" to the Xen command line.
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

>>> On 11.11.13 at 11:14, Ian Campbell <Ian.Campbell@citrix.com>
wrote:
> On Sun, 2013-11-10 at 14:25 -0800, Matt Wilson wrote:
>> On Thu, Oct 10, 2013 at 10:42:14AM +0100, Andrew Cooper wrote:
>> > In the Xen model, domains are responsible for clearing any
sensitive
>> > data they have out of memory before shutdown.
>> 
>> This isn''t strictly true. Memory is scrubbed by Xen when the
domain
>> cannot do it for itself (i.e., when a domain is dying during
>> shutdown).
> 
> Isn''t this only when the domain is killed by the toolstack or
crashes
> etc. On a graceful shutdown I thought the guest was still responsible
> for clearing any memory it cared about.
No, the scrubbing is independent of the shutdown reason:

        /*
         * Normally we expect a domain to clear pages before freeing them, if 
         * it cares about the secrecy of their contents. However, after a 
         * domain has died we assume responsibility for erasure.
         */
        if ( unlikely(d->is_dying) )
            for ( i = 0; i < (1 << order); i++ )
                scrub_one_page(&pg[i]);

Jan

On Mon, 2013-11-11 at 10:33 +0000, Jan Beulich wrote:
> >>> On 11.11.13 at 11:14, Ian Campbell
<Ian.Campbell@citrix.com> wrote:
> > On Sun, 2013-11-10 at 14:25 -0800, Matt Wilson wrote:
> >> On Thu, Oct 10, 2013 at 10:42:14AM +0100, Andrew Cooper wrote:
> >> > In the Xen model, domains are responsible for clearing any
sensitive
> >> > data they have out of memory before shutdown.
> >> 
> >> This isn''t strictly true. Memory is scrubbed by Xen when
the domain
> >> cannot do it for itself (i.e., when a domain is dying during
> >> shutdown).
> > 
> > Isn''t this only when the domain is killed by the toolstack or
crashes
> > etc. On a graceful shutdown I thought the guest was still responsible
> > for clearing any memory it cared about.
> 
> No, the scrubbing is independent of the shutdown reason:
> 
>         /*
>          * Normally we expect a domain to clear pages before freeing them,
if
>          * it cares about the secrecy of their contents. However, after a 
>          * domain has died we assume responsibility for erasure.
>          */
>         if ( unlikely(d->is_dying) )
>             for ( i = 0; i < (1 << order); i++ )
>                 scrub_one_page(&pg[i]);
My mistake, thanks for the correction.

This does seem safer/wiser in any case...

Ian.