Xen users - Jun 2013 - Xen 4.2.2 /etc/init.d/xendomains save and restore of domains does not work

Hello,

in the last days I switched from Xen 4.2.1 to Xen 4.2.2 (source from git 
repository tag 4.2.2)


commit a125ec6a4a2f028f97f0bdd8946d347a36dbe76b
Author: Jan Beulich <jbeulich@suse.com>
Date:   Tue Apr 23 18:42:55 2013 +0200

     update Xen version to 4.2.2


My "base system" is a debian squeeze. For Xen I use the xl toolstack.

Since updating to 4.2.2 the autosave and autorestore of domains dos not 
work any more.

The reason is that there are some bugs in /etc/init.d/xendomains

At the  end of the mail there is a diff of my corrections. The 
correction are only tested on my system under xl toolstack


1)  @@ -182,7 +182,7 @@ The rdname does not report any domain name

2)  @@ -254,8 +254,8 @@ The header of a saved domain does not contain 
"LinuxGuestRecord" instead it contains "Xen saved domain".

3) @@ -365,11 +365,11 @@  The escape character `\´  in front of `$´  in 
`case \"\$name\"´ in suspress Variable expansion
I think The leading `(´  in ($NAMES) is wrong in sh/bash is wrong 
because than the content of `$name´ will be compared to 
`(whateverinnames´ and that could not match. I think the same bug is in 
`(*)´

4) @@ -430,6 +430,7 @@
+5) @@ -446,6 +447,7 @@ The variables `name´ and `id´ must be cleared. 
If not you get on the next loop name and id mixed from different domains.

I hope that helps.

Best wishes


Andreas

------BEGIN Attachment -------------------------------------------------
 > diff -u .xendomains.original xendomains
--- .xendomains.original        2013-06-25 07:42:29.000000000 +0200
+++ xendomains  2013-06-30 10:48:06.000000000 +0200
@@ -182,7 +182,7 @@
  rdname()
  {
      NM=$($CMD create --quiet --dryrun --defconfig "$1" |
-         sed -n ''s/^.*(name \(.*\))$/\1/p'')
+         sed -n ''s/^.*"name":
"\(.*\)",$/\1/p'')
  }

  rdnames()
@@ -254,8 +254,8 @@
         saved_domains=`ls $XENDOMAINS_SAVE`
          for dom in $XENDOMAINS_SAVE/*; do
              if [ -f $dom ] ; then
-                HEADER=`head -c 16 $dom | head -n 1 2> /dev/null`
-                if [ $HEADER = "LinuxGuestRecord" ]; then
+                HEADER=`{ head -c 16 $dom; echo ""; } | head -n 1
2>
/dev/null`
+                if [ "$HEADER" = "LinuxGuestRecord" -o
"$HEADER" = "Xen
saved domain" ]; then
                      echo -n " ${dom##*/}"
                      XMR=`$CMD restore $dom 2>&1 1>/dev/null`
                      #$CMD restore $dom
@@ -365,11 +365,11 @@
         echo -n " $name"
         if test "$XENDOMAINS_AUTO_ONLY" = "true"; then
             eval "
-           case \"\$name\" in
-               ($NAMES)
+           case \"$name\" in
+               $NAMES)
                     # nothing
                     ;;
-               (*)
+                *)
                     echo -e ''(skip)''
                     continue
                     ;;
@@ -430,6 +430,7 @@
                 kill $WDOG_PID >/dev/null 2>&1
                 echo -e .
                 usleep 1000
+               name=;id                 continue
             fi
         fi
@@ -446,6 +447,7 @@
             fi
             kill $WDOG_PID >/dev/null 2>&1
         fi
+       name=;id      done < <($CMD list -l | grep $LIST_GREP)

      # NB. this shuts down ALL Xen domains (politely), not just the ones in
------END Attachment ---------------------------------------------

On 30/06/13 10:49, Andreas Greve wrote:
> Hello,
>
> in the last days I switched from Xen 4.2.1 to Xen 4.2.2 (source from 
> git repository tag 4.2.2)
>
>
> commit a125ec6a4a2f028f97f0bdd8946d347a36dbe76b
> Author: Jan Beulich <jbeulich@suse.com>
> Date:   Tue Apr 23 18:42:55 2013 +0200
>
>     update Xen version to 4.2.2
>
>
> My "base system" is a debian squeeze. For Xen I use the xl
toolstack.
>
> Since updating to 4.2.2 the autosave and autorestore of domains dos 
> not work any more.
>
> The reason is that there are some bugs in /etc/init.d/xendomains
>
> At the  end of the mail there is a diff of my corrections. The 
> correction are only tested on my system under xl toolstack
>
There are some fixes going into Xen 4.3. My suggestion would be try one 
commit after RC6, namely...

http://xenbits.xen.org/gitweb/?p=xen.git;a=commit;h=543a2657182dbb9237d1feeb1d3193096ab2cb2d

I am surprised you did not have issue with 4.2.1 because the header 
issues have been present ever since xl became the default/preferred 
toolstack, unless your Xen 4.2.1 came from a third-party.

Hello Ian,

On 06/30/13 17:30, Ian Murray wrote:
> On 30/06/13 10:49, Andreas Greve wrote:
>> Hello,
>>
>> in the last days I switched from Xen 4.2.1 to Xen 4.2.2 (source from 
>> git repository tag 4.2.2)
>>
>>
>> commit a125ec6a4a2f028f97f0bdd8946d347a36dbe76b
>> Author: Jan Beulich <jbeulich@suse.com>
>> Date:   Tue Apr 23 18:42:55 2013 +0200
>>
>>     update Xen version to 4.2.2
>>
>>
>> My "base system" is a debian squeeze. For Xen I use the xl
toolstack.
>>
>> Since updating to 4.2.2 the autosave and autorestore of domains dos 
>> not work any more.
>>
>> The reason is that there are some bugs in /etc/init.d/xendomains
>>
>> At the  end of the mail there is a diff of my corrections. The 
>> correction are only tested on my system under xl toolstack
>>
>
> There are some fixes going into Xen 4.3. My suggestion would be try 
> one commit after RC6, namely...
>
>
http://xenbits.xen.org/gitweb/?p=xen.git;a=commit;h=543a2657182dbb9237d1feeb1d3193096ab2cb2d
>
  I download the file by from git and give him a chance. It works nearly 
perfect.
On xendomains stop there was all ok.

On xendomains start it restores the saved domains but after that it 
tries to start them again and produce some error messages like domain is 
already running.

The reason is  the sed script in rdname() does not work with xl output.
I''ll changed it in the way as you have done with HEADCOMP (see the diff
below).
After that all was nice for me


------------------------------------------------
root@srv01:/etc/init.d# diff -u .xendomains.4.3.original xendomains
--- .xendomains.4.3.original    2013-06-30 20:54:14.000000000 +0200
+++ xendomains  2013-06-30 23:27:44.000000000 +0200
@@ -31,11 +31,13 @@

  CMD=${SBINDIR}/xm
  HEADCOMP="LinuxGuestRecord"
+RDNAMESED=''s/^.*(name \(.*\))$/\1/p''
  $CMD list &> /dev/null
  if test $? -ne 0
  then
         CMD=${SBINDIR}/xl
         HEADCOMP="Xen saved domain"
+        RDNAMESED=''s/^.*"name":
"\(.*\)",$/\1/p''
  fi

  $CMD list &> /dev/null
@@ -185,8 +187,8 @@
  # read name from xen config file
  rdname()
  {
-    NM=$($CMD create --quiet --dryrun --defconfig "$1" |
-         sed -n ''s/^.*(name \(.*\))$/\1/p'')
+    NM=$( $CMD create --quiet --dryrun --defconfig "$1" |
+         sed -n "${RDNAMESED}" )
  }

  rdnames()
----------------------------------------------------
>
> I am surprised you did not have issue with 4.2.1 because the header 
> issues have been present ever since xl became the default/preferred 
> toolstack, unless your Xen 4.2.1 came from a third-party.
Your are right I remember that there were some problems with 4.2.1 too. 
As I switched from 4.1 to 4.2.1.

Sorry at that time I had not much time. If I remember right, I fixed 
that in any way for me, but forget to report.

Thanks for your help. I learned a lot about bash (bla) seams to be equal 
to bla) in case instructions. That was new for me.


Best wishes


Andreas
>
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xen.org
> http://lists.xen.org/xen-users

>  I download the file by from git and give him a chance. It works 
> nearly perfect.
> On xendomains stop there was all ok.
>
> On xendomains start it restores the saved domains but after that it 
> tries to start them again and produce some error messages like domain 
> is already running.
>
It appears to be working fine for me. I am running 4.3 rc 6 + next 
commit. Rather than saying the output has changed for xl, I think you 
may be referring to the fact that xl now can output JSON as well as
xm''s
SXP. Having said this, xendomains works both properly for me both using 
JSON and SXP although I am not sure why the skipping of the autostart 
domains is working when I select JSON in xl.conf. Certainly the xl ... 
sed line fails when I execute it manually when using JSON.

I think your solution is not the right approach because xl can produce 
JSON and SXP format and that is defined in xl.conf. Your solution will 
have a problem when setting xl to produce SXP, I think.

root@xen6:/etc/xen/auto# service xendomains stop
Shutting down Xen domains: 
ubuntu-email(save)................................
  vpn2(save)....
  *   [done]
root@xen6:/etc/xen/auto# service xendomains start
Restoring Xen domains: ubuntu-email vpn2
Starting auto Xen domains: ubuntu-email(skip) vpn2(skip) *   [done]

Not tested against xm as I have no means to do so.

> The reason is  the sed script in rdname() does not work with xl output.
> I''ll changed it in the way as you have done with HEADCOMP (see the
> diff below).
> After that all was nice for me
>
>
> ------------------------------------------------
> root@srv01:/etc/init.d# diff -u .xendomains.4.3.original xendomains
> --- .xendomains.4.3.original    2013-06-30 20:54:14.000000000 +0200
> +++ xendomains  2013-06-30 23:27:44.000000000 +0200
> @@ -31,11 +31,13 @@
>
>  CMD=${SBINDIR}/xm
>  HEADCOMP="LinuxGuestRecord"
> +RDNAMESED=''s/^.*(name \(.*\))$/\1/p''
>  $CMD list &> /dev/null
>  if test $? -ne 0
>  then
>         CMD=${SBINDIR}/xl
>         HEADCOMP="Xen saved domain"
> +        RDNAMESED=''s/^.*"name":
"\(.*\)",$/\1/p''
>  fi
>
>  $CMD list &> /dev/null
> @@ -185,8 +187,8 @@
>  # read name from xen config file
>  rdname()
>  {
> -    NM=$($CMD create --quiet --dryrun --defconfig "$1" |
> -         sed -n ''s/^.*(name \(.*\))$/\1/p'')
> +    NM=$( $CMD create --quiet --dryrun --defconfig "$1" |
> +         sed -n "${RDNAMESED}" )
>  }
>
>  rdnames()
> ----------------------------------------------------
>
>>
>> I am surprised you did not have issue with 4.2.1 because the header 
>> issues have been present ever since xl became the default/preferred 
>> toolstack, unless your Xen 4.2.1 came from a third-party.
> Your are right I remember that there were some problems with 4.2.1 
> too. As I switched from 4.1 to 4.2.1.
>
> Sorry at that time I had not much time. If I remember right, I fixed 
> that in any way for me, but forget to report.
>
> Thanks for your help. I learned a lot about bash (bla) seams to be 
> equal to bla) in case instructions. That was new for me.
>
>
> Best wishes
>
>
> Andreas
>
>>
>> _______________________________________________
>> Xen-users mailing list
>> Xen-users@lists.xen.org
>> http://lists.xen.org/xen-users
>
>

On Sun, 2013-06-30 at 16:30 +0100, Ian Murray wrote:
> On 30/06/13 10:49, Andreas Greve wrote:
> > Hello,
> >
> > in the last days I switched from Xen 4.2.1 to Xen 4.2.2 (source from 
> > git repository tag 4.2.2)
> >
> >
> > commit a125ec6a4a2f028f97f0bdd8946d347a36dbe76b
> > Author: Jan Beulich <jbeulich@suse.com>
> > Date:   Tue Apr 23 18:42:55 2013 +0200
> >
> >     update Xen version to 4.2.2
> >
> >
> > My "base system" is a debian squeeze. For Xen I use the xl
toolstack.
> >
> > Since updating to 4.2.2 the autosave and autorestore of domains dos 
> > not work any more.
> >
> > The reason is that there are some bugs in /etc/init.d/xendomains
> >
> > At the  end of the mail there is a diff of my corrections. The 
> > correction are only tested on my system under xl toolstack
> >
> 
> There are some fixes going into Xen 4.3. My suggestion would be try one 
> commit after RC6, namely...
> 
>
http://xenbits.xen.org/gitweb/?p=xen.git;a=commit;h=543a2657182dbb9237d1feeb1d3193096ab2cb2d
Ian (J) can you add this commit to your list of things to backport to
4.2-testing please? I think we also want 13b37b6e0867 as a
precursor/independent fix. I think we might also want 0991e2dd346d.

Ian.

first sorry that I explained it bad but my English is not very well:

second I realize why you could not reproduce my problems under xl JSON 
format:
short: my symlink names in $XENDOMAINS_AUTO differ from the "real" 
domain name. Later more about that

Problem description detail:

File: /etc/init.d/xendomains (from 4.3 git 
commit543a2657182dbb9237d1feeb1d3193096ab2cb2d )

The sed "expression in function rdname()

rdname()
{
     NM=$($CMD create --quiet --dryrun --defconfig "$1" |
          sed -n ''s/^.*(name \(.*\))$/\1/p'')
}

     called by

is_running $dom

    called from

start

    in the if block

if contains_something "$XENDOMAINS_AUTO"
then
[....]
if [ $? -eq 0 ] || is_running $dom; then

     works only for the SXP format and not for the  JSON format. For 
JSON fromat the funktion is_running $dom  will nearly always return false


Example to show that:

root@srv01:~# xl create --quiet --dryrun --defconfig 
/etc/xen/auto/03_gnomedag | head -10
{
     "domid": null,
     "config": {
         "c_info": {
             "type": "pv",
             "hap": "<default>",
             "oos": "<default>",
             "ssidref": 0,
             "name": "gnomedag",
             "uuid": "2d33d9c8-7efe-4060-8e61-b47af2796a5c",
root@srv01:~#


root@srv01:~# xl create --quiet --dryrun --defconfig 
/etc/xen/auto/03_gnomedag \
| sed -n ''s/^.*(name \(.*\))$/\1/p''
root@srv01:~#


So $NM will always be empty and is_running() will nearly always return 
false;

is_running()
{
     rdname $1
     RC=1
     name=;id     while read LN; do
         parseln "$LN" || continue
         if test $id = 0; then continue; fi
         case $name in
             ($NM)
             RC=0
             ;;
         esac
     done < <($CMD list -l | grep "$LIST_GREP")
     return $RC
}


In contrast my sed "expression" will work for JSON format but not for 
SXP (Yes with that you are right. Sorry ! Up to now I did not know or 
have forgotten that xl supports both formats).

root@srv01:~# xl create --quiet --dryrun --defconfig 
/etc/xen/auto/03_gnomedag \
| sed -n ''s/^.*"name": "\(.*\)",$/\1/p''
gnomedag
root@srv01:~#

That is what I expected for JSON format.


I found why our both test have different results for the JSON fromat:

On my System (historical reason) the name of the symlinks in 
$XENDOMAINS_AUTO/* differ from the "real" domain name (example: domain
name: gnomedag   symlink name: /etc/xen/auto/03_gnomedag -> 
/etc/xen/gnomedag.cfg).

         shortdom=$(echo $dom | sed -n ''s/^.*\/\(.*\)$/\1/p'')
         echo $saved_domains | grep -w $shortdom > /dev/null
         if [ $? -eq 0 ] || is_running $dom; then
               echo -n "(skip)"
         else

In your configuration the restored domain names  are filtered by the 
echo ... | grep -w ... instruction. So
is_running $dom is never called for them. So you will never see the 
semantical error produced  by rdname().

In my system  the "real" domain name stored  in saved_domains differ 
from the value in $dom. So
is_running $dom is always called and always returns false. Which leads 
to a "second start" of the already restored domain.


In the moment I have no good idea how to combine the two needed sed 
expressions for the rdname() function

sed -n ''s/^.*(name \(.*\))$/\1/p''     #SXP

sed -n ''s/^.*"name": "\(.*\)",$/\1/p''
#JSON



On 07/01/13 02:06, Ian Murray wrote:
>
>>  I download the file by from git and give him a chance. It works 
>> nearly perfect.
>> On xendomains stop there was all ok.
>>
>> On xendomains start it restores the saved domains but after that it 
>> tries to start them again and produce some error messages like domain 
>> is already running.
>>
> It appears to be working fine for me. I am running 4.3 rc 6 + next 
> commit. Rather than saying the output has changed for xl, I think you 
> may be referring to the fact that xl now can output JSON as well as 
> xm''s SXP. Having said this, xendomains works both properly for me
both
> using JSON and SXP although I am not sure why the skipping of the 
> autostart domains is working when I select JSON in xl.conf. Certainly 
> the xl ... sed line fails when I execute it manually when using JSON.
>
> I think your solution is not the right approach because xl can produce 
> JSON and SXP format and that is defined in xl.conf. Your solution will 
> have a problem when setting xl to produce SXP, I think.
>
> root@xen6:/etc/xen/auto# service xendomains stop
> Shutting down Xen domains: 
> ubuntu-email(save)................................
>  vpn2(save)....
>  *   [done]
> root@xen6:/etc/xen/auto# service xendomains start
> Restoring Xen domains: ubuntu-email vpn2
> Starting auto Xen domains: ubuntu-email(skip) vpn2(skip) *   [done]
>
> Not tested against xm as I have no means to do so.
>
>
>> The reason is  the sed script in rdname() does not work with xl output.
>> I''ll changed it in the way as you have done with HEADCOMP (see
the
>> diff below).
>> After that all was nice for me
>>
>>
>> ------------------------------------------------
>> root@srv01:/etc/init.d# diff -u .xendomains.4.3.original xendomains
>> --- .xendomains.4.3.original    2013-06-30 20:54:14.000000000 +0200
>> +++ xendomains  2013-06-30 23:27:44.000000000 +0200
>> @@ -31,11 +31,13 @@
>>
>>  CMD=${SBINDIR}/xm
>>  HEADCOMP="LinuxGuestRecord"
>> +RDNAMESED=''s/^.*(name \(.*\))$/\1/p''
>>  $CMD list &> /dev/null
>>  if test $? -ne 0
>>  then
>>         CMD=${SBINDIR}/xl
>>         HEADCOMP="Xen saved domain"
>> +        RDNAMESED=''s/^.*"name":
"\(.*\)",$/\1/p''
>>  fi
>>
>>  $CMD list &> /dev/null
>> @@ -185,8 +187,8 @@
>>  # read name from xen config file
>>  rdname()
>>  {
>> -    NM=$($CMD create --quiet --dryrun --defconfig "$1" |
>> -         sed -n ''s/^.*(name \(.*\))$/\1/p'')
>> +    NM=$( $CMD create --quiet --dryrun --defconfig "$1" |
>> +         sed -n "${RDNAMESED}" )
>>  }
>>
>>  rdnames()
>> ----------------------------------------------------
>>
>>>
>>> I am surprised you did not have issue with 4.2.1 because the header
>>> issues have been present ever since xl became the default/preferred
>>> toolstack, unless your Xen 4.2.1 came from a third-party.
>> Your are right I remember that there were some problems with 4.2.1 
>> too. As I switched from 4.1 to 4.2.1.
>>
>> Sorry at that time I had not much time. If I remember right, I fixed 
>> that in any way for me, but forget to report.
>>
>> Thanks for your help. I learned a lot about bash (bla) seams to be 
>> equal to bla) in case instructions. That was new for me.
>>
>>
>> Best wishes
>>
>>
>> Andreas
>>
>>>
>>> _______________________________________________
>>> Xen-users mailing list
>>> Xen-users@lists.xen.org
>>> http://lists.xen.org/xen-users
>>
>>
>

----- Original Message -----
> From: Andreas Greve <greve-ml@a-greve.de>
> To: Ian Murray <murrayie@yahoo.co.uk>
> Cc: andreas.greve@a-greve.de; xen-users <xen-users@lists.xen.org>
> Sent: Monday, 1 July 2013, 14:59
> Subject: Re: [Xen-users] Xen 4.2.2 /etc/init.d/xendomains save and restore
 of domains does not work
> 
> first sorry that I explained it bad but my English is not very well:
> 
> second I realize why you could not reproduce my problems under xl JSON 
> format:
> short: my symlink names in $XENDOMAINS_AUTO differ from the
"real"
> domain name. Later more about that
> 
> Problem description detail:
> 
> File: /etc/init.d/xendomains (from 4.3 git 
> commit543a2657182dbb9237d1feeb1d3193096ab2cb2d )
> 
> The sed "expression in function rdname()
> 
> rdname()
> {
>      NM=$($CMD create --quiet --dryrun --defconfig "$1" |
>           sed -n ''s/^.*(name \(.*\))$/\1/p'')
> }
> 
>      called by
> 
> is_running $dom
> 
>     called from
> 
> start
> 
>     in the if block
> 
> if contains_something "$XENDOMAINS_AUTO"
> then
> [....]
> if [ $? -eq 0 ] || is_running $dom; then
> 
>      works only for the SXP format and not for the  JSON format. For 
> JSON fromat the funktion is_running $dom  will nearly always return false
> 
> 
> Example to show that:
> 
> root@srv01:~# xl create --quiet --dryrun --defconfig 
> /etc/xen/auto/03_gnomedag | head -10
> {
>      "domid": null,
>      "config": {
>          "c_info": {
>              "type": "pv",
>              "hap": "<default>",
>              "oos": "<default>",
>              "ssidref": 0,
>              "name": "gnomedag",
>              "uuid":
"2d33d9c8-7efe-4060-8e61-b47af2796a5c",
> root@srv01:~#
> 
> 
> root@srv01:~# xl create --quiet --dryrun --defconfig 
> /etc/xen/auto/03_gnomedag \
> | sed -n ''s/^.*(name \(.*\))$/\1/p''
> root@srv01:~#
> 
> 
> So $NM will always be empty and is_running() will nearly always return 
> false;
> 
> is_running()
> {
>      rdname $1
>      RC=1
>      name=;id>      while read LN; do
>          parseln "$LN" || continue
>          if test $id = 0; then continue; fi
>          case $name in
>              ($NM)
>              RC=0
>              ;;
>          esac
>      done < <($CMD list -l | grep "$LIST_GREP")
>      return $RC
> }
> 
> 
> In contrast my sed "expression" will work for JSON format but not
for
> SXP (Yes with that you are right. Sorry ! Up to now I did not know or 
> have forgotten that xl supports both formats).
I went through the exact same the week before last.
> 
> root@srv01:~# xl create --quiet --dryrun --defconfig 
> /etc/xen/auto/03_gnomedag \
> | sed -n ''s/^.*"name":
"\(.*\)",$/\1/p''
> gnomedag
> root@srv01:~#
> 
> That is what I expected for JSON format.
> 
> 
> I found why our both test have different results for the JSON fromat:
> 
> On my System (historical reason) the name of the symlinks in 
> $XENDOMAINS_AUTO/* differ from the "real" domain name (example:
domain
> 
> name: gnomedag   symlink name: /etc/xen/auto/03_gnomedag -> 
> /etc/xen/gnomedag.cfg).
> 
>          shortdom=$(echo $dom | sed -n 
> ''s/^.*\/\(.*\)$/\1/p'')
>          echo $saved_domains | grep -w $shortdom > /dev/null
>          if [ $? -eq 0 ] || is_running $dom; then
>                echo -n "(skip)"
>          else
> 
> In your configuration the restored domain names  are filtered by the 
> echo ... | grep -w ... instruction. So
> is_running $dom is never called for them. So you will never see the 
> semantical error produced  by rdname().
> 
> In my system  the "real" domain name stored  in saved_domains
differ
> from the value in $dom. So
> is_running $dom is always called and always returns false. Which leads 
> to a "second start" of the already restored domain.
> 
Right, I understand in principle. I will need to look at the script again to
understand in detail, but am happy to accept your word on it. :) What you say
makes sense to me.


> 
> In the moment I have no good idea how to combine the two needed sed 
> expressions for the rdname() function
> 
> sed -n ''s/^.*(name \(.*\))$/\1/p''     #SXP
> 
> sed -n ''s/^.*"name": "\(.*\)",$/\1/p''
> #JSON
> 
Could we not have somthing like (pseudo)
sedresult = cmd... | sed... (name
if sedresult = "" then
    sedresult = cmd ... | sed... "name":
fi

then either way sedresult should be blank for ones it should be and populated
for those where it should be?

A discussion on the dev list with David Sutton and Ian Campbell concluded that
the whole thing should probably be re-written from scratch because the SXP vs
JSON has made this script quite scrappy. There are still issues with the
handling of zombies, as $state isn''t populated and can''t be
using xl list -l

> 
> 
> On 07/01/13 02:06, Ian Murray wrote:
>> 
>>>   I download the file by from git and give him a chance. It works 
>>>  nearly perfect.
>>>  On xendomains stop there was all ok.
>>> 
>>>  On xendomains start it restores the saved domains but after that
it
>>>  tries to start them again and produce some error messages like
domain
>>>  is already running.
>>> 
>>  It appears to be working fine for me. I am running 4.3 rc 6 + next 
>>  commit. Rather than saying the output has changed for xl, I think you 
>>  may be referring to the fact that xl now can output JSON as well as 
>>  xm''s SXP. Having said this, xendomains works both properly
for me both
>>  using JSON and SXP although I am not sure why the skipping of the 
>>  autostart domains is working when I select JSON in xl.conf. Certainly 
>>  the xl ... sed line fails when I execute it manually when using JSON.
>> 
>>  I think your solution is not the right approach because xl can produce
>>  JSON and SXP format and that is defined in xl.conf. Your solution will
>>  have a problem when setting xl to produce SXP, I think.
>> 
>>  root@xen6:/etc/xen/auto# service xendomains stop
>>  Shutting down Xen domains: 
>>  ubuntu-email(save)................................
>>   vpn2(save)....
>>   *   [done]
>>  root@xen6:/etc/xen/auto# service xendomains start
>>  Restoring Xen domains: ubuntu-email vpn2
>>  Starting auto Xen domains: ubuntu-email(skip) vpn2(skip) *   [done]
>> 
>>  Not tested against xm as I have no means to do so.
>> 
>> 
>>>  The reason is  the sed script in rdname() does not work with xl
output.
>>>  I''ll changed it in the way as you have done with HEADCOMP
(see the
>>>  diff below).
>>>  After that all was nice for me
>>> 
>>> 
>>>  ------------------------------------------------
>>>  root@srv01:/etc/init.d# diff -u .xendomains.4.3.original
xendomains
>>>  --- .xendomains.4.3.original    2013-06-30 20:54:14.000000000
+0200
>>>  +++ xendomains  2013-06-30 23:27:44.000000000 +0200
>>>  @@ -31,11 +31,13 @@
>>> 
>>>   CMD=${SBINDIR}/xm
>>>   HEADCOMP="LinuxGuestRecord"
>>>  +RDNAMESED=''s/^.*(name \(.*\))$/\1/p''
>>>   $CMD list &> /dev/null
>>>   if test $? -ne 0
>>>   then
>>>          CMD=${SBINDIR}/xl
>>>          HEADCOMP="Xen saved domain"
>>>  +        RDNAMESED=''s/^.*"name": 
> "\(.*\)",$/\1/p''
>>>   fi
>>> 
>>>   $CMD list &> /dev/null
>>>  @@ -185,8 +187,8 @@
>>>   # read name from xen config file
>>>   rdname()
>>>   {
>>>  -    NM=$($CMD create --quiet --dryrun --defconfig "$1"
|
>>>  -         sed -n ''s/^.*(name \(.*\))$/\1/p'')
>>>  +    NM=$( $CMD create --quiet --dryrun --defconfig "$1"
|
>>>  +         sed -n "${RDNAMESED}" )
>>>   }
>>> 
>>>   rdnames()
>>>  ----------------------------------------------------
>>> 
>>>> 
>>>>  I am surprised you did not have issue with 4.2.1 because the
header
> 
>>>>  issues have been present ever since xl became the
default/preferred
> 
>>>>  toolstack, unless your Xen 4.2.1 came from a third-party.
>>>  Your are right I remember that there were some problems with 4.2.1
>>>  too. As I switched from 4.1 to 4.2.1.
>>> 
>>>  Sorry at that time I had not much time. If I remember right, I
fixed
>>>  that in any way for me, but forget to report.
>>> 
>>>  Thanks for your help. I learned a lot about bash (bla) seams to be
>>>  equal to bla) in case instructions. That was new for me.
>>> 
>>> 
>>>  Best wishes
>>> 
>>> 
>>>  Andreas
>>> 
>>>> 
>>>>  _______________________________________________
>>>>  Xen-users mailing list
>>>>  Xen-users@lists.xen.org
>>>>  http://lists.xen.org/xen-users
>>> 
>>> 
>> 
>

I have an idea.

assumptions:

SXP format:
root@srv01:~# xl create --quiet --dryrun --defconfig 
/etc/xen/auto/03_gnomedag | head -10
(domain
         (domid -1)
         (create_info)
         (hvm 0)
         (hap <default>)
         (oos <default>)
         (ssidref 0)
         (name gnomedag)
         (uuid <unknown>)
         (cpupool Pool-0)
[...]
)
root@srv01:~#

JSON format:
xl create --quiet --dryrun --defconfig /etc/xen/auto/03_gnomedag | head -10
{
     "domid": null,
     "config": {
         "c_info": {
             "type": "pv",
             "hap": "<default>",
             "oos": "<default>",
             "ssidref": 0,
             "name": "gnomedag",
             "uuid": "d9a9eba0-e7b8-4d16-ad6e-bb4cac05fd14",
[...]
}

Let use sed  pattern address ranges to distinguish between JSON and SXP.


xl create --quiet --dryrun --defconfig /etc/xen/auto/03_gnomedag \
|  sed -n -e ''/^[{]$/,/^[}]$/ { s/^.*"name":
"\(.*\)",$/\1/p }
/^[(]domain$/,/^[)]$/ { s/^.*(name \(.*\))$/\1/p }''

/^[{]$/,/^[}]$/  identify JSON
/^[(]domain$/,/^[)]$/  identify SXP

the line break between ...\1/p} and /^[(]domain$... is syntactical 
needed by sed.


sed -n -e ''/^[{]$/,$ { s/^.*"name": "\(.*\)",$/\1/p
}
/^[(]domain$/,$ { s/^.*(name \(.*\))$/\1/p }'' < json.out

in the example above  ,$ in the address range part  means until  end of file

For the JSON filter perhaps you can use somthing like this
/^[ ]*"c_info": [{]$/,$
as address range because `"c_info": {´ is comes close before
`"name:"
"gnomedag"´ but I
know not enough about xl to estimate if that is always true.

but further I have to look at the stop() function at the block

     if test "$XENDOMAINS_AUTO_ONLY" = "true"; then
        eval
        [...]
     fi

for bad side effects of the now working rdnames() -> rdname()



On 07/01/13 16:51, Ian Murray wrote:
>
>
>
> ----- Original Message -----
>> From: Andreas Greve<greve-ml@a-greve.de>
>> To: Ian Murray<murrayie@yahoo.co.uk>
>> Cc: andreas.greve@a-greve.de; xen-users<xen-users@lists.xen.org>
>> Sent: Monday, 1 July 2013, 14:59
>> Subject: Re: [Xen-users] Xen 4.2.2 /etc/init.d/xendomains save and
restore
>   of domains does not work
>> first sorry that I explained it bad but my English is not very well:
>>
>> second I realize why you could not reproduce my problems under xl JSON
>> format:
>> short: my symlink names in $XENDOMAINS_AUTO differ from the
"real"
>> domain name. Later more about that
>>
>> Problem description detail:
>>
>> File: /etc/init.d/xendomains (from 4.3 git
>> commit543a2657182dbb9237d1feeb1d3193096ab2cb2d )
>>
>> The sed "expression in function rdname()
>>
>> rdname()
>> {
>>       NM=$($CMD create --quiet --dryrun --defconfig "$1" |
>>            sed -n ''s/^.*(name \(.*\))$/\1/p'')
>> }
>>
>>       called by
>>
>> is_running $dom
>>
>>      called from
>>
>> start
>>
>>      in the if block
>>
>> if contains_something "$XENDOMAINS_AUTO"
>> then
>> [....]
>> if [ $? -eq 0 ] || is_running $dom; then
>>
>>       works only for the SXP format and not for the  JSON format. For
>> JSON fromat the funktion is_running $dom  will nearly always return
false
>>
>>
>> Example to show that:
>>
>> root@srv01:~# xl create --quiet --dryrun --defconfig
>> /etc/xen/auto/03_gnomedag | head -10
>> {
>>       "domid": null,
>>       "config": {
>>           "c_info": {
>>               "type": "pv",
>>               "hap": "<default>",
>>               "oos": "<default>",
>>               "ssidref": 0,
>>               "name": "gnomedag",
>>               "uuid":
"2d33d9c8-7efe-4060-8e61-b47af2796a5c",
>> root@srv01:~#
>>
>>
>> root@srv01:~# xl create --quiet --dryrun --defconfig
>> /etc/xen/auto/03_gnomedag \
>> | sed -n ''s/^.*(name \(.*\))$/\1/p''
>> root@srv01:~#
>>
>>
>> So $NM will always be empty and is_running() will nearly always return
>> false;
>>
>> is_running()
>> {
>>       rdname $1
>>       RC=1
>>       name=;id>>       while read LN; do
>>           parseln "$LN" || continue
>>           if test $id = 0; then continue; fi
>>           case $name in
>>               ($NM)
>>               RC=0
>>               ;;
>>           esac
>>       done<  <($CMD list -l | grep "$LIST_GREP")
>>       return $RC
>> }
>>
>>
>> In contrast my sed "expression" will work for JSON format but
not for
>> SXP (Yes with that you are right. Sorry ! Up to now I did not know or
>> have forgotten that xl supports both formats).
> I went through the exact same the week before last.
>
>> root@srv01:~# xl create --quiet --dryrun --defconfig
>> /etc/xen/auto/03_gnomedag \
>> | sed -n ''s/^.*"name":
"\(.*\)",$/\1/p''
>> gnomedag
>> root@srv01:~#
>>
>> That is what I expected for JSON format.
>>
>>
>> I found why our both test have different results for the JSON fromat:
>>
>> On my System (historical reason) the name of the symlinks in
>> $XENDOMAINS_AUTO/* differ from the "real" domain name
(example: domain
>>
>> name: gnomedag   symlink name: /etc/xen/auto/03_gnomedag ->
>> /etc/xen/gnomedag.cfg).
>>
>>           shortdom=$(echo $dom | sed -n
>> ''s/^.*\/\(.*\)$/\1/p'')
>>           echo $saved_domains | grep -w $shortdom>  /dev/null
>>           if [ $? -eq 0 ] || is_running $dom; then
>>                 echo -n "(skip)"
>>           else
>>
>> In your configuration the restored domain names  are filtered by the
>> echo ... | grep -w ... instruction. So
>> is_running $dom is never called for them. So you will never see the
>> semantical error produced  by rdname().
>>
>> In my system  the "real" domain name stored  in saved_domains
differ
>> from the value in $dom. So
>> is_running $dom is always called and always returns false. Which leads
>> to a "second start" of the already restored domain.
>>
> Right, I understand in principle. I will need to look at the script again
to understand in detail, but am happy to accept your word on it. :) What you say
makes sense to me.
>
>
>
>> In the moment I have no good idea how to combine the two needed sed
>> expressions for the rdname() function
>>
>> sed -n ''s/^.*(name \(.*\))$/\1/p''     #SXP
>>
>> sed -n ''s/^.*"name":
"\(.*\)",$/\1/p''
>> #JSON
>>
> Could we not have somthing like (pseudo)
> sedresult = cmd... | sed... (name
> if sedresult = "" then
>      sedresult = cmd ... | sed... "name":
> fi
>
> then either way sedresult should be blank for ones it should be and
populated for those where it should be?
>
> A discussion on the dev list with David Sutton and Ian Campbell concluded
that the whole thing should probably be re-written from scratch because the SXP
vs JSON has made this script quite scrappy. There are still issues with the
handling of zombies, as $state isn''t populated and can''t be
using xl list -l
>
>
>>
>> On 07/01/13 02:06, Ian Murray wrote:
>>>>    I download the file by from git and give him a chance. It
works
>>>>   nearly perfect.
>>>>   On xendomains stop there was all ok.
>>>>
>>>>   On xendomains start it restores the saved domains but after
that it
>>>>   tries to start them again and produce some error messages
like domain
>>>>   is already running.
>>>>
>>>   It appears to be working fine for me. I am running 4.3 rc 6 +
next
>>>   commit. Rather than saying the output has changed for xl, I think
you
>>>   may be referring to the fact that xl now can output JSON as well
as
>>>   xm''s SXP. Having said this, xendomains works both
properly for me both
>>>   using JSON and SXP although I am not sure why the skipping of the
>>>   autostart domains is working when I select JSON in xl.conf.
Certainly
>>>   the xl ... sed line fails when I execute it manually when using
JSON.
>>>
>>>   I think your solution is not the right approach because xl can
produce
>>>   JSON and SXP format and that is defined in xl.conf. Your solution
will
>>>   have a problem when setting xl to produce SXP, I think.
>>>
>>>   root@xen6:/etc/xen/auto# service xendomains stop
>>>   Shutting down Xen domains:
>>>   ubuntu-email(save)................................
>>>    vpn2(save)....
>>>    *   [done]
>>>   root@xen6:/etc/xen/auto# service xendomains start
>>>   Restoring Xen domains: ubuntu-email vpn2
>>>   Starting auto Xen domains: ubuntu-email(skip) vpn2(skip) *  
[done]
>>>
>>>   Not tested against xm as I have no means to do so.
>>>
>>>
>>>>   The reason is  the sed script in rdname() does not work with
xl output.
>>>>   I''ll changed it in the way as you have done with
HEADCOMP (see the
>>>>   diff below).
>>>>   After that all was nice for me
>>>>
>>>>
>>>>   ------------------------------------------------
>>>>   root@srv01:/etc/init.d# diff -u .xendomains.4.3.original
xendomains
>>>>   --- .xendomains.4.3.original    2013-06-30 20:54:14.000000000
+0200
>>>>   +++ xendomains  2013-06-30 23:27:44.000000000 +0200
>>>>   @@ -31,11 +31,13 @@
>>>>
>>>>    CMD=${SBINDIR}/xm
>>>>    HEADCOMP="LinuxGuestRecord"
>>>>   +RDNAMESED=''s/^.*(name \(.*\))$/\1/p''
>>>>    $CMD list&>  /dev/null
>>>>    if test $? -ne 0
>>>>    then
>>>>           CMD=${SBINDIR}/xl
>>>>           HEADCOMP="Xen saved domain"
>>>>   +        RDNAMESED=''s/^.*"name":
>> "\(.*\)",$/\1/p''
>>>>    fi
>>>>
>>>>    $CMD list&>  /dev/null
>>>>   @@ -185,8 +187,8 @@
>>>>    # read name from xen config file
>>>>    rdname()
>>>>    {
>>>>   -    NM=$($CMD create --quiet --dryrun --defconfig
"$1" |
>>>>   -         sed -n ''s/^.*(name
\(.*\))$/\1/p'')
>>>>   +    NM=$( $CMD create --quiet --dryrun --defconfig
"$1" |
>>>>   +         sed -n "${RDNAMESED}" )
>>>>    }
>>>>
>>>>    rdnames()
>>>>   ----------------------------------------------------
>>>>
>>>>>   I am surprised you did not have issue with 4.2.1 because
the header
>>>>>   issues have been present ever since xl became the
default/preferred
>>>>>   toolstack, unless your Xen 4.2.1 came from a third-party.
>>>>   Your are right I remember that there were some problems with
4.2.1
>>>>   too. As I switched from 4.1 to 4.2.1.
>>>>
>>>>   Sorry at that time I had not much time. If I remember right,
I fixed
>>>>   that in any way for me, but forget to report.
>>>>
>>>>   Thanks for your help. I learned a lot about bash (bla) seams
to be
>>>>   equal to bla) in case instructions. That was new for me.
>>>>
>>>>
>>>>   Best wishes
>>>>
>>>>
>>>>   Andreas
>>>>
>>>>>   _______________________________________________
>>>>>   Xen-users mailing list
>>>>>   Xen-users@lists.xen.org
>>>>>   http://lists.xen.org/xen-users
>>>>

----- Original Message -----
> From: Andreas Greve <greve-ml@a-greve.de>
> To: Ian Murray <murrayie@yahoo.co.uk>
> Cc: greve-ml@a-greve.de; xen-users <xen-users@lists.xen.org>;
andreas.greve@a-greve.de
> Sent: Monday, 1 July 2013, 17:44
> Subject: Re: [Xen-users] Xen 4.2.2 /etc/init.d/xendomains save and restore
 of domains does not work
> 
> I have an idea.
> 
> assumptions:
> 
> SXP format:
> root@srv01:~# xl create --quiet --dryrun --defconfig 
> /etc/xen/auto/03_gnomedag | head -10
> (domain
>          (domid -1)
>          (create_info)
>          (hvm 0)
>          (hap <default>)
>          (oos <default>)
>          (ssidref 0)
>          (name gnomedag)
>          (uuid <unknown>)
>          (cpupool Pool-0)
> [...]
> )
> root@srv01:~#
> 
> JSON format:
> xl create --quiet --dryrun --defconfig /etc/xen/auto/03_gnomedag | head -10
> {
>      "domid": null,
>      "config": {
>          "c_info": {
>              "type": "pv",
>              "hap": "<default>",
>              "oos": "<default>",
>              "ssidref": 0,
>              "name": "gnomedag",
>              "uuid":
"d9a9eba0-e7b8-4d16-ad6e-bb4cac05fd14",
> [...]
> }
> 
> Let use sed  pattern address ranges to distinguish between JSON and SXP.
> 
> 
> xl create --quiet --dryrun --defconfig /etc/xen/auto/03_gnomedag \
> |  sed -n -e ''/^[{]$/,/^[}]$/ { s/^.*"name": 
> "\(.*\)",$/\1/p }
> /^[(]domain$/,/^[)]$/ { s/^.*(name \(.*\))$/\1/p }''
> 
> /^[{]$/,/^[}]$/  identify JSON
> /^[(]domain$/,/^[)]$/  identify SXP
> 
> the line break between ...\1/p} and /^[(]domain$... is syntactical 
> needed by sed.
> 
> 
> sed -n -e ''/^[{]$/,$ { s/^.*"name": 
> "\(.*\)",$/\1/p }
> /^[(]domain$/,$ { s/^.*(name \(.*\))$/\1/p }'' < json.out
> 
> in the example above  ,$ in the address range part  means until  end of
file
> 
> For the JSON filter perhaps you can use somthing like this
> /^[ ]*"c_info": [{]$/,$
> as address range because `"c_info": {´ is comes close before 
> `"name:" 
> "gnomedag"´ but I
> know not enough about xl to estimate if that is always true.
> 
> but further I have to look at the stop() function at the block
> 
>      if test "$XENDOMAINS_AUTO_ONLY" = "true"; then
>         eval
>         [...]
>      fi
> 
> for bad side effects of the now working rdnames() -> rdname()
You''ve gone beyond my sed/regex skills so I can''t really
assist on the above. I think the best thing is for you to try to get it into a
working patch and I''d be happy to test it, then you could submit it to
the dev list for inclusion. Also beware that there is an bug in xl list -l in
sxp format. The domain id comes out as -1 for a running domain. I agreed to
adapt a commit as a backport when we discussed this on dev list.

> 
> 
> 
> On 07/01/13 16:51, Ian Murray wrote:
>> 
>> 
>> 
>>  ----- Original Message -----
>>>  From: Andreas Greve<greve-ml@a-greve.de>
>>>  To: Ian Murray<murrayie@yahoo.co.uk>
>>>  Cc: andreas.greve@a-greve.de;
xen-users<xen-users@lists.xen.org>
>>>  Sent: Monday, 1 July 2013, 14:59
>>>  Subject: Re: [Xen-users] Xen 4.2.2 /etc/init.d/xendomains save and
> restore
>>    of domains does not work
>>>  first sorry that I explained it bad but my English is not very
well:
>>> 
>>>  second I realize why you could not reproduce my problems under xl
JSON
>>>  format:
>>>  short: my symlink names in $XENDOMAINS_AUTO differ from the 
> "real"
>>>  domain name. Later more about that
>>> 
>>>  Problem description detail:
>>> 
>>>  File: /etc/init.d/xendomains (from 4.3 git
>>>  commit543a2657182dbb9237d1feeb1d3193096ab2cb2d )
>>> 
>>>  The sed "expression in function rdname()
>>> 
>>>  rdname()
>>>  {
>>>        NM=$($CMD create --quiet --dryrun --defconfig "$1"
|
>>>             sed -n ''s/^.*(name \(.*\))$/\1/p'')
>>>  }
>>> 
>>>        called by
>>> 
>>>  is_running $dom
>>> 
>>>       called from
>>> 
>>>  start
>>> 
>>>       in the if block
>>> 
>>>  if contains_something "$XENDOMAINS_AUTO"
>>>  then
>>>  [....]
>>>  if [ $? -eq 0 ] || is_running $dom; then
>>> 
>>>        works only for the SXP format and not for the  JSON format.
For
>>>  JSON fromat the funktion is_running $dom  will nearly always
return
> false
>>> 
>>> 
>>>  Example to show that:
>>> 
>>>  root@srv01:~# xl create --quiet --dryrun --defconfig
>>>  /etc/xen/auto/03_gnomedag | head -10
>>>  {
>>>        "domid": null,
>>>        "config": {
>>>            "c_info": {
>>>                "type": "pv",
>>>                "hap": "<default>",
>>>                "oos": "<default>",
>>>                "ssidref": 0,
>>>                "name": "gnomedag",
>>>                "uuid": 
> "2d33d9c8-7efe-4060-8e61-b47af2796a5c",
>>>  root@srv01:~#
>>> 
>>> 
>>>  root@srv01:~# xl create --quiet --dryrun --defconfig
>>>  /etc/xen/auto/03_gnomedag \
>>>  | sed -n ''s/^.*(name \(.*\))$/\1/p''
>>>  root@srv01:~#
>>> 
>>> 
>>>  So $NM will always be empty and is_running() will nearly always
return
>>>  false;
>>> 
>>>  is_running()
>>>  {
>>>        rdname $1
>>>        RC=1
>>>        name=;id>>>        while read LN; do
>>>            parseln "$LN" || continue
>>>            if test $id = 0; then continue; fi
>>>            case $name in
>>>                ($NM)
>>>                RC=0
>>>                ;;
>>>            esac
>>>        done<  <($CMD list -l | grep "$LIST_GREP")
>>>        return $RC
>>>  }
>>> 
>>> 
>>>  In contrast my sed "expression" will work for JSON
format but
> not for
>>>  SXP (Yes with that you are right. Sorry ! Up to now I did not know
or
>>>  have forgotten that xl supports both formats).
>>  I went through the exact same the week before last.
>> 
>>>  root@srv01:~# xl create --quiet --dryrun --defconfig
>>>  /etc/xen/auto/03_gnomedag \
>>>  | sed -n ''s/^.*"name": 
> "\(.*\)",$/\1/p''
>>>  gnomedag
>>>  root@srv01:~#
>>> 
>>>  That is what I expected for JSON format.
>>> 
>>> 
>>>  I found why our both test have different results for the JSON
fromat:
>>> 
>>>  On my System (historical reason) the name of the symlinks in
>>>  $XENDOMAINS_AUTO/* differ from the "real" domain name 
> (example: domain
>>> 
>>>  name: gnomedag   symlink name: /etc/xen/auto/03_gnomedag ->
>>>  /etc/xen/gnomedag.cfg).
>>> 
>>>            shortdom=$(echo $dom | sed -n
>>>  ''s/^.*\/\(.*\)$/\1/p'')
>>>            echo $saved_domains | grep -w $shortdom>  /dev/null
>>>            if [ $? -eq 0 ] || is_running $dom; then
>>>                  echo -n "(skip)"
>>>            else
>>> 
>>>  In your configuration the restored domain names  are filtered by
the
>>>  echo ... | grep -w ... instruction. So
>>>  is_running $dom is never called for them. So you will never see
the
>>>  semantical error produced  by rdname().
>>> 
>>>  In my system  the "real" domain name stored  in
saved_domains
> differ
>>>  from the value in $dom. So
>>>  is_running $dom is always called and always returns false. Which
leads
>>>  to a "second start" of the already restored domain.
>>> 
>>  Right, I understand in principle. I will need to look at the script
again
> to understand in detail, but am happy to accept your word on it. :) What
you say
> makes sense to me.
>> 
>> 
>> 
>>>  In the moment I have no good idea how to combine the two needed
sed
>>>  expressions for the rdname() function
>>> 
>>>  sed -n ''s/^.*(name \(.*\))$/\1/p''     #SXP
>>> 
>>>  sed -n ''s/^.*"name": 
> "\(.*\)",$/\1/p''
>>>  #JSON
>>> 
>>  Could we not have somthing like (pseudo)
>>  sedresult = cmd... | sed... (name
>>  if sedresult = "" then
>>       sedresult = cmd ... | sed... "name":
>>  fi
>> 
>>  then either way sedresult should be blank for ones it should be and 
> populated for those where it should be?
>> 
>>  A discussion on the dev list with David Sutton and Ian Campbell
concluded
> that the whole thing should probably be re-written from scratch because the
SXP
> vs JSON has made this script quite scrappy. There are still issues with the
> handling of zombies, as $state isn''t populated and can''t
be using xl
> list -l
>> 
>> 
>>> 
>>>  On 07/01/13 02:06, Ian Murray wrote:
>>>>>     I download the file by from git and give him a chance.
It
> works
>>>>>    nearly perfect.
>>>>>    On xendomains stop there was all ok.
>>>>> 
>>>>>    On xendomains start it restores the saved domains but
after
> that it
>>>>>    tries to start them again and produce some error
messages
> like domain
>>>>>    is already running.
>>>>> 
>>>>    It appears to be working fine for me. I am running 4.3 rc 6
+
> next
>>>>    commit. Rather than saying the output has changed for xl, I
think
> you
>>>>    may be referring to the fact that xl now can output JSON as
well
> as
>>>>    xm''s SXP. Having said this, xendomains works both
properly
> for me both
>>>>    using JSON and SXP although I am not sure why the skipping
of the
>>>>    autostart domains is working when I select JSON in xl.conf. 
> Certainly
>>>>    the xl ... sed line fails when I execute it manually when
using
> JSON.
>>>> 
>>>>    I think your solution is not the right approach because xl
can
> produce
>>>>    JSON and SXP format and that is defined in xl.conf. Your
solution
> will
>>>>    have a problem when setting xl to produce SXP, I think.
>>>> 
>>>>   root@xen6:/etc/xen/auto# service xendomains stop
>>>>    Shutting down Xen domains:
>>>>    ubuntu-email(save)................................
>>>>     vpn2(save)....
>>>>     *   [done]
>>>>   root@xen6:/etc/xen/auto# service xendomains start
>>>>    Restoring Xen domains: ubuntu-email vpn2
>>>>    Starting auto Xen domains: ubuntu-email(skip) vpn2(skip) *  
> [done]
>>>> 
>>>>    Not tested against xm as I have no means to do so.
>>>> 
>>>> 
>>>>>    The reason is  the sed script in rdname() does not work
with
> xl output.
>>>>>    I''ll changed it in the way as you have done
with HEADCOMP
> (see the
>>>>>    diff below).
>>>>>    After that all was nice for me
>>>>> 
>>>>> 
>>>>>    ------------------------------------------------
>>>>>   root@srv01:/etc/init.d# diff -u .xendomains.4.3.original 
> xendomains
>>>>>    --- .xendomains.4.3.original    2013-06-30
20:54:14.000000000
> +0200
>>>>>    +++ xendomains  2013-06-30 23:27:44.000000000 +0200
>>>>>    @@ -31,11 +31,13 @@
>>>>> 
>>>>>     CMD=${SBINDIR}/xm
>>>>>     HEADCOMP="LinuxGuestRecord"
>>>>>    +RDNAMESED=''s/^.*(name \(.*\))$/\1/p''
>>>>>     $CMD list&>  /dev/null
>>>>>     if test $? -ne 0
>>>>>     then
>>>>>            CMD=${SBINDIR}/xl
>>>>>            HEADCOMP="Xen saved domain"
>>>>>    +        RDNAMESED=''s/^.*"name":
>>>  "\(.*\)",$/\1/p''
>>>>>     fi
>>>>> 
>>>>>     $CMD list&>  /dev/null
>>>>>    @@ -185,8 +187,8 @@
>>>>>     # read name from xen config file
>>>>>     rdname()
>>>>>     {
>>>>>    -    NM=$($CMD create --quiet --dryrun --defconfig 
> "$1" |
>>>>>    -         sed -n ''s/^.*(name 
> \(.*\))$/\1/p'')
>>>>>    +    NM=$( $CMD create --quiet --dryrun --defconfig 
> "$1" |
>>>>>    +         sed -n "${RDNAMESED}" )
>>>>>     }
>>>>> 
>>>>>     rdnames()
>>>>>    ----------------------------------------------------
>>>>> 
>>>>>>    I am surprised you did not have issue with 4.2.1
because
> the header
>>>>>>    issues have been present ever since xl became the 
> default/preferred
>>>>>>    toolstack, unless your Xen 4.2.1 came from a
third-party.
>>>>>    Your are right I remember that there were some problems
with
> 4.2.1
>>>>>    too. As I switched from 4.1 to 4.2.1.
>>>>> 
>>>>>    Sorry at that time I had not much time. If I remember
right,
> I fixed
>>>>>    that in any way for me, but forget to report.
>>>>> 
>>>>>    Thanks for your help. I learned a lot about bash (bla)
seams
> to be
>>>>>    equal to bla) in case instructions. That was new for me.
>>>>> 
>>>>> 
>>>>>    Best wishes
>>>>> 
>>>>> 
>>>>>    Andreas
>>>>> 
>>>>>>    _______________________________________________
>>>>>>    Xen-users mailing list
>>>>>>   Xen-users@lists.xen.org
>>>>>>   http://lists.xen.org/xen-users
>>>>> 
>

On Mon, 2013-07-01 at 18:44 +0200, Andreas Greve wrote:
> I have an idea.
> 
> assumptions:
> 
> SXP format:
[...]

The SXP output format for xl is really just there as a backwards
compatibility layer for xm, for the benefit of people with local
scripting which relies on SXP.

For the xendomains use case perhaps we should just force the JSON output
mode when using xl and then select the appropriate sed snippet based on
the toolstack?

Even better would be to just do away with this madness of parsing JSON
or SXP in shell (keeping it only for xm compatibility) and add proper xl
commands to save and restore all domains to/from a given directory.

Ian.

> Even better would be to just do away with this madness of parsing JSON
> or SXP in shell (keeping it only for xm compatibility) and add proper xl
> commands to save and restore all domains to/from a given directory.
Xendomains does quite a bit of high level stuff.... it''s a
"convenience" script IMHO which not only deals with suspending and
restoring but also starting the auto domains and dealing with things that appear
in both (i.e. the skipping of auto start stuff that has just been restored).
From a design point of view, is it the right thing to do to move all that logic
in xl? ... or keep it in a seperate tool (but definitely re-written.)

On Tue, 2013-07-02 at 10:40 +0100, Ian Murray wrote:
> 
> > Even better would be to just do away with this madness of parsing JSON
> > or SXP in shell (keeping it only for xm compatibility) and add proper
xl
> > commands to save and restore all domains to/from a given directory.
> 
> Xendomains does quite a bit of high level stuff.... it''s a
> "convenience" script IMHO which not only deals with suspending
and
> restoring but also starting the auto domains and dealing with things
> that appear in both (i.e. the skipping of auto start stuff that has
> just been restored). From a design point of view, is it the right
> thing to do to move all that logic in xl? ... or keep it in a seperate
> tool (but definitely re-written.)
Probably one to RFC on the devel list before committing lots of code
too, but I think its the sort of thing we could accept.

Doing it in xl would be convenient, but a separate tool might also be
workable. The main point is "don''t parse JSON in shell"!

You could also consider implementing a more shell friendly output mode
though, which just produces FOO-separated lines of text with the name
and domid in them...

Ian.

----- Original Message -----
> From: Ian Campbell <Ian.Campbell@citrix.com>
> To: Ian Murray <murrayie@yahoo.co.uk>
> Cc: "greve-ml@a-greve.de" <greve-ml@a-greve.de>; xen-users
<xen-users@lists.xen.org>; "andreas.greve@a-greve.de"
<andreas.greve@a-greve.de>
> Sent: Tuesday, 2 July 2013, 10:44
> Subject: Re: [Xen-users] Xen 4.2.2 /etc/init.d/xendomains save and restore
 of domains does not work
> 
> On Tue, 2013-07-02 at 10:40 +0100, Ian Murray wrote:
>> 
>>  > Even better would be to just do away with this madness of parsing
JSON
>>  > or SXP in shell (keeping it only for xm compatibility) and add
proper
> xl
>>  > commands to save and restore all domains to/from a given
directory.
>> 
>>  Xendomains does quite a bit of high level stuff.... it''s a
>>  "convenience" script IMHO which not only deals with
suspending
> and
>>  restoring but also starting the auto domains and dealing with things
>>  that appear in both (i.e. the skipping of auto start stuff that has
>>  just been restored). From a design point of view, is it the right
>>  thing to do to move all that logic in xl? ... or keep it in a seperate
>>  tool (but definitely re-written.)
> 
> Probably one to RFC on the devel list before committing lots of code
> too, but I think its the sort of thing we could accept.
I think a discussion about what xendomains is supposed to do (especially dealing
with the more obscure stuff like zombies and system requests) would be a decent
place to start I think that would shape the technical response.
> 
> Doing it in xl would be convenient, but a separate tool might also be
> workable. The main point is "don''t parse JSON in shell"!
> 
I think everyone would agree with that.
> You could also consider implementing a more shell friendly output mode
> though, which just produces FOO-separated lines of text with the name
> and domid in them...
I think state would be needed as well.

> 
> Ian.
> 
I

On Tue, Jul 2, 2013 at 4:44 AM, Ian Campbell
<Ian.Campbell@citrix.com>wrote:
> On Tue, 2013-07-02 at 10:40 +0100, Ian Murray wrote:
> >
> > > Even better would be to just do away with this madness of parsing
JSON
> > > or SXP in shell (keeping it only for xm compatibility) and add
proper
> xl
> > > commands to save and restore all domains to/from a given
directory.
> >
> > Xendomains does quite a bit of high level stuff.... it''s a
> > "convenience" script IMHO which not only deals with
suspending and
> > restoring but also starting the auto domains and dealing with things
> > that appear in both (i.e. the skipping of auto start stuff that has
> > just been restored). From a design point of view, is it the right
> > thing to do to move all that logic in xl? ... or keep it in a seperate
> > tool (but definitely re-written.)
>
> Probably one to RFC on the devel list before committing lots of code
> too, but I think its the sort of thing we could accept.
>
> Doing it in xl would be convenient, but a separate tool might also be
> workable. The main point is "don''t parse JSON in shell"!
>
> You could also consider implementing a more shell friendly output mode
> though, which just produces FOO-separated lines of text with the name
> and domid in them...
>
> Ian.
>
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xen.org
> http://lists.xen.org/xen-users
>
Ian,

  The "Don''t parse JSON in shell" concept is why I tried the
experiment of
changing xendomains over to using xenstore commands instead; it seemed that
the majority of the data needed was available there (the state information
being the only part missing). This isn''t to say that this would be the
best
method, just that there is an example of another method already in the wild
(patch was posted to the user list for anyone interested)

Regards,

  David


_______________________________________________
Xen-users mailing list
Xen-users@lists.xen.org
http://lists.xen.org/xen-users

On Tue, 2013-07-02 at 09:09 -0500, David Sutton wrote:
>   The "Don''t parse JSON in shell" concept is why I tried
the
> experiment of changing xendomains over to using xenstore commands
> instead; it seemed that the majority of the data needed was available
> there
You''d also want to be sure that you were only accessing keys which were
part of the over ABI and not some toolstack internal thing, i.e. by
comparing with docs/misc/xenstore-paths.markdown.

For libxl we have a policy of never requiring toolstacks to need to go
and frob with xenstore directly, instead we provide an API over the top.
I think the same should probably apply to toolstack "consumers" such
as
the xendomains script.
> (the state information being the only part missing).
Right, this changes too dynamically to be in xenstore.

Ian.

>
>  The "Don''t parse JSON in shell" concept is why I tried
the experiment of changing xendomains over to using xenstore commands instead;
it seemed that the majority of the data needed was available there (the state
information being the only part missing). This isn''t to say that this
would be the best method, just that there is an example of another method
already in the wild (patch was posted to the user list for anyone interested)
>
>
In fact, David, I went back to your patch to take another look (after Ian
C''s last comments) at how the state information was handled because I
thought that was what you were trying to achieve with the stuff you did, but it
seems, as you say, it doesn''t. As I think xendomains is usable now, I
am not intending to spend much more time on it, so feel free to take up the case
for a re-write.

Ok   my actual first target is to fix what I have and thats at the 
moment the stable version 4.2.2 ( in that  i am a little bit self 
centered ;-) )  For that I have to live  with parsing sxp and json in 
shell and sed best as I can. (I don''t like it but bash and sed are nice
swiss army knifes)

I know nearly nothing concrete about xl internals and sorry very short 
at time. So I couldn''t read source code or things like that to become 
familiarly with the internals now.

But some idea from the point of an adminstrating end-user I have.

I think putting the logic for the handling in to an compiled is not a 
good idea

( a horrible example was the introduction of the mountall monster of the 
ubuntu distro. When it is working
it is ok. But they had an bug in it to deal with network mounts and 
because it is an binary you have no chance to fix and they need years to 
fix it. The elder ubuntu versions works with the classic script files 
and there you have a chance to adapt the logic to your personal needs or 
to correct a bug using only vi and your brain )

My future wish is to start and stop domains in an fixed order because 
they depends on each other in some way. If the starting logic is hard 
coded in xl I have no chance to implement such things for myself.

I would prefer at tool perhaps xl I can give simple questions and get 
short shell purchasable answers
examples

xl query <domid>|<domname> <attribute> and the answer is a
single value
or fails if domid is not running

xl query gnomedag domid  ---> produce     1 or ''''

xl queryall <attribute>  returns a list <domainame>
<value>\n<domainame>
value ....

perhaps getattribute is a better token for query

As a replacement for the create --dryrun ....
xl queryconf <configfile> <attribute> and the answer is a single
value

An other nice to have  will be to query for an attribute list with the 
answer <value1> <value2> <value3>
respectively <domainame> <value1> <value2>
...\n<domainame> <value1>
<value2> ...

The attribute should not be limited to configuration attributes. I wish 
to query dynamic values like domain state, cpu usage, disk usage and 
other things like that too.

Please excuse if the above is jolty worded.


Andreas

On 07/02/13 12:20, Ian Murray wrote:
>
>
>
> ----- Original Message -----
>> From: Ian Campbell<Ian.Campbell@citrix.com>
>> To: Ian Murray<murrayie@yahoo.co.uk>
>> Cc: "greve-ml@a-greve.de"<greve-ml@a-greve.de>;
xen-users<xen-users@lists.xen.org>;
"andreas.greve@a-greve.de"<andreas.greve@a-greve.de>
>> Sent: Tuesday, 2 July 2013, 10:44
>> Subject: Re: [Xen-users] Xen 4.2.2 /etc/init.d/xendomains save and
restore
>   of domains does not work
>> On Tue, 2013-07-02 at 10:40 +0100, Ian Murray wrote:
>>>   >  Even better would be to just do away with this madness of
parsing JSON
>>>   >  or SXP in shell (keeping it only for xm compatibility) and
add proper
>> xl
>>>   >  commands to save and restore all domains to/from a given
directory.
>>>
>>>   Xendomains does quite a bit of high level stuff.... it''s
a
>>>   "convenience" script IMHO which not only deals with
suspending
>> and
>>>   restoring but also starting the auto domains and dealing with
things
>>>   that appear in both (i.e. the skipping of auto start stuff that
has
>>>   just been restored). From a design point of view, is it the right
>>>   thing to do to move all that logic in xl? ... or keep it in a
seperate
>>>   tool (but definitely re-written.)
>> Probably one to RFC on the devel list before committing lots of code
>> too, but I think its the sort of thing we could accept.
> I think a discussion about what xendomains is supposed to do (especially
dealing with the more obscure stuff like zombies and system requests) would be a
decent place to start I think that would shape the technical response.
>
>> Doing it in xl would be convenient, but a separate tool might also be
>> workable. The main point is "don''t parse JSON in
shell"!
>>
> I think everyone would agree with that.
>
>> You could also consider implementing a more shell friendly output mode
>> though, which just produces FOO-separated lines of text with the name
>> and domid in them...
> I think state would be needed as well.
>
>
>> Ian.
>>
> I
>

On Wed, 2013-07-03 at 00:31 +0200, Andreas Greve wrote:
> My future wish is to start and stop domains in an fixed order because 
> they depends on each other in some way. If the starting logic is hard 
> coded in xl I have no chance to implement such things for myself.
FWIW I think this sounds like useful functionality for general use, so
whichever way you implement it it''d be great if it could go upstream
too.
> I would prefer at tool perhaps xl I can give simple questions and get 
> short shell purchasable answers
Yes, that was one of the ideas I proposed in
<1372758282.27768.57.camel@zakaz.uk.xensource.com>
> examples
> 
> xl query <domid>|<domname> <attribute> and the answer is
a single value
> or fails if domid is not running
> 
> xl query gnomedag domid  ---> produce     1 or ''''
> 
> xl queryall <attribute>  returns a list <domainame>
<value>\n<domainame>
> value ....
> 
> perhaps getattribute is a better token for query
> 
> As a replacement for the create --dryrun ....
> xl queryconf <configfile> <attribute> and the answer is a
single value
> 
> An other nice to have  will be to query for an attribute list with the 
> answer <value1> <value2> <value3>
> respectively <domainame> <value1> <value2>
...\n<domainame> <value1>
> <value2> ...
> 
> The attribute should not be limited to configuration attributes. I wish 
> to query dynamic values like domain state, cpu usage, disk usage and 
> other things like that too.
> 
> Please excuse if the above is jolty worded.
It all sounds plausible. We''ve just opened the 4.4 development branch
so
now is as good a time as any to start working on these things.

Ian.
> 
> 
> Andreas
> 
> On 07/02/13 12:20, Ian Murray wrote:
> >
> >
> >
> > ----- Original Message -----
> >> From: Ian Campbell<Ian.Campbell@citrix.com>
> >> To: Ian Murray<murrayie@yahoo.co.uk>
> >> Cc: "greve-ml@a-greve.de"<greve-ml@a-greve.de>;
xen-users<xen-users@lists.xen.org>;
"andreas.greve@a-greve.de"<andreas.greve@a-greve.de>
> >> Sent: Tuesday, 2 July 2013, 10:44
> >> Subject: Re: [Xen-users] Xen 4.2.2 /etc/init.d/xendomains save and
restore
> >   of domains does not work
> >> On Tue, 2013-07-02 at 10:40 +0100, Ian Murray wrote:
> >>>   >  Even better would be to just do away with this madness
of parsing JSON
> >>>   >  or SXP in shell (keeping it only for xm compatibility)
and add proper
> >> xl
> >>>   >  commands to save and restore all domains to/from a
given directory.
> >>>
> >>>   Xendomains does quite a bit of high level stuff....
it''s a
> >>>   "convenience" script IMHO which not only deals
with suspending
> >> and
> >>>   restoring but also starting the auto domains and dealing
with things
> >>>   that appear in both (i.e. the skipping of auto start stuff
that has
> >>>   just been restored). From a design point of view, is it the
right
> >>>   thing to do to move all that logic in xl? ... or keep it in
a seperate
> >>>   tool (but definitely re-written.)
> >> Probably one to RFC on the devel list before committing lots of
code
> >> too, but I think its the sort of thing we could accept.
> > I think a discussion about what xendomains is supposed to do
(especially dealing with the more obscure stuff like zombies and system
requests) would be a decent place to start I think that would shape the
technical response.
> >
> >> Doing it in xl would be convenient, but a separate tool might also
be
> >> workable. The main point is "don''t parse JSON in
shell"!
> >>
> > I think everyone would agree with that.
> >
> >> You could also consider implementing a more shell friendly output
mode
> >> though, which just produces FOO-separated lines of text with the
name
> >> and domid in them...
> > I think state would be needed as well.
> >
> >
> >> Ian.
> >>
> > I
> >
>

I have created a created a solution that save and restore domains with 
xl in sxp and json
format. It is only tested on my System:
     Debian squeeze
     Kernel 3.9.8 von kernel.org
     Xen 4.2.2

As the base for my working I took xendomains
  ( 
http://xenbits.xen.org/gitweb/?p=xen.git;a=commit;h=543a2657182dbb9237d1feeb1d3193096ab2cb2d
)

My System has 5 domains started by symlinks in /etc/xen/auto/*
the domain config file names differ from the domain name

domain fwgw     /etc/xen/auto/00_fwgw   ->   /etc/xen/fwgw.cfg  etc.

The new "version" contains some comments starting with #AG
This comments are a little bit unordered loud thinking about my idea an 
can be removed and
sometimes the old implementation. They can be removed


There is one question about an eval instruction where I am not sure if 
it is save:

+          #AG I am not a fan of using eval on variables with uncheck 
content.
+          #AG Some body could name a domain like "$(touch
/tmp/dummy)"
+          #AG that is possible  xl create --dryrun --defconfig ... 
except such a domain names
+          #AG I did  not try to create i finally
+          #AG I tryed some some combination to do a hack on this 
expression but I always failed.
+          #AG I think the `\´ in front of `$´ makes it save but I am 
not shure if this is always right.
             eval "
             case \"\$name\" in


Ordered auto starting of domains is not yet implemented. I think about 
using alphabetic order of
/etc/xen/auto/* for starting and reverse order for stopping. I think to 
implement more dependencies
would be to complicated for xendomains and a big project for its own.

The patch fix the sxp -1 domain id bug by using xl list -l <domainname>
...
The patch fix the empty state problem by using xl list <domainname> ...


I replaced the LIST_GREP expression by a function which translate the 
sxp/json formated output
from xl list -l  to a stream of lines of `id:<value> name:<value>´.
This
lines are handled by the new
readln implementation. First I thought about to using eval to set the 
variables. But I think the risk is to
high, if there is something bad in the values.

I only tested save and restore of domains.
I did not test  XENDOMAINS_SYSRQ, XENDOMAINS_MIGRATE and XENDOMAINS_SHUTDOWN
for different reasons ( not enough skill, no destination to migrate to, 
there is nothing changed in this blocks except that state in watchdog 
has now a value,  and my family becomes angry  about the up and downs of 
the server ;-) )

I would be glad if you give it a chance and test it.

Thanks

Question: Is it ok to use local for variable declarations in functions 
of ? Normally I prefer that.

Andreas

--vvvv xendomains patch -----------------------------
--- xendomains.4.3    2013-07-01 11:48:41.000000000 +0200
+++ xendomains    2013-07-04 23:55:03.000000000 +0200
@@ -26,15 +26,18 @@
  # Description:       Start / stop domains automatically when domain 0
  #                    boots / shuts down.
  ### END INIT INFO
+#AG set -x
+#AG set -v

  . /etc/xen/scripts/hotplugpath.sh

-CMD=${SBINDIR}/xm
+CMD="${SBINDIR}/xm"
  HEADCOMP="LinuxGuestRecord"
+
  $CMD list &> /dev/null
  if test $? -ne 0
  then
-    CMD=${SBINDIR}/xl
+        CMD=${SBINDIR}/xl
      HEADCOMP="Xen saved domain"
  fi

@@ -185,8 +188,35 @@
  # read name from xen config file
  rdname()
  {
-    NM=$($CMD create --quiet --dryrun --defconfig "$1" |
-         sed -n ''s/^.*(name \(.*\))$/\1/p'')
+    #AG the old implementation works only for sxp formated output
+    #AG NM=$($CMD create --quiet --dryrun --defconfig "$1" |
+    #AG     sed -n ''s/^.*(name \(.*\))$/\1/p'')
+
+    #AG The new implemnetation works with an pattern matching address range
+    #AG but only if the output of xl is in correct json or sxp format
+    #AG For json the output must start with a line containing only a `{´
+    #AG and ending with a line containing only a `}´. All lines bevor 
"start" and after "end"
+    #AG are ignored. For sxp the output must start with a line 
containing only a `(domain´
+    #AG and ending with a line containing only a `)´
+    #AG
+    #AG /^[{]$/,/^[}]$/ {...} is an pattern address range match it 
executes the block {...}
+    #AG for every line between the start pattern `^[{]$´ and the end 
pattern `^[}]$´
+    #AG including the lines with start and end pattern. `[{]´ is the 
same as `\{´ . I prefer
+    #AG the first one for sed scripts in shell scripts because it make 
its clear that it
+    #AG belongs to the sed script and not to the shell script.
+    #AG Here the first addres range  /^[{]$/,/^[}]$/ {...} gets aktive 
if the output is in
+    #AG JSON format and the second /^[(]domain$/,/^[)]$/ {...} when the 
output is in sxp fromat.
+    #AG Thanks to the author of 
http://www.grymoire.com/Unix/Sed.html#uh-25 for explaining sed
+    #AG so clear.
+    NM=$(
+      ${CMD} create --quiet --dryrun --defconfig "${1}" \
+      |  sed -n -e ''
+          /^[{]$/,/^[}]$/ {
+              s/^.*"name": "\(.*\)",$/\1/p }
+          /^[(]domain$/,/^[)]$/ {
+              s/^.*(name \(.*\))$/\1/p }
+      ''
+    )
  }

  rdnames()
@@ -206,22 +236,113 @@
      done
  }

-LIST_GREP=''(domain\|(domid\|(name\|^   
{$\|"name":\|"domid":''
-parseln()
-{
-    if [[ "$1" =~ ''(domain'' ]] || [[ "$1"
= "{" ]]; then
-        name=;id-    elif [[ "$1" =~ ''(name'' ]];
then
-        name=$(echo $1 | sed -e ''s/^.*(name \(.*\))$/\1/'')
-    elif [[ "$1" =~ ''(domid'' ]]; then
-        id=$(echo $1 | sed -e ''s/^.*(domid \(.*\))$/\1/'')
-    elif [[ "$1" =~ ''"name":'' ]]; then
-        name=$(echo $1 | sed -e ''s/^.*"name":
"\(.*\)",$/\1/'')
-    elif [[ "$1" =~ ''"domid":'' ]]; then
-        id=$(echo $1 | sed -e ''s/^.*"domid":
\(.*\),$/\1/'')
-    fi

-    [ -n "$name" -a -n "$id" ] && return 0 ||
return 1
+getDomStateByName () {
+    local _name+    local _id+    local _mem+    local _vcpu+    local _state+ 
local _time+    local _dummy+    while read _name _id _mem _vcpu _state _time
_dummy; do
+      [ -n "${_state}" ] && break;
+    done <<EOF
+$( ${CMD} list  "${1}" | sed -n -e ''/^Name[     ][     ]*ID[
][
]*Mem[     ][     ]*VCPUs[     ][     ]*State/! p'' )
+EOF
+    echo "${_state}"
+}
+
+getDomIdByName () {
+    $CMD list -l "${1}" \
+    | sed -n -e ''/^{$/,/^}$/ { s/^.*"domid":
"\(-\?[0-9][0-9]*\)",$/\1/p }
+        /^(domain$/,/^)$/ { s/^.*(domid \(-\?[0-9][0-9]*\))$/\1/p }''
+}
+
+#AG LIST_GREP=''(domain\|(domid\|(name\|^   
{$\|"name":\|"domid":''
+#AG list_domians() is the replacement for the expression  ($CMD list -l 
| grep "$LIST_GREP")
+#AG here the sxp and json parsing is impelmented
+#AG for every domain it produce a line with `id:<value>
name:<value>´
+#AG Perhaps it is possible to combine the two sed piped sed script to a 
single one but at
+#AG the moment I couldn'' see the solution.
+list_domains () {
+  #AG ^[     ] contain a space and a ht 0x09
+  $CMD list -l \
+  | sed -n -e ''
+      /^    {$/,/^    },$/ {
+        s/^[     ][     ]*"domid":
\(-\?[0-9][0-9]*\),$/id="\1" / p
+        s/^[     ][     ]*"name":
"\(.*\)",$/name="\1" / p
+        s/^    }\(,\|\)$/BREAK/ p
+      }
+      /^(domain$/,/^)$/ {
+        s/^[     ][     ]*(domid \(-\?[0-9][0-9]*\))$/id="\1" / p
+        s/^[     ][    ]*(name \(.*\))$/name="\1" / p
+        s/^)$/BREAK/ p
+      }
+    '' \
+  | sed -n  -e ''
+      /BREAK$/! { H }
+      /BREAK$/ {
+        x
+        s/\n//g
+        p
+        s/.*//
+        x
+      }
+    ''
+}
+
+#AG parseln()
+#AG {
+#AG     if [[ "$1" =~ ''(domain'' ]] || [[
"$1" = "{" ]]; then
+#AG         name=;id+#AG     elif [[ "$1" =~
''(name'' ]]; then
+#AG         name=$(echo $1 | sed -e ''s/^.*(name
\(.*\))$/\1/'')
+#AG     elif [[ "$1" =~ ''(domid'' ]]; then
+#AG         id=$(echo $1 | sed -e ''s/^.*(domid \(.*\))$/\1/'')
+#AG     elif [[ "$1" =~ ''"name":'' ]];
then
+#AG         name=$(echo $1 | sed -e ''s/^.*"name":
"\(.*\)",$/\1/'')
+#AG     elif [[ "$1" =~ ''"domid":'' ]];
then
+#AG        id=$(echo $1 | sed -e ''s/^.*"domid":
\(.*\),$/\1/'')
+#AG     fi
+#AG
+#AG     [ -n "$name" -a -n "$id" ] && return 0 ||
return 1
+#AG }
+parseln () {
+    id+    name+    state+    for _i in ${LN}; do
+        _k="${_i%=*}"
+    case "${_k}" in
+        (name)
+        _v="${_i#${_k}=}"
+        if [ "${_v}" != "${_i}" ]; then
+            _v="${_v//\"/}"
+            name="${_v}"
+        fi
+        ;;
+        (id)
+        _v="${_i#${_k}=}"
+        if [ "${_v}" != "${_i}" ]; then
+            _v="${_v//\"/}"
+            id="${_v}"
+        fi
+        ;;
+    esac
+    done
+    if [ -n "${name}" -a \( -z "${id}" -o "${id}"
= "-1" \) ]; then
+    v="$(getDomIdByName ${name})"
+    if [ -n "${v}" ]; then
+        id="${v}"
+    fi
+    fi
+    if [ -n "${name}" -a  -n "${id}" ]; then
+    state="$(getDomStateByName ${name})"
+    return 0
+    else
+    return 1
+    fi
  }

  is_running()
@@ -237,7 +358,10 @@
          RC=0
          ;;
      esac
-    done < <($CMD list -l | grep "$LIST_GREP")
+    done <<EOF
+$( list_domains )
+EOF
+    #AG done < <($CMD list -l | grep "$LIST_GREP")
      return $RC
  }

@@ -319,7 +443,10 @@
      if test "$state" != "-b---d" -a "$state" !=
"-----d"; then
          return 1;
      fi
-    done < <($CMD list -l | grep "$LIST_GREP")
+    done <<EOF
+$( list_domains )
+EOF
+    #AG done < <($CMD list -l | grep "$LIST_GREP")
      return 0
  }

@@ -368,6 +495,12 @@
      if test $id = 0; then continue; fi
      echo -n " $name"
      if test "$XENDOMAINS_AUTO_ONLY" = "true"; then
+          #AG I am not a fan of using eval on variables with uncheck 
content.
+          #AG Some body could name a domain like "$(touch
/tmp/dummy)"
+          #AG that is possible  xl create --dryrun --defconfig ... 
except such a domain names
+          #AG I did  not try to create i finally
+          #AG I tryed some some combination to do a hack on this 
expression but I always failed.
+          #AG I think the `\´ in front of `$´ makes it save but I am 
not shure if this is always right.
          eval "
          case \"\$name\" in
          ($NAMES)
@@ -450,7 +583,10 @@
          fi
          kill $WDOG_PID >/dev/null 2>&1
      fi
-    done < <($CMD list -l | grep "$LIST_GREP")
+    done <<EOF
+$( list_domains )
+EOF
+    #AG done < <($CMD list -l | grep "$LIST_GREP")

      # NB. this shuts down ALL Xen domains (politely), not just the ones in
      # AUTODIR/*
@@ -487,7 +623,10 @@
          return 0
          ;;
      esac
-    done < <($CMD list -l | grep "$LIST_GREP")
+    done <<EOF
+$( list_domains)
+EOF
+    #AG done < <($CMD list -l | grep "$LIST_GREP")
      return 1
  }

--^^^^ xendomains patch -----------------------------

On 07/01/13 19:09, Ian Murray wrote:
>
>
>
> ----- Original Message -----
>> From: Andreas Greve<greve-ml@a-greve.de>
>> To: Ian Murray<murrayie@yahoo.co.uk>
>> Cc: greve-ml@a-greve.de; xen-users<xen-users@lists.xen.org>;
andreas.greve@a-greve.de
>> Sent: Monday, 1 July 2013, 17:44
>> Subject: Re: [Xen-users] Xen 4.2.2 /etc/init.d/xendomains save and
restore
>   of domains does not work
>> I have an idea.
>>
>> assumptions:
>>
>> SXP format:
>> root@srv01:~# xl create --quiet --dryrun --defconfig
>> /etc/xen/auto/03_gnomedag | head -10
>> (domain
>>           (domid -1)
>>           (create_info)
>>           (hvm 0)
>>           (hap<default>)
>>           (oos<default>)
>>           (ssidref 0)
>>           (name gnomedag)
>>           (uuid<unknown>)
>>           (cpupool Pool-0)
>> [...]
>> )
>> root@srv01:~#
>>
>> JSON format:
>> xl create --quiet --dryrun --defconfig /etc/xen/auto/03_gnomedag | head
-10
>> {
>>       "domid": null,
>>       "config": {
>>           "c_info": {
>>               "type": "pv",
>>               "hap": "<default>",
>>               "oos": "<default>",
>>               "ssidref": 0,
>>               "name": "gnomedag",
>>               "uuid":
"d9a9eba0-e7b8-4d16-ad6e-bb4cac05fd14",
>> [...]
>> }
>>
>> Let use sed  pattern address ranges to distinguish between JSON and
SXP.
>>
>>
>> xl create --quiet --dryrun --defconfig /etc/xen/auto/03_gnomedag \
>> |  sed -n -e ''/^[{]$/,/^[}]$/ { s/^.*"name":
>> "\(.*\)",$/\1/p }
>> /^[(]domain$/,/^[)]$/ { s/^.*(name \(.*\))$/\1/p }''
>>
>> /^[{]$/,/^[}]$/  identify JSON
>> /^[(]domain$/,/^[)]$/  identify SXP
>>
>> the line break between ...\1/p} and /^[(]domain$... is syntactical
>> needed by sed.
>>
>>
>> sed -n -e ''/^[{]$/,$ { s/^.*"name":
>> "\(.*\)",$/\1/p }
>> /^[(]domain$/,$ { s/^.*(name \(.*\))$/\1/p }''<  json.out
>>
>> in the example above  ,$ in the address range part  means until  end of
file
>>
>> For the JSON filter perhaps you can use somthing like this
>> /^[ ]*"c_info": [{]$/,$
>> as address range because `"c_info": {´ is comes close before
>> `"name:"
>> "gnomedag"´ but I
>> know not enough about xl to estimate if that is always true.
>>
>> but further I have to look at the stop() function at the block
>>
>>       if test "$XENDOMAINS_AUTO_ONLY" = "true";
then
>>          eval
>>          [...]
>>       fi
>>
>> for bad side effects of the now working rdnames() ->  rdname()
> You''ve gone beyond my sed/regex skills so I can''t really
assist on the above. I think the best thing is for you to try to get it into a
working patch and I''d be happy to test it, then you could submit it to
the dev list for inclusion. Also beware that there is an bug in xl list -l in
sxp format. The domain id comes out as -1 for a running domain. I agreed to
adapt a commit as a backport when we discussed this on dev list.
>
>
>> [...]
>>