Xen users - Oct 2009 - HVM with stub domains fails at startup: ERROR Internal error: Could not open grant table interface (22 = Invalid argument)

Hi,

I tried to use stub domains for some performance testing with this interesting
environment. Unfortunately, I did not find so much information so far, so I am
not too surprised not having success here ;-) Anyway, some hints on what is
going on may be helpful. I got a very brief description from a German IT mag,
which I tried to follow, but I only got limited success:

My Environment:
Host: CentOS 5.3 64bit with Xen 3.4.1 RPMs from gitco
Domain: 32 bit DomU (SuSE based)

Config for DomU (named hvm_stub.conf):
name = "hvm_stub"
device_model = ''/usr/lib/xen/bin/stubdom-dm''
builder = "hvm"
kernel =  "/usr/lib/xen/boot/hvmloader"
memory = "1024"
disk =  [ ''tap:aio:/test/test.raw,ioemu:xvda,w'' ]
vif = [
''type=ioemu,mac=02:00:00:00:00:01'',''type=ioemu,bridge=eth0,mac=02:00:00:00:01:86''
]
sdl = 0
vnc = 1
vncviewer = 0

Config for HVM Stubdom (named hvm_stubdom-dm.conf):
kernel =  "/usr/lib/xen/boot/ioemu-stubdom.gz"
disk =  [ ''tap:aio:/test/testraw,ioemu:xvda,w'' ]
vif = [
''type=ioemu,mac=02:00:00:00:00:01'',''type=ioemu,bridge=eth0,mac=02:00:00:00:01:86''
]

Procedure:
1. start fs-backend in the background (as user root)
/usr/sbin/fs-backend
2. start VM with xm create -c hvm_stub.conf
3. error logs on command line, both DomUs are listed by xm, but stay with 0 CPU
time

Error (output from cmd line, no error at all in qemu- or xend logs):
Started domain hvm_stub (id=12)          
Path changed backend/vfs/exports/requests/13/0/frontend
Handling connection from dom=13, for export=0                           
Frontend found at: /local/domain/13/device/vfs/0 (gref=2047, evtchn=4)  
ERROR Internal error: Could not open grant table interface (22 = Invalid
argument)
ERROR: Couldn''t open gnttab!
Awaiting next connection.
Path changed backend/vfs/exports/requests/13/0/frontend 

Hints:
1. I thought I would have to use a 64bit stubdom device model
(/usr/lib64/xen/bin/stubdom-dm instead of /usr/lib/xen/bin/stubdom-dm), but that
does not exist. Might this be a problem?
2. is 64/32bit mixing allowed in this config?

Any help is appreciated, thanks in advance!

Regards,
Martin
________________________________________________________________
Neu: WEB.DE Doppel-FLAT mit Internet-Flatrate + Telefon-Flatrate
für nur 19,99 Euro/mtl.!* http://produkte.web.de/go/02/


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Martin,
> 1. start fs-backend in the background (as user root)
> /usr/sbin/fs-backend
> 2. start VM with xm create -c hvm_stub.conf
> 3. error logs on command line, both DomUs are listed by xm, but stay with 
> 0 CPU time
I have seen a similar problem here, running the latest pv_ops and 
xen-unstable. In my case, I am trying to run Windows Server x64 HVM and 
stubdom messages print to /var/log/xen/qemu-dm-testvds-dm.log:

domid: 101
Warning: vlan 0 is not connected to host network
Watching /local/domain/0/device-model/101/logdirty/cmd
Watching /local/domain/0/device-model/101/command
char device redirected to /dev/pts/10
char device redirected to /dev/pts/11
/usr/src/xen-unstable.hg/tools/ioemu-dir/hw/xen_blktap.c:628: Init blktap 
pipes
ERROR Internal error: Could not open grant table interface (22 = Invalid 
argument)
xen be core: xen be core: can''t open gnttab device
can''t open gnttab device
ERROR Internal error: Could not open grant table interface (22 = Invalid 
argument)
xen be core: xen be core: can''t open gnttab device
can''t open gnttab device
Log-dirty: no command yet.

Maybe we''re doing the same thing wrong here?

Have you tried with a LVM or file: backing store? Do normal HVM and PV 
domains work properly for you?

-John 


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

John,

John wrote:
> Have you tried with a LVM or file: backing store?
I tried it, but it does not show any difference at all - same error.
> Do normal HVM and PV domains work properly for you?
Yep, the domain starts fine as HVM and PV domain.

Regards,
Martin

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On Mon, Oct 5, 2009 at 4:01 PM, Martin Troester <TroyMcClure@web.de>
wrote:
> John,
>
> John wrote:
>>
>> Have you tried with a LVM or file: backing store?
>
> I tried it, but it does not show any difference at all - same error.
>
>> Do normal HVM and PV domains work properly for you?
>
> Yep, the domain starts fine as HVM and PV domain.
>
Someone please correct me if I''m wrong but I think the recent
RHEL/CentOS kernels do not support stub domains.


Keith Coleman

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Hi, Keith,

Keith Coleman wrote:
> Someone please correct me if I''m wrong but I think the recent
> RHEL/CentOS kernels do not support stub domains.
That would be interesting to know... Which brings me to the next 
beginner''s questions: what options do I need to successfully compile 
Dom0 and/or DomU kernels with working stub support? Do both need 
modifications? Is there an easy way to rebuild the kernel(s), based on 
the RHEL/CentOS source rpms? Any recent sources of documentation?

Thanks for all hints on this topic!

Regards,
Martin

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On Tue, Oct 06, 2009 at 09:30:48PM +0200, Martin Troester
wrote:
> Hi, Keith,
> 
> Keith Coleman wrote:
> >Someone please correct me if I''m wrong but I think the recent
> >RHEL/CentOS kernels do not support stub domains.
> That would be interesting to know... Which brings me to the next 
> beginner''s questions: what options do I need to successfully
compile
> Dom0 and/or DomU kernels with working stub support? Do both need 
> modifications? Is there an easy way to rebuild the kernel(s), based on 
> the RHEL/CentOS source rpms? Any recent sources of documentation?
> 
> Thanks for all hints on this topic!
> 
I think it''s mainly related to compiling new version of Xen hypervisor,
and tools, including the stubdom itself. 

The same (non-modified) dom0 kernel might work, and definitely same domU kernels
should work.

-- Pasi


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Pasi,

 > I think it''s mainly related to compiling new version of Xen
ypervisor,
 > and tools, including the stubdom itself.
 > The same (non-modified) dom0 kernel might work, and definitely same
 > domU kernels should work.
any idea how to find out if the xen 3.4.1 hypervisor from gitco.de/repo 
has been built with stub dom support?
I am not using the CentOS Hypervisor, but this one, so if it''s only the
hypervisor, this would be a contradiction to Keith''s assumption that 
CentOS does not support stub doms. I thought stub dom support would be a 
basic feature on Xen 3.4.1, nothing to be compiled in as an extra 
feature, but this seems a wrong assumption.

Regards,
Martin

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users