Xen users - Jul 2005 - Xen questions - Swap, networking and errors

Hi,

This is my first time trying Xen.  I have installed Xen-2.0.6, on a
Debian Sarge server.  I have a few questions that I haven''t been able
to
find answers to in the documentation or FAQs.

1)When I boot xen0, the specified amount of RAM shows up in top
correctly, but the swap partitions I have are not activated.  When I
attempt to activate swap myself, I recieve the following error message:
	xen:/# swapon /dev/hde2
	swapon: /dev/hde2: No such device or address
The server has a mirrored RAID1 setup, and both MD devices are detected
and mounted properly.  However, I am confused about why the kernel does
not see the hard disk partitions themselves.  The disks are SATA drives,
and so at first I thought perhaps it didn''t recongize the drives, but
if
that were the case, I don''t think either MD device would be detected
properly.  Is there a boot option that I have not seen to address this
issue?

As a side note, what is a proper amount of RAM needed for xen0?  I have
not found any documentation really stating how much RAM is need for the
priveledged Xen instance.  The server itself has 4GB of RAM.  Should
xen0 have all 4GB of RAM to partition to the xenU domains?

2)With the default xend config, when I start xend the bridge device
xen-br0 is created.  However, after that I lose all network activity to
xen0.  I noticed that no vif devices were created, only xen-br0.
Initially, xen-br0 doesn''t have an IP.  I''ve tried setting it
to the IP
of eth0, and a 192.168.0.x IP, but I still cannot get networking up.

I changed the xend config to use the network-route rather than network,
and when I start xend, networking doesn''t go down, but xen-br0 is not
created.  I am not sure why the default network config is not working.
Do I need to make changes to the routing tables?

3)When I create my test xenU instance, it appears to crash.  Here is the
output from xend-debug.log:

VIRTUAL MEMORY ARRANGEMENT:
 Loaded kernel: c0100000->c0341b84
 Init. ramdisk: c0342000->c0342000
 Phys-Mach map: c0342000->c0362000
 Page tables:   c0362000->c0364000
 Start info:    c0364000->c0365000
 Boot stack:    c0365000->c0366000
 TOTAL:         c0000000->c0400000
 ENTRY ADDRESS: c0100000
ioctl: LOOP_SET_FD: Device or resource busy
vif-route up vif=vif1.0 domain=TestDomain mac=aa:00:00:62:e9:12
bridge=xen-br0
recv_fe_driver_status> {''status'': 1}

recv_fe_driver_status>

recv_fe_interface_connect {''tx_shmem_frame'': 6874,
''rx_shmem_frame'':
6873, ''handle'': 0}
VirqClient.virqReceived> 4
vif-route down vif=vif1.0 domain=TestDomain mac=aa:00:00:62:e9:12
bridge=xen-br0

I am not sure why the error message LOOP_SET_FD: Device or resource busy
appears, since the loopback device I created the root filesystem on is
not mounted or being used.

I have attached my xend-config and xenU config that appears to be
crashing.  Any help that can be offered is appreciated, especially links
to documentation that I might have missed.

Regards,

Dan Lang
-- 
Daniel Lang

Systems Administrator
SevenL Networks
416.642.1235
1.866.639.5123



_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

> This is my first time trying Xen.  I have installed Xen-2.0.6, on a
> Debian Sarge server.  I have a few questions that I haven''t been
able to
> find answers to in the documentation or FAQs.
OK, welcome to the community! I''ll try and answer your questions. If 
there''s anything you feel should be in a FAQ, feel free to get a Wiki 
account and add it at http://wiki.xensource.com.
> 1)When I boot xen0, the specified amount of RAM shows up in top
> correctly, but the swap partitions I have are not activated.  When I
> attempt to activate swap myself, I recieve the following error message:
> 	xen:/# swapon /dev/hde2
> 	swapon: /dev/hde2: No such device or address
> The server has a mirrored RAID1 setup, and both MD devices are detected
> and mounted properly.  However, I am confused about why the kernel does
> not see the hard disk partitions themselves.  The disks are SATA drives,
> and so at first I thought perhaps it didn''t recongize the drives,
but if
> that were the case, I don''t think either MD device would be
detected
> properly.  Is there a boot option that I have not seen to address this
Are all the drives in the machine the same? Are other partitions on hde in 
the RAID configuration.

If the drives are on different controllers (e.g. a couple of PATA and a 
SATA, as on some systems) then the default kernel config may not be 
building the drivers for hde but still builds enough drivers for the MD 
devices to work.
> As a side note, what is a proper amount of RAM needed for xen0?  I have
> not found any documentation really stating how much RAM is need for the
> priveledged Xen instance.  The server itself has 4GB of RAM.  Should
> xen0 have all 4GB of RAM to partition to the xenU domains?
RAM that you give to dom0 can''t be allocated to domUs. 256MB should 
probably be enough for a slim dom0. If you use LVM snapshots that''ll
need
to be larger though.

You can use the "balloon driver" to reduce the size of dom0 after
booting
if you need more memory for other domains.
> 2)With the default xend config, when I start xend the bridge device
> xen-br0 is created.  However, after that I lose all network activity to
> xen0.  I noticed that no vif devices were created, only xen-br0.
> Initially, xen-br0 doesn''t have an IP.  I''ve tried
setting it to the IP
> of eth0, and a 192.168.0.x IP, but I still cannot get networking up.
Sounds a bit broken... What should happen is that eth0''s IP is
transferred
to xen-br0.

Do you have any existing bridge setup? Anything else unusual in your 
networking config?
> I changed the xend config to use the network-route rather than network,
> and when I start xend, networking doesn''t go down, but xen-br0 is
not
> created.  I am not sure why the default network config is not working.
> Do I need to make changes to the routing tables?
You won''t get a xen-br0 if you''re using the routed config -
you don''t need
it. Don''t know about the routing tables as I''ve never used the
routed
config myself.

<snip>
> ioctl: LOOP_SET_FD: Device or resource busy
<snip>
> vif-route up vif=vif1.0 domain=TestDomain mac=aa:00:00:62:e9:12
> I am not sure why the error message LOOP_SET_FD: Device or resource busy
> appears, since the loopback device I created the root filesystem on is
> not mounted or being used.
What does ls -l /dev/loop* give you?

Cheers,
Mark
> I have attached my xend-config and xenU config that appears to be
> crashing.  Any help that can be offered is appreciated, especially links
> to documentation that I might have missed.
> 
> Regards,
> 
> Dan Lang
> 
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Daniel Lang wrote:
>1)When I boot xen0, the specified amount of RAM shows up in top
>correctly, but the swap partitions I have are not activated.  When I
>attempt to activate swap myself, I recieve the following error message:
>	xen:/# swapon /dev/hde2
>	swapon: /dev/hde2: No such device or address
>The server has a mirrored RAID1 setup, and both MD devices are detected
>and mounted properly.  However, I am confused about why the kernel does
>not see the hard disk partitions themselves.  The disks are SATA drives,
>and so at first I thought perhaps it didn''t recongize the drives,
but if
>that were the case, I don''t think either MD device would be
detected
>properly.  Is there a boot option that I have not seen to address this
>issue?
>
Make sure the device exists with #fdisk -l /dev/hde
I guess it''s possible that your drives are now allocated a different
node.
On your original kernel, it would appear from what you''re saying,
that your SATA drives all came up as hdX.
However, on you dom0 kernel, they are sdX.
>As a side note, what is a proper amount of RAM needed for xen0?  I have
>not found any documentation really stating how much RAM is need for the
>priveledged Xen instance.  The server itself has 4GB of RAM.  Should
>xen0 have all 4GB of RAM to partition to the xenU domains?
>  
>
The kernel stanza in grub''s menu.lst specifies the amount of memory for
dom0 to use, not how much it has to share. I''m not certain about the
actual
memory overheads involved in running domains, but I''ve found that 128MB
is more than enough to run a couple of domains, NFS server, etc.
>2)With the default xend config, when I start xend the bridge device
>xen-br0 is created.  However, after that I lose all network activity to
>xen0.  I noticed that no vif devices were created, only xen-br0.
>Initially, xen-br0 doesn''t have an IP.  I''ve tried setting
it to the IP
>of eth0, and a 192.168.0.x IP, but I still cannot get networking up.
>
>I changed the xend config to use the network-route rather than network,
>and when I start xend, networking doesn''t go down, but xen-br0 is
not
>created.  I am not sure why the default network config is not working.
>Do I need to make changes to the routing tables?
>  
>
I initially had this problem too, and like you, I switched to routing
until I
got everything else working and realised that routing just didn''t cut
it!
I''d suggest you get everything else going first.

[snip]
>disk = [ ''file:/clients/test/root_fs,sda1,w'',
''file:/clients/test/swap_fs,sda2,w'' ]
>  
>
[snip]
># Set root device.
>root = "/dev/hda1 ro"
>  
>
[snip]

Your root device seems incorrectly set. Your device export line suggests
that the line should be:
root = "/dev/sda1 ro"


Hope that helps, and good luck :)

Marcus.

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

M.A. Williamson wrote:
>> This is my first time trying Xen.  I have installed Xen-2.0.6, on a
>> Debian Sarge server.  I have a few questions that I haven''t
been able to
>> find answers to in the documentation or FAQs.
> 
> 
> OK, welcome to the community! I''ll try and answer your questions.
If
> there''s anything you feel should be in a FAQ, feel free to get a
Wiki
> account and add it at http://wiki.xensource.com.
> 
>> 1)When I boot xen0, the specified amount of RAM shows up in top
>> correctly, but the swap partitions I have are not activated.  When I
>> attempt to activate swap myself, I recieve the following error message:
>>     xen:/# swapon /dev/hde2
>>     swapon: /dev/hde2: No such device or address
>> The server has a mirrored RAID1 setup, and both MD devices are detected
>> and mounted properly.  However, I am confused about why the kernel does
>> not see the hard disk partitions themselves.  The disks are SATA
drives,
>> and so at first I thought perhaps it didn''t recongize the
drives, but if
>> that were the case, I don''t think either MD device would be
detected
>> properly.  Is there a boot option that I have not seen to address this
> 
> 
> Are all the drives in the machine the same? Are other partitions on hde
> in the RAID configuration.
> 
> If the drives are on different controllers (e.g. a couple of PATA and a
> SATA, as on some systems) then the default kernel config may not be
> building the drivers for hde but still builds enough drivers for the MD
> devices to work.
> 
>> As a side note, what is a proper amount of RAM needed for xen0?  I have
>> not found any documentation really stating how much RAM is need for the
>> priveledged Xen instance.  The server itself has 4GB of RAM.  Should
>> xen0 have all 4GB of RAM to partition to the xenU domains?
> 
> 
I went back and checked /proc/partitions.  Under a normal kernel, all of
the partitions show up as /dev/hde* and /dev/hdg*, but with the Xen
kernel, all of the partitions are /dev/sda* and /dev/sdb*  Now that
I''ve
found the correct device names, swap is turned on.
> RAM that you give to dom0 can''t be allocated to domUs. 256MB
should
> probably be enough for a slim dom0. If you use LVM snapshots
that''ll
> need to be larger though.
> 
> You can use the "balloon driver" to reduce the size of dom0 after
> booting if you need more memory for other domains.
> 
>> 2)With the default xend config, when I start xend the bridge device
>> xen-br0 is created.  However, after that I lose all network activity to
>> xen0.  I noticed that no vif devices were created, only xen-br0.
>> Initially, xen-br0 doesn''t have an IP.  I''ve tried
setting it to the IP
>> of eth0, and a 192.168.0.x IP, but I still cannot get networking up.
> 
> 
> Sounds a bit broken... What should happen is that eth0''s IP is
> transferred to xen-br0.
> 
> Do you have any existing bridge setup? Anything else unusual in your
> networking config?
This is a brand new server meant to run Xen.  It has an Intel gigabit
NIC, but only one.  I was using the default configuration from a source
install of Xen-2.0.6.   There are/were not any other bridge devices
working.  The thing that stood out to me was that no vif devices were
created, which from the documentation I''ve found, seems unusual.
> 
>> I changed the xend config to use the network-route rather than network,
>> and when I start xend, networking doesn''t go down, but xen-br0
is not
>> created.  I am not sure why the default network config is not working.
>> Do I need to make changes to the routing tables?
> 
> 
> You won''t get a xen-br0 if you''re using the routed config
- you don''t
> need it. Don''t know about the routing tables as I''ve
never used the
> routed config myself.
> 
> <snip>
> 
>> ioctl: LOOP_SET_FD: Device or resource busy
> 
> <snip>
> 
>> vif-route up vif=vif1.0 domain=TestDomain mac=aa:00:00:62:e9:12
> 
> 
>> I am not sure why the error message LOOP_SET_FD: Device or resource
busy
>> appears, since the loopback device I created the root filesystem on is
>> not mounted or being used.
> 
> 
> What does ls -l /dev/loop* give you?
xen:~# ls -l /dev/loop*
brw-rw----  1 root disk 7, 0 Jun 29 16:32 /dev/loop0
brw-rw----  1 root disk 7, 1 Jun 29 16:32 /dev/loop1
brw-rw----  1 root disk 7, 2 Jun 29 16:32 /dev/loop2
brw-rw----  1 root disk 7, 3 Jun 29 16:32 /dev/loop3
brw-rw----  1 root disk 7, 4 Jun 29 16:32 /dev/loop4
brw-rw----  1 root disk 7, 5 Jun 29 16:32 /dev/loop5
brw-rw----  1 root disk 7, 6 Jun 29 16:32 /dev/loop6
brw-rw----  1 root disk 7, 7 Jun 29 16:32 /dev/loop7

Thanks for the help so far.  One problem taken care of.  Are there
developer docs that might help explain the reason for the xenU instance
crashing?

Regards,

Dan Lang

-- 
Daniel Lang

Systems Administrator
SevenL Networks
416.642.1235
1.866.639.5123

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Marcus Brown wrote:
> 
> Daniel Lang wrote:
> 
> 
>>1)When I boot xen0, the specified amount of RAM shows up in top
>>correctly, but the swap partitions I have are not activated.  When I
>>attempt to activate swap myself, I recieve the following error message:
>>	xen:/# swapon /dev/hde2
>>	swapon: /dev/hde2: No such device or address
>>The server has a mirrored RAID1 setup, and both MD devices are detected
>>and mounted properly.  However, I am confused about why the kernel does
>>not see the hard disk partitions themselves.  The disks are SATA drives,
>>and so at first I thought perhaps it didn''t recongize the
drives, but if
>>that were the case, I don''t think either MD device would be
detected
>>properly.  Is there a boot option that I have not seen to address this
>>issue?
>>
> 
> Make sure the device exists with #fdisk -l /dev/hde
> I guess it''s possible that your drives are now allocated a
different node.
> On your original kernel, it would appear from what you''re saying,
> that your SATA drives all came up as hdX.
> However, on you dom0 kernel, they are sdX.
> 
Yes, I discovered this.  Thank you pointing it out.
> 
>>As a side note, what is a proper amount of RAM needed for xen0?  I have
>>not found any documentation really stating how much RAM is need for the
>>priveledged Xen instance.  The server itself has 4GB of RAM.  Should
>>xen0 have all 4GB of RAM to partition to the xenU domains?
>> 
>>
> 
> The kernel stanza in grub''s menu.lst specifies the amount of
memory for
> dom0 to use, not how much it has to share. I''m not certain about
the actual
> memory overheads involved in running domains, but I''ve found that
128MB
> is more than enough to run a couple of domains, NFS server, etc.
> 
> 
>>2)With the default xend config, when I start xend the bridge device
>>xen-br0 is created.  However, after that I lose all network activity to
>>xen0.  I noticed that no vif devices were created, only xen-br0.
>>Initially, xen-br0 doesn''t have an IP.  I''ve tried
setting it to the IP
>>of eth0, and a 192.168.0.x IP, but I still cannot get networking up.
>>
>>I changed the xend config to use the network-route rather than network,
>>and when I start xend, networking doesn''t go down, but xen-br0
is not
>>created.  I am not sure why the default network config is not working.
>>Do I need to make changes to the routing tables?
>> 
>>
> 
> I initially had this problem too, and like you, I switched to routing
> until I
> got everything else working and realised that routing just didn''t
cut it!
> I''d suggest you get everything else going first.
> 
I''ve got the test domain running now, and networking for xenU
doesn''t
seem to work.  Here is the current output from ifconfig:

xen:~# ifconfig
eth0      Link encap:Ethernet  HWaddr 00:0E:0C:3D:C2:7F
          inet addr:204.225.94.83  Bcast:204.225.94.127
Mask:255.255.255.192
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:121293 errors:0 dropped:0 overruns:0 frame:0
          TX packets:52205 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:8393755 (8.0 MiB)  TX bytes:4168294 (3.9 MiB)
          Base address:0xec00 Memory:defa0000-defc0000

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:3494 errors:0 dropped:0 overruns:0 frame:0
          TX packets:3494 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:296403 (289.4 KiB)  TX bytes:296403 (289.4 KiB)

vif3.0    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
          inet addr:169.254.1.0  Bcast:169.254.255.255  Mask:255.255.255.255
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:11 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:462 (462.0 b)  TX bytes:0 (0.0 b)

Is the network settings for vif3.0 the result of the not configuring the
vif variable in my xenU config?  I am hesitant specifying a mac address
as I''m afraid of fubar''ing something.
> [snip]
> 
> 
>>disk = [ ''file:/clients/test/root_fs,sda1,w'',
''file:/clients/test/swap_fs,sda2,w'' ]
>> 
>>
> 
> [snip]
> 
> 
>># Set root device.
>>root = "/dev/hda1 ro"
>> 
>>
> 
> [snip]
> 
> Your root device seems incorrectly set. Your device export line suggests
> that the line should be:
> root = "/dev/sda1 ro"
> 
Aww! Yes this was it.  I always seem to miss some mundane detail such as
that.  Thank you for being an extra set of eyes.
> 
> Hope that helps, and good luck :)
> 
Thanks for the help so far.
> Marcus.
> 
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xensource.com
> http://lists.xensource.com/xen-users

-- 
Daniel Lang

Systems Administrator
SevenL Networks
416.642.1235
1.866.639.5123

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users