sven.kretzschmar@gmx.de
2003-Oct-08 22:02 UTC
[Xen-devel] Trying to get HyperSCSI and Xen to work... ;-)
After some puzzling I was able to compile, load and use the
HyperSCSI client(!) module in Xen''s domain 0.
It reported that it found the eth0 interface, but that there are no
HyperSCSI servers on the local network.
The same try with booting a "normal/standard" kernel on the same
machine
worked out OK, so it''s a problem with Xen''s VFR, I think.
Then I added the following code to ..../xen/common/network.c as a quick
hack:
net_vif_t *net_get_target_vif(u8 *data, unsigned int len, net_vif_t
*src_vif)
{
.....
nh_raw = data + ETH_HLEN;
switch ( ntohs(*(unsigned short *)(data + 12)) )
{
....
case ETH_P_IP:
if ( len < (ETH_HLEN + 20) ) goto drop;
h_raw = data + ((*(unsigned char *)(nh_raw)) & 0x0f) * 4;
/* NB. For now we ignore ports. */
target = net_find_rule((u8)ETH_P_IP, *(u8 *)(data + 9),
ntohl(*(u32 *)(nh_raw + 12)),
ntohl(*(u32 *)(nh_raw + 16)),
0,
0,
src_vif_val);
break;
// ===>> new code for routing HyperSCSI ether packets...
case ETH_P_HSCSI:
//no len drops yet...
if (src_vif != VIF_PHYS) //If it originates from a VIF....
return VIF_PHYS; //...send it to the phys. interface
else
return find_vif_by_id(0); //If it originates from the
//phys.
interface(LAN)...
//...send it to VIF0
of dom0.
break; //not reached...
}
return target;
.....
}
I tried to route any incoming ether packets of type ETH_P_HSCSI
which came from the physical interface to the first VIF
of domain 0.
Also any HSCSI packets going out from the first VIF of domain 0
should always go to the physical interface.
Currently, I don''t care about MAC-Address controled routing
and other fine tuning.
I only want to make it work with domain0 for now.
How is this done correctly ???? (Of course, the above code did NOT work ;-)
Perhaps one of the Xen developers is able to give some comments ?
Thanks a lot :)
After reading a lot of the network sources of Xen, I have some
questions left:
1.) I am not sure about the exact difference between VIF_PHYS and
VIF_PHYSICAL_INTERFACE ?
2.) What exactly is the meaning and use of VIF_SPECIAL ?
3.) I am not really understanding the use of VIF_DOMAIN_SHIFT and
VIF_DOMAIN_MASK (some way to identify the domain # only from
the id by shifting ?) ?
Any help and comments are apreciated.
It would be really nice to get HyperSCSI to work in Xen.
Thanks for this great software :))
Best Regards
Sven
--
NEU FÜR ALLE - GMX MediaCenter - für Fotos, Musik, Dateien...
Fotoalbum, File Sharing, MMS, Multimedia-Gruß, GMX FotoService
Jetzt kostenlos anmelden unter http://www.gmx.net
+++ GMX - die erste Adresse für Mail, Message, More! +++
-------------------------------------------------------
This SF.net email is sponsored by: SF.net Giveback Program.
SourceForge.net hosts over 70,000 Open Source Projects.
See the people who have HELPED US provide better services:
Click here: http://sourceforge.net/supporters.php
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xen-devel
Keir Fraser
2003-Oct-08 23:25 UTC
Re: [Xen-devel] Trying to get HyperSCSI and Xen to work... ;-)
> After some puzzling I was able to compile, load and use the > HyperSCSI client(!) module in Xen''s domain 0. > It reported that it found the eth0 interface, but that there are no > HyperSCSI servers on the local network. > The same try with booting a "normal/standard" kernel on the same machine > worked out OK, so it''s a problem with Xen''s VFR, I think.Absolutely. Out of the box Xen currently only groks IP and ARP packets. I''ve appended a patch to (hopefully) fix this for the first VIF of domain 0.> How is this done correctly ???? (Of course, the above code did NOT work ;-)Your fix to network.c is unnecessary -- by default unrouteable packets from DOM0 are sent to the physical interface, and incoming packets are sent to DOM0. However, a fix *is* required to net/dev.c. I''ve appended an alternative patch, to net/dev.c. It compiles but I haven''t actually tested it out ;-)> 1.) I am not sure about the exact difference between VIF_PHYS and > VIF_PHYSICAL_INTERFACE ?The latter is used in the network rule lists. The former is returned by get_target_vif to inform the network data path to route a packet to teh physical interface.> 2.) What exactly is the meaning and use of VIF_SPECIAL ?It''s a revolting hack to get the currently broken packet-forwarding code to work properly. I very much want to get rid of it as soon as possible :-)> 3.) I am not really understanding the use of VIF_DOMAIN_SHIFT and > VIF_DOMAIN_MASK (some way to identify the domain # only from > the id by shifting ?) ?It''s a packed representation for addressing VIFs. VIFs are indexed per domain. E.g. DOM2, idx 3. The packed representation of that VIF, as passed to get_vif_by_id(), is (2<<VIF_DOMAIN_SHIFT)|3. There ought to be neat access macros to hide the packed representation. They haven''t been written yet. :-) A lot of the network control code (e.g., routing of packets) isn''t great. We want to rework it at some point in the near future -- adding more flexible packet forwarding, filtering and rewriting. The code has evolved into its current form rather than being cleanly designed :-( Regards, Keir --- 1.62/xen/net/dev.c Tue Sep 30 12:47:02 2003 +++ edited/xen/net/dev.c Thu Oct 9 00:14:19 2003 @@ -1800,20 +1800,28 @@ return 0; } -inline int init_tx_header(u8 *data, unsigned int len, struct net_device *dev) +inline int init_tx_header(net_vif_t *vif, u8 *data, + unsigned int len, struct net_device *dev) { + int proto = ntohs(*(unsigned short *)(data + 12)); + memcpy(data + ETH_ALEN, dev->dev_addr, ETH_ALEN); - switch ( ntohs(*(unsigned short *)(data + 12)) ) + switch ( proto ) { case ETH_P_ARP: if ( len < 42 ) break; memcpy(data + 22, dev->dev_addr, ETH_ALEN); - return ETH_P_ARP; + break; case ETH_P_IP: - return ETH_P_IP; + break; + default: + /* Unsupported protocols are onyl allowed to/from VIF0/0. */ + if ( (vif->domain->domain != 0) || (vif->idx != 0) ) + proto = 0; + break; } - return 0; + return proto; } @@ -1884,7 +1892,7 @@ g_data = map_domain_mem(tx.addr); protocol = __constant_htons( - init_tx_header(g_data, tx.size, the_dev)); + init_tx_header(vif, g_data, tx.size, the_dev)); if ( protocol == 0 ) { __make_tx_response(vif, tx.id, RING_STATUS_BAD_PAGE); ------------------------------------------------------- This SF.net email is sponsored by: SF.net Giveback Program. SourceForge.net hosts over 70,000 Open Source Projects. See the people who have HELPED US provide better services: Click here: http://sourceforge.net/supporters.php _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Nick Craig-Wood
2003-Oct-09 16:10 UTC
Re: [Xen-devel] Trying to get HyperSCSI and Xen to work... ;-)
On Thu, Oct 09, 2003 at 12:25:15AM +0100, Keir Fraser wrote:> A lot of the network control code (e.g., routing of packets) isn''t > great. We want to rework it at some point in the near future -- adding > more flexible packet forwarding, filtering and rewriting. The code has > evolved into its current form rather than being cleanly designed :-(Are there any plans to add rate limiting of network packets per domain (in and out)? This seemed to be the only feature missing from Xen for use in a hostile shared hosting environment. I read it has a fair scheduler for packet traffic which is great. Being able to put a cap on the absolute traffic levels would be even better. -- Nick Craig-Wood ncw1@axis.demon.co.uk ------------------------------------------------------- This SF.net email is sponsored by: SF.net Giveback Program. SourceForge.net hosts over 70,000 Open Source Projects. See the people who have HELPED US provide better services: Click here: http://sourceforge.net/supporters.php _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Keir Fraser
2003-Oct-09 16:16 UTC
Re: [Xen-devel] Trying to get HyperSCSI and Xen to work... ;-)
> On Thu, Oct 09, 2003 at 12:25:15AM +0100, Keir Fraser wrote: > > A lot of the network control code (e.g., routing of packets) isn''t > > great. We want to rework it at some point in the near future -- adding > > more flexible packet forwarding, filtering and rewriting. The code has > > evolved into its current form rather than being cleanly designed :-( > > Are there any plans to add rate limiting of network packets per domain > (in and out)? This seemed to be the only feature missing from Xen for > use in a hostile shared hosting environment. I read it has a fair > scheduler for packet traffic which is great. Being able to put a cap > on the absolute traffic levels would be even better.Yes, several people are interested in implementing more versatile I/O and CPU scheduler. This should see some progress in the near future. I think ading simple rate-limiting to the current network scheduler would not be much work. -- Keir ------------------------------------------------------- This SF.net email is sponsored by: SF.net Giveback Program. SourceForge.net hosts over 70,000 Open Source Projects. See the people who have HELPED US provide better services: Click here: http://sourceforge.net/supporters.php _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Ian Pratt
2003-Oct-09 16:18 UTC
Re: [Xen-devel] Trying to get HyperSCSI and Xen to work... ;-)
> On Thu, Oct 09, 2003 at 12:25:15AM +0100, Keir Fraser wrote: > > A lot of the network control code (e.g., routing of packets) isn''t > > great. We want to rework it at some point in the near future -- adding > > more flexible packet forwarding, filtering and rewriting. The code has > > evolved into its current form rather than being cleanly designed :-( > > Are there any plans to add rate limiting of network packets per domain > (in and out)? This seemed to be the only feature missing from Xen for > use in a hostile shared hosting environment. I read it has a fair > scheduler for packet traffic which is great. Being able to put a cap > on the absolute traffic levels would be even better.Absolutely. Adding a simple per-domain leaky bucket rate control scheme is on our TODO list. We implemented a very simple yet effective scheme for something else a few years back, and is should do the job fine. It''s described in the following paper: http://www.cl.cam.ac.uk/Research/SRG/netos/arsenic/gige.ps Implementing it in Xen should be very straightforward. Ian ------------------------------------------------------- This SF.net email is sponsored by: SF.net Giveback Program. SourceForge.net hosts over 70,000 Open Source Projects. See the people who have HELPED US provide better services: Click here: http://sourceforge.net/supporters.php _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Nick Craig-Wood
2003-Oct-09 20:11 UTC
Re: [Xen-devel] Trying to get HyperSCSI and Xen to work... ;-)
On Thu, Oct 09, 2003 at 05:18:30PM +0100, Ian Pratt wrote:> Adding a simple per-domain leaky bucket rate control scheme is on > our TODO list. We implemented a very simple yet effective scheme > for something else a few years back, and is should do the job > fine.Excellent news!> It''s described in the following paper: > > http://www.cl.cam.ac.uk/Research/SRG/netos/arsenic/gige.psI''ll have a read! Thanks -- Nick Craig-Wood ncw1@axis.demon.co.uk ------------------------------------------------------- This SF.net email is sponsored by: SF.net Giveback Program. SourceForge.net hosts over 70,000 Open Source Projects. See the people who have HELPED US provide better services: Click here: http://sourceforge.net/supporters.php _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Sven Kretzschmar
2003-Oct-09 21:16 UTC
[Xen-devel] Re:Trying to get HyperSCSI and Xen to work... ;-)
Thanks a lot for the patch, Keir !
It''s working now in domain 0 :))
I now wanted to make parts of the mounted HyperSCSI disk available to
domains > 0 via vd and vbd.
I used xenctl and added partition "sda". Worked OK.
Then I created a vd and a vbd based on the key of the vd. Worked OK.
Now my question:
1.) Is it possible to access the created vd (virtual disk) or the created
vbd (virtual block device) _without_ booting and logging in to domain 1 ??
I have found no way to access either the created vd or vbd in domain 0
in order to create a filesystem on it and to fill it with something usefull.
2.) Is it possible to boot domains > 1 from a vbd (virtual block device)
which is defined as the root device in the kernel command line ?
3.) Does the number of the created vbd coresspond to the name of
the block device during boot time of domains > 0, such that, if
the vbd number is 1 then the device name is /dev/xvda1 (according
to the README.CD file) ?
At least in domain 0 I could not find such a device entry before
or after creating the vbd in domain 0 ?
Thanks again and regards
Sven
-------------------------------------------------------
This SF.net email is sponsored by: SF.net Giveback Program.
SourceForge.net hosts over 70,000 Open Source Projects.
See the people who have HELPED US provide better services:
Click here: http://sourceforge.net/supporters.php
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xen-devel
Ian Pratt
2003-Oct-09 21:50 UTC
Re: [Xen-devel] Re:Trying to get HyperSCSI and Xen to work... ;-)
> 1.) Is it possible to access the created vd (virtual disk) or the created > vbd (virtual block device) _without_ booting and logging in to domain 1 ?? > I have found no way to access either the created vd or vbd in domain 0 > in order to create a filesystem on it and to fill it with something usefull.You should be able to use "xenctl vdb create -k<key> -v0 -w -n0" to make the virtual block device accessible from domain0 as /dev/xvda (major 125, minor 0 -- use tools/misc/xen-mkdevnodes if the device doesn''t exist) I haven''t tried this in a while, but it should work unless it got accidentally broken when the "xenctl physical grant" stuff got added.> 2.) Is it possible to boot domains > 1 from a vbd (virtual block device) > which is defined as the root device in the kernel command line ?Yes, e.g. root=/dev/xvdb3 (4th attached vbd, 3rd partition). See xenolinux-2.4.22/init/do_mounts.c for the grim way that Linux selects boot devices.> 3.) Does the number of the created vbd coresspond to the name of > the block device during boot time of domains > 0, such that, if > the vbd number is 1 then the device name is /dev/xvda1 (according > to the README.CD file) ? > At least in domain 0 I could not find such a device entry before > or after creating the vbd in domain 0 ?The numbering is per domain, i.e. it is controlled by the -vX parameter to "xenctl vbd create". You can connect a virtual disk (vd) as the first vbd in one domain (/dev/xvda) and simultaneously as e.g. the 3rd vbd in another domain (/dev/xvdc). The optional number suffixed to a disk is a PC partition number if present. If you don''t put a partition table on a virtual disk, Linux is pretty good at guessing that you mean the whole device. Hope this helps. Ian ------------------------------------------------------- This SF.net email is sponsored by: SF.net Giveback Program. SourceForge.net hosts over 70,000 Open Source Projects. See the people who have HELPED US provide better services: Click here: http://sourceforge.net/supporters.php _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel