thr3ads.net - Xen devel - [Xen-devel] Trying to get HyperSCSI and Xen to work... ;-) [Oct 2003]

If this information is useful, please help other people find it:
Share via:

sven.kretzschmar@gmx.de

2003-Oct-08 22:02 UTC

[Xen-devel] Trying to get HyperSCSI and Xen to work... ;-)

After some puzzling I was able to compile, load and use the 
HyperSCSI client(!) module in Xen''s domain 0.
It reported that it found the eth0 interface, but that there are no
HyperSCSI servers on the local network.
The same try with booting a "normal/standard" kernel on the same
machine
worked out OK, so it''s a problem with Xen''s VFR, I think.

Then I added the following code to ..../xen/common/network.c  as a quick
hack:

net_vif_t *net_get_target_vif(u8 *data, unsigned int len, net_vif_t
*src_vif)
{
    .....
    nh_raw = data + ETH_HLEN;
    switch ( ntohs(*(unsigned short *)(data + 12)) )
    {
        ....
        case ETH_P_IP:
        if ( len < (ETH_HLEN + 20) ) goto drop;
        h_raw =  data + ((*(unsigned char *)(nh_raw)) & 0x0f) * 4;
        /* NB. For now we ignore ports. */
        target = net_find_rule((u8)ETH_P_IP,  *(u8 *)(data + 9),
                               ntohl(*(u32 *)(nh_raw + 12)),
                               ntohl(*(u32 *)(nh_raw + 16)),
                               0,
                               0, 
                               src_vif_val);
        break;
        // ===>> new code for routing HyperSCSI ether packets...
        case ETH_P_HSCSI:
              //no len drops yet...
              if (src_vif != VIF_PHYS)  //If it originates from a VIF....
                   return VIF_PHYS;      //...send it to the phys. interface
              else
                   return find_vif_by_id(0);  //If it originates from the
                                                        //phys.
interface(LAN)...
                                                        //...send it to VIF0
of dom0.
              break;    //not reached...

    }
    return target;
    .....
}

I tried to route any incoming ether packets of type ETH_P_HSCSI 
which came from the physical interface to the first VIF 
of domain 0.
Also any HSCSI packets going out from the first VIF of domain 0 
should always go to the physical interface.
Currently, I don''t care about MAC-Address controled routing
and other fine tuning.
I only want to make it work with domain0 for now.

How is this done correctly ????  (Of course, the above code did NOT work ;-)

Perhaps one of the Xen developers is able to give some comments ? 
Thanks a lot :)

After reading a lot of the network sources of Xen, I have some
questions left:

1.) I am not sure about the exact difference between VIF_PHYS and
VIF_PHYSICAL_INTERFACE ?

2.) What exactly is the meaning and use of VIF_SPECIAL ?

3.) I am not really understanding the use of VIF_DOMAIN_SHIFT and 
     VIF_DOMAIN_MASK  (some way to identify the domain # only from
     the id by shifting ?)  ?

Any help and comments are apreciated. 
It would be really nice to get HyperSCSI to work in Xen.

Thanks for this great software :))


Best Regards

Sven

-- 
NEU FÜR ALLE - GMX MediaCenter - für Fotos, Musik, Dateien...
Fotoalbum, File Sharing, MMS, Multimedia-Gruß, GMX FotoService

Jetzt kostenlos anmelden unter http://www.gmx.net

+++ GMX - die erste Adresse für Mail, Message, More! +++



-------------------------------------------------------
This SF.net email is sponsored by: SF.net Giveback Program.
SourceForge.net hosts over 70,000 Open Source Projects.
See the people who have HELPED US provide better services:
Click here: http://sourceforge.net/supporters.php
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xen-devel

Keir Fraser

2003-Oct-08 23:25 UTC

head link

Re: [Xen-devel] Trying to get HyperSCSI and Xen to work... ;-)

> After some puzzling I was able to compile, load and use the 
> HyperSCSI client(!) module in Xen''s domain 0.
> It reported that it found the eth0 interface, but that there are no
> HyperSCSI servers on the local network.
> The same try with booting a "normal/standard" kernel on the same
machine
> worked out OK, so it''s a problem with Xen''s VFR, I think.
Absolutely. Out of the box Xen currently only groks IP and ARP
packets. I''ve appended a patch to (hopefully) fix this for the first
VIF of domain 0. 
> How is this done correctly ????  (Of course, the above code did NOT work
;-)
Your fix to network.c is unnecessary -- by default unrouteable packets
from DOM0 are sent to the physical interface, and incoming packets are
sent to DOM0. However, a fix *is* required to net/dev.c.

I''ve appended an alternative patch, to net/dev.c. It compiles but I
haven''t actually tested it out ;-) 
> 1.) I am not sure about the exact difference between VIF_PHYS and
> VIF_PHYSICAL_INTERFACE ?
The latter is used in the network rule lists. The former is returned
by get_target_vif to inform the network data path to route a packet to
teh physical interface.
> 2.) What exactly is the meaning and use of VIF_SPECIAL ?
It''s a revolting hack to get the currently broken packet-forwarding
code to work properly. I very much want to get rid of it as soon as
possible :-)
> 3.) I am not really understanding the use of VIF_DOMAIN_SHIFT and 
>      VIF_DOMAIN_MASK  (some way to identify the domain # only from
>      the id by shifting ?)  ?
It''s a packed representation for addressing VIFs. VIFs are indexed per
domain. E.g. DOM2, idx 3. The packed representation of that VIF, as
passed to get_vif_by_id(), is (2<<VIF_DOMAIN_SHIFT)|3.

There ought to be neat access macros to hide the packed
representation. They haven''t been written yet. :-)

A lot of the network control code (e.g., routing of packets) isn''t
great. We want to rework it at some point in the near future -- adding
more flexible packet forwarding, filtering and rewriting. The code has
evolved into its current form rather than being cleanly designed :-(

 Regards,
 Keir

--- 1.62/xen/net/dev.c	Tue Sep 30 12:47:02 2003
+++ edited/xen/net/dev.c	Thu Oct  9 00:14:19 2003
@@ -1800,20 +1800,28 @@
     return 0;
 }
 
-inline int init_tx_header(u8 *data, unsigned int len, struct net_device *dev)
+inline int init_tx_header(net_vif_t *vif, u8 *data, 
+                          unsigned int len, struct net_device *dev)
 {
+    int proto = ntohs(*(unsigned short *)(data + 12));
+
     memcpy(data + ETH_ALEN, dev->dev_addr, ETH_ALEN);
         
-    switch ( ntohs(*(unsigned short *)(data + 12)) )
+    switch ( proto )
     {
     case ETH_P_ARP:
         if ( len < 42 ) break;
         memcpy(data + 22, dev->dev_addr, ETH_ALEN);
-        return ETH_P_ARP;
+        break;
     case ETH_P_IP:
-        return ETH_P_IP;
+        break;
+    default:
+        /* Unsupported protocols are onyl allowed to/from VIF0/0. */
+        if ( (vif->domain->domain != 0) || (vif->idx != 0) )
+            proto = 0;
+        break;
     }
-    return 0;
+    return proto;
 }
 
 
@@ -1884,7 +1892,7 @@
         g_data = map_domain_mem(tx.addr);
 
         protocol = __constant_htons(
-            init_tx_header(g_data, tx.size, the_dev));
+            init_tx_header(vif, g_data, tx.size, the_dev));
         if ( protocol == 0 )
         {
             __make_tx_response(vif, tx.id, RING_STATUS_BAD_PAGE);


-------------------------------------------------------
This SF.net email is sponsored by: SF.net Giveback Program.
SourceForge.net hosts over 70,000 Open Source Projects.
See the people who have HELPED US provide better services:
Click here: http://sourceforge.net/supporters.php
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xen-devel

Nick Craig-Wood

2003-Oct-09 16:10 UTC

head link

Re: [Xen-devel] Trying to get HyperSCSI and Xen to work... ;-)

On Thu, Oct 09, 2003 at 12:25:15AM +0100, Keir Fraser
wrote:> A lot of the network control code (e.g., routing of packets) isn''t
> great. We want to rework it at some point in the near future -- adding
> more flexible packet forwarding, filtering and rewriting. The code has
> evolved into its current form rather than being cleanly designed :-(
Are there any plans to add rate limiting of network packets per domain
(in and out)?  This seemed to be the only feature missing from Xen for
use in a hostile shared hosting environment.  I read it has a fair
scheduler for packet traffic which is great.  Being able to put a cap
on the absolute traffic levels would be even better.

-- 
Nick Craig-Wood
ncw1@axis.demon.co.uk


-------------------------------------------------------
This SF.net email is sponsored by: SF.net Giveback Program.
SourceForge.net hosts over 70,000 Open Source Projects.
See the people who have HELPED US provide better services:
Click here: http://sourceforge.net/supporters.php
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xen-devel

Keir Fraser

2003-Oct-09 16:16 UTC

head link

Re: [Xen-devel] Trying to get HyperSCSI and Xen to work... ;-)

> On Thu, Oct 09, 2003 at 12:25:15AM +0100, Keir Fraser wrote:
> > A lot of the network control code (e.g., routing of packets)
isn''t
> > great. We want to rework it at some point in the near future -- adding
> > more flexible packet forwarding, filtering and rewriting. The code has
> > evolved into its current form rather than being cleanly designed :-(
> 
> Are there any plans to add rate limiting of network packets per domain
> (in and out)?  This seemed to be the only feature missing from Xen for
> use in a hostile shared hosting environment.  I read it has a fair
> scheduler for packet traffic which is great.  Being able to put a cap
> on the absolute traffic levels would be even better.
Yes, several people are interested in implementing more versatile I/O
and CPU scheduler. This should see some progress in the near future.

I think ading simple rate-limiting to the current network scheduler
would not be much work.

 -- Keir


-------------------------------------------------------
This SF.net email is sponsored by: SF.net Giveback Program.
SourceForge.net hosts over 70,000 Open Source Projects.
See the people who have HELPED US provide better services:
Click here: http://sourceforge.net/supporters.php
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xen-devel

Ian Pratt

2003-Oct-09 16:18 UTC

head link

Re: [Xen-devel] Trying to get HyperSCSI and Xen to work... ;-)

> On Thu, Oct 09, 2003 at 12:25:15AM +0100, Keir Fraser wrote:
> > A lot of the network control code (e.g., routing of packets)
isn''t
> > great. We want to rework it at some point in the near future -- adding
> > more flexible packet forwarding, filtering and rewriting. The code has
> > evolved into its current form rather than being cleanly designed :-(
> 
> Are there any plans to add rate limiting of network packets per domain
> (in and out)?  This seemed to be the only feature missing from Xen for
> use in a hostile shared hosting environment.  I read it has a fair
> scheduler for packet traffic which is great.  Being able to put a cap
> on the absolute traffic levels would be even better.
Absolutely. 

Adding a simple per-domain leaky bucket rate control scheme is on
our TODO list. We implemented a very simple yet effective scheme
for something else a few years back, and is should do the job
fine. It''s described in the following paper:

 http://www.cl.cam.ac.uk/Research/SRG/netos/arsenic/gige.ps

Implementing it in Xen should be very straightforward.

Ian


-------------------------------------------------------
This SF.net email is sponsored by: SF.net Giveback Program.
SourceForge.net hosts over 70,000 Open Source Projects.
See the people who have HELPED US provide better services:
Click here: http://sourceforge.net/supporters.php
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xen-devel

Nick Craig-Wood

2003-Oct-09 20:11 UTC

head link

Re: [Xen-devel] Trying to get HyperSCSI and Xen to work... ;-)

On Thu, Oct 09, 2003 at 05:18:30PM +0100, Ian Pratt
wrote:> Adding a simple per-domain leaky bucket rate control scheme is on
> our TODO list. We implemented a very simple yet effective scheme
> for something else a few years back, and is should do the job
> fine. 
Excellent news!
> It''s described in the following paper:
> 
>  http://www.cl.cam.ac.uk/Research/SRG/netos/arsenic/gige.ps
I''ll have a read!

Thanks
-- 
Nick Craig-Wood
ncw1@axis.demon.co.uk


-------------------------------------------------------
This SF.net email is sponsored by: SF.net Giveback Program.
SourceForge.net hosts over 70,000 Open Source Projects.
See the people who have HELPED US provide better services:
Click here: http://sourceforge.net/supporters.php
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xen-devel

Sven Kretzschmar

2003-Oct-09 21:16 UTC

head link

[Xen-devel] Re:Trying to get HyperSCSI and Xen to work... ;-)

Thanks a lot for the patch, Keir ! 
It''s working now in domain 0  :))

I now wanted to make parts of the mounted HyperSCSI disk available to 
domains > 0 via vd and vbd.
I used xenctl and added partition "sda".  Worked OK.
Then I created a vd and a vbd based on the key of the vd. Worked OK.

Now my question:

1.) Is it possible to access the created vd (virtual disk) or the created 
vbd (virtual block device)  _without_ booting and logging in to domain 1 ??
I have found no way to access either the created vd or vbd in domain 0
in order to create a filesystem on it and to fill it with something usefull.

2.) Is it possible to boot domains > 1 from a vbd (virtual block device)
     which is defined as the root device in the kernel command line ?

3.) Does the number of the created vbd coresspond to the name of
     the block device during boot time of domains > 0,  such that, if
     the vbd number is 1 then the device name is /dev/xvda1 (according
     to the README.CD file) ?
     At least in domain 0  I could not find such a device entry before
     or after creating the vbd in domain 0 ?

Thanks again and regards

Sven





-------------------------------------------------------
This SF.net email is sponsored by: SF.net Giveback Program.
SourceForge.net hosts over 70,000 Open Source Projects.
See the people who have HELPED US provide better services:
Click here: http://sourceforge.net/supporters.php
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xen-devel

Ian Pratt

2003-Oct-09 21:50 UTC

head link

Re: [Xen-devel] Re:Trying to get HyperSCSI and Xen to work... ;-)

> 1.) Is it possible to access the created vd (virtual disk) or the created 
> vbd (virtual block device)  _without_ booting and logging in to domain 1 ??
> I have found no way to access either the created vd or vbd in domain 0
> in order to create a filesystem on it and to fill it with something
usefull.
You should be able to use "xenctl vdb create -k<key> -v0 -w -n0"
to make the virtual block device accessible from domain0 as
/dev/xvda (major 125, minor 0 -- use tools/misc/xen-mkdevnodes if
the device doesn''t exist)

I haven''t tried this in a while, but it should work unless it got
accidentally broken when the "xenctl physical grant" stuff got
added. 
 > 2.) Is it possible to boot domains > 1 from a vbd (virtual block device)
>      which is defined as the root device in the kernel command line ?
Yes, e.g. root=/dev/xvdb3  (4th attached vbd, 3rd partition).

See xenolinux-2.4.22/init/do_mounts.c for the grim way that Linux
selects boot devices.
> 3.) Does the number of the created vbd coresspond to the name of
>      the block device during boot time of domains > 0,  such that, if
>      the vbd number is 1 then the device name is /dev/xvda1 (according
>      to the README.CD file) ?
>      At least in domain 0  I could not find such a device entry before
>      or after creating the vbd in domain 0 ?
The numbering is per domain, i.e. it is controlled by the -vX
parameter to "xenctl vbd create". You can connect a virtual disk
(vd) as the first vbd in one domain (/dev/xvda) and
simultaneously as e.g. the 3rd vbd in another domain
(/dev/xvdc). The optional number suffixed to a disk is a PC
partition number if present.  If you don''t put a partition table
on a virtual disk, Linux is pretty good at guessing that you mean
the whole device.

Hope this helps.

Ian


-------------------------------------------------------
This SF.net email is sponsored by: SF.net Giveback Program.
SourceForge.net hosts over 70,000 Open Source Projects.
See the people who have HELPED US provide better services:
Click here: http://sourceforge.net/supporters.php
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xen-devel

Xen devel - Oct 2003 - Trying to get HyperSCSI and Xen to work... ;-)

[Xen-devel] Trying to get HyperSCSI and Xen to work... ;-)

Re: [Xen-devel] Trying to get HyperSCSI and Xen to work... ;-)

Re: [Xen-devel] Trying to get HyperSCSI and Xen to work... ;-)

Re: [Xen-devel] Trying to get HyperSCSI and Xen to work... ;-)

Re: [Xen-devel] Trying to get HyperSCSI and Xen to work... ;-)

Re: [Xen-devel] Trying to get HyperSCSI and Xen to work... ;-)

[Xen-devel] Re:Trying to get HyperSCSI and Xen to work... ;-)

Re: [Xen-devel] Re:Trying to get HyperSCSI and Xen to work... ;-)