On a 2.4-xen0 domain-0, netstat reports lots of TX drops on the vif for packets destined to xenU. Is that normal? xen0# netstat -i Kernel Interface table Iface MTU Met RX-OK RX-ERR RX-DRP RX-OVR TX-OK TX-ERR TX-DRP TX-OVR Flg eth1 1500 0 7497330 0 0 0 1031962 0 0 0 BMRU lo 16436 0 13515 0 0 0 13515 0 0 0 LRU vif4. 1500 0 574388 0 0 0 6413103 0 5243 0 BMRU xen-b 1500 0 5759045 0 0 0 317122 0 0 0 BMRU Here tx drop rate stats from about 50 machines. ok 5247675 drop 3336 rate 0.064 ok 5354094 drop 3760 rate 0.070 ok 5382219 drop 4041 rate 0.075 ok 5237801 drop 3971 rate 0.076 ok 5365707 drop 4143 rate 0.077 ok 5365726 drop 4185 rate 0.078 ok 4622468 drop 3663 rate 0.079 ok 5380767 drop 4236 rate 0.079 ok 4622951 drop 3725 rate 0.081 ok 4625124 drop 3788 rate 0.082 ok 4625563 drop 3823 rate 0.083 ok 5355340 drop 4503 rate 0.084 ok 4626115 drop 4243 rate 0.092 ok 4634068 drop 4310 rate 0.093 ok 4622073 drop 4393 rate 0.095 ok 4634365 drop 4427 rate 0.096 ok 5377578 drop 5200 rate 0.097 ok 5378639 drop 5227 rate 0.097 ok 4646310 drop 4627 rate 0.100 ok 4634241 drop 4687 rate 0.101 ok 4620564 drop 4719 rate 0.102 ok 5354846 drop 5500 rate 0.103 ok 5354830 drop 5582 rate 0.104 ok 4634278 drop 4947 rate 0.107 ok 5351365 drop 5826 rate 0.109 ok 5349713 drop 5955 rate 0.111 ok 5354567 drop 6175 rate 0.115 ok 5377851 drop 6195 rate 0.115 ok 5378396 drop 6397 rate 0.119 ok 5378232 drop 6569 rate 0.122 ok 5577610 drop 6873 rate 0.123 ok 5348725 drop 6959 rate 0.130 ok 1761944 drop 2487 rate 0.141 ok 4632893 drop 6767 rate 0.146 ok 5347566 drop 8667 rate 0.162 ok 5971517 drop 9684 rate 0.162 ok 5587120 drop 9115 rate 0.163 ok 5977107 drop 10023 rate 0.168 ok 5609610 drop 9885 rate 0.176 ok 5863496 drop 10654 rate 0.182 ok 5603311 drop 10481 rate 0.187 ok 6077008 drop 11523 rate 0.190 ok 5586420 drop 10668 rate 0.191 ok 5612229 drop 11301 rate 0.201 ok 6260650 drop 12869 rate 0.206 ok 5998453 drop 13610 rate 0.227 ok 6298489 drop 14415 rate 0.229 ok 5595978 drop 12872 rate 0.230 ok 6199668 drop 14246 rate 0.230 ok 6459684 drop 15827 rate 0.245 ok 6298486 drop 15656 rate 0.249 ok 5804212 drop 15735 rate 0.271 ok 6215310 drop 18704 rate 0.301 ok 5778922 drop 237733 rate 4.114 ok 5332122 drop 235716 rate 4.421 ------------------------------------------------------- This SF.net email is sponsored by: IT Product Guide on ITManagersJournal Use IT products in your business? Tell us what you think of them. Give us Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more http://productguide.itmanagersjournal.com/guidepromo.tmpl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Are the drops happening at a slow, steady rate? Or are they perhaps all happening during domain startup and then there are no more drops? -- Keir> > On a 2.4-xen0 domain-0, netstat reports lots of TX drops on the vif for > packets destined to xenU. Is that normal? > > xen0# netstat -i > Kernel Interface table > Iface MTU Met RX-OK RX-ERR RX-DRP RX-OVR TX-OK TX-ERR TX-DRP TX-OVR Flg > eth1 1500 0 7497330 0 0 0 1031962 0 0 0 BMRU > lo 16436 0 13515 0 0 0 13515 0 0 0 LRU > vif4. 1500 0 574388 0 0 0 6413103 0 5243 0 BMRU > xen-b 1500 0 5759045 0 0 0 317122 0 0 0 BMRU------------------------------------------------------- This SF.net email is sponsored by: IT Product Guide on ITManagersJournal Use IT products in your business? Tell us what you think of them. Give us Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more http://productguide.itmanagersjournal.com/guidepromo.tmpl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
" Are the drops happening at a slow, steady rate? Or are they perhaps " all happening during domain startup and then there are no more drops? The drop rate is high right after a xenU startup (16% when I just tried it), then the rate slows to just under 1% of packets in steady state. ------------------------------------------------------- This SF.net email is sponsored by: IT Product Guide on ITManagersJournal Use IT products in your business? Tell us what you think of them. Give us Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more http://productguide.itmanagersjournal.com/guidepromo.tmpl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
> > " Are the drops happening at a slow, steady rate? Or are they perhaps > " all happening during domain startup and then there are no more drops? > > The drop rate is high right after a xenU startup (16% when I just tried it), > then the rate slows to just under 1% of packets in steady state. >Please try changing the following line in netfront.c: #define RX_MIN_TARGET 8 to: #define RX_MIN_TARGET NETIF_RX_RING_SIZE This will turn off the attempts to dynamically adjust the number of receive buffers queued up to receive packets. This strategy can lose packets if traffic is bursty -- also, if the domain is occasionally preempted, packets can queue up and overflow a buffer allowance that is otherwise sufficient. If you find that this sorts out your packet-loss problem then I''ll take a look at the adjustment strategy and/or make it possible to disable it in the kernel configuration. -- Keir ------------------------------------------------------- This SF.net email is sponsored by: IT Product Guide on ITManagersJournal Use IT products in your business? Tell us what you think of them. Give us Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more http://productguide.itmanagersjournal.com/guidepromo.tmpl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
" Please try changing the following line in netfront.c: " #define RX_MIN_TARGET 8 " to: " #define RX_MIN_TARGET NETIF_RX_RING_SIZE WIth this change, netstat -i on the xen0 shows 40 to 60 tx drops on the vif at boot and no further drops during steady state operation. A few xenU had no drops at all. I still see the multiple second pauses in maybe 10% of domains I create, so the problem is not due to the vif packet drops. Even on the same machine with two xenU domains, one xenU will suffer from pauses and the other not. ------------------------------------------------------- This SF.net email is sponsored by: IT Product Guide on ITManagersJournal Use IT products in your business? Tell us what you think of them. Give us Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more http://productguide.itmanagersjournal.com/guidepromo.tmpl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
" " Please try changing the following line in netfront.c: " " #define RX_MIN_TARGET 8 " " to: " " #define RX_MIN_TARGET NETIF_RX_RING_SIZE It looks like xen0 or the bridge stuff is not getting some packets destined for xenU. I traced an ssh connection from a machine running stock 2.4.25 to a xenU domain by running tcpdump on xen0 and on the other machine. The trace on xen0 shows a 4 second gap where no ssh packets were received for the xenU domain even though the other machine was sending retransmits. Only the last retransmit appears in the xen0 dump. It is unlikely the network dropped the missing packets. This is low bandwidth stuff and somewhat repeatable. I''ve never seen the effect on xen0, and in this case, there is another xenU domain on the same host showing no pause symptoms. The ''netstat -i'' statistics on xen0 and xenU do not show any dropped packets. On xen0, tcpdump -i eth0 shows the 4 second gap between packets going to xenU: 13:58:24.247209 IP firefly-batch.39292 > batch009.ssh: . ack 8065 win 40544 <nop,nop,timestamp 431700765 1532967> 13:58:28.432536 IP firefly-batch.39292 > batch009.ssh: P 1296:1392(96) ack 8065 win 40544 <nop,nop,timestamp 431701184 1532967> On the other host tcpdump shows those two packets, plus 5 more duplicates sent during the gap: 13:58:24.262055 IP firefly-batch.39292 > batch009.ssh: . ack 2145 win 40544 <nop,nop,timestamp 431700765 1532967> 13:58:25.302477 IP firefly-batch.39292 > batch009.ssh: P 720:768(48) ack 2145 win 40544 <nop,nop,timestamp 431700869 1532967> 13:58:25.414312 IP firefly-batch.39292 > batch009.ssh: P 768:816(48) ack 2145 win 40544 <nop,nop,timestamp 431700880 1532967> 13:58:25.507337 IP firefly-batch.39292 > batch009.ssh: P 720:816(96) ack 2145 win 40544 <nop,nop,timestamp 431700890 1532967> 13:58:25.927331 IP firefly-batch.39292 > batch009.ssh: P 720:816(96) ack 2145 win 40544 <nop,nop,timestamp 431700932 1532967> 13:58:26.767331 IP firefly-batch.39292 > batch009.ssh: P 720:816(96) ack 2145 win 40544 <nop,nop,timestamp 431701016 1532967> 13:58:28.447331 IP firefly-batch.39292 > batch009.ssh: P 720:816(96) ack 2145 win 40544 <nop,nop,timestamp 431701184 1532967> ------------------------------------------------------- This SF.net email is sponsored by: IT Product Guide on ITManagersJournal Use IT products in your business? Tell us what you think of them. Give us Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more http://productguide.itmanagersjournal.com/guidepromo.tmpl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
> It looks like xen0 or the bridge stuff is not getting some packets > destined for xenU. > > I traced an ssh connection from a machine running stock 2.4.25 > to a xenU domain by running tcpdump on xen0 and on the other machine. > > The trace on xen0 shows a 4 second gap where no ssh packets were received > for the xenU domain even though the other machine was sending retransmits. > Only the last retransmit appears in the xen0 dump. > > It is unlikely the network dropped the missing packets. This is low bandwidth > stuff and somewhat repeatable. I''ve never seen the effect on xen0, and > in this case, there is another xenU domain on the same host > showing no pause symptoms. > > The ''netstat -i'' statistics on xen0 and xenU do not show any dropped packets.It does very much look as though the packets simply aren''t getting to the host machine. Perhaps a bad interaction with an Ethernet switch? One thing to check is that, if you are not choosing your own MAC addresses, that the random ones chosen by xend turn out to be unique across the cluster --- any duplicates will simply beat each other up in the switch forwarding caches!! This is *definitely* worth checking out at this point, and can be fixed by picking your own MAC addresses. -- Keir ------------------------------------------------------- This SF.net email is sponsored by: IT Product Guide on ITManagersJournal Use IT products in your business? Tell us what you think of them. Give us Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more http://productguide.itmanagersjournal.com/guidepromo.tmpl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
" This is *definitely* worth checking " out at this point, and can be fixed by picking your own MAC addresses. Yep. How big a space do you randomize for mac addrs? # grep -i AA:00:00:24:22:F3 a batch009 (172.16.12.19) at AA:00:00:24:22:F3 [ether] on eth0 batch075 (172.16.12.85) at AA:00:00:24:22:F3 [ether] on eth0 ------------------------------------------------------- This SF.net email is sponsored by: IT Product Guide on ITManagersJournal Use IT products in your business? Tell us what you think of them. Give us Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more http://productguide.itmanagersjournal.com/guidepromo.tmpl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
> > " This is *definitely* worth checking > " out at this point, and can be fixed by picking your own MAC addresses. > > Yep. How big a space do you randomize for mac addrs? > > # grep -i AA:00:00:24:22:F3 a > batch009 (172.16.12.19) at AA:00:00:24:22:F3 [ether] on eth0 > batch075 (172.16.12.85) at AA:00:00:24:22:F3 [ether] on eth0 >I think AA:00:00:00:00:00 - AA:00:00:7F:FF:FF (i.e., 8 million addresses). But the random-number generator is crap. You''re definitely best off generating your own if creating any decent number of VMs. -- Keir ------------------------------------------------------- This SF.net email is sponsored by: IT Product Guide on ITManagersJournal Use IT products in your business? Tell us what you think of them. Give us Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more http://productguide.itmanagersjournal.com/guidepromo.tmpl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
" But the random-number generator is crap.
I''ll say. Out of 55 recently created VMs, 15 got dup mac addrs.
count addr
3 AA:00:00:21:81:CF
2 AA:00:00:76:32:D9
2 AA:00:00:68:1F:40
2 AA:00:00:55:B7:91
2 AA:00:00:39:E0:21
2 AA:00:00:24:22:F3
2 AA:00:00:20:57:40
" You''re definitely best off generating your own if creating any
decent
" number of VMs.
will do. Thanks.
-------------------------------------------------------
This SF.net email is sponsored by: IT Product Guide on ITManagersJournal
Use IT products in your business? Tell us what you think of them. Give us
Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more
http://productguide.itmanagersjournal.com/guidepromo.tmpl
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xen-devel
> > " But the random-number generator is crap. > > I''ll say. Out of 55 recently created VMs, 15 got dup mac addrs. > count addrmac = [ 0xaa, 0x00, 0x00, random.randint(0x00, 0x7f), random.randint(0x00, 0xff), random.randint(0x00, 0xff) ] return '':''.join(map(lambda x: "%02x" % x, mac)) You could try adding the following ahead of the above code in tools/python/xen/xm/create.py: random.seed(time.time()) (you may need to add an ''import time'' at the top of the file). Ian ------------------------------------------------------- This SF.net email is sponsored by: IT Product Guide on ITManagersJournal Use IT products in your business? Tell us what you think of them. Give us Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more http://productguide.itmanagersjournal.com/guidepromo.tmpl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
> > > > " But the random-number generator is crap. > > > > I''ll say. Out of 55 recently created VMs, 15 got dup mac addrs. > > count addr> You could try adding the following ahead of the above code in > tools/python/xen/xm/create.py: > > random.seed(time.time()) > > (you may need to add an ''import time'' at the top of the file).Yes, this is worth a try. I''ve experimented with random.randint() but I am totally unable to make it perform so abysmally as to generate 15 dupes in 55 selections from a space of 8 million. I''m getting about 1 dupe every 2000 selections. -- Keir ------------------------------------------------------- This SF.net email is sponsored by: IT Product Guide on ITManagersJournal Use IT products in your business? Tell us what you think of them. Give us Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more http://productguide.itmanagersjournal.com/guidepromo.tmpl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
> > > > > > " But the random-number generator is crap. > > > > > > I''ll say. Out of 55 recently created VMs, 15 got dup mac addrs. > > > count addr > > > You could try adding the following ahead of the above code in > > tools/python/xen/xm/create.py: > > > > random.seed(time.time()) > > > > (you may need to add an ''import time'' at the top of the file). > > Yes, this is worth a try. > > I''ve experimented with random.randint() but I am totally unable to > make it perform so abysmally as to generate 15 dupes in 55 selections > from a space of 8 million. I''m getting about 1 dupe every 2000 > selections. > > -- KeirActually theg change is trivial (''random.seed()'') so I checked it in. Hopefully it will avoid embarrassingly bad ''random'' choices. -- Keir ------------------------------------------------------- This SF.net email is sponsored by: IT Product Guide on ITManagersJournal Use IT products in your business? Tell us what you think of them. Give us Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more http://productguide.itmanagersjournal.com/guidepromo.tmpl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
I temporarilly solved this by converting the IP to hexidecimal and setting the first 2 bytes of the mac to DE:AD. Unfortunately xen increments the 3rd if you set the mac. We need a way to set the mac on both ends of the vif device. Maybe we can have things this way. options... a: set the mac on one end and have a standard "increment" bit somewhere higher up(say first byte). b: set the macs on both ends c: set a specific range of macs to use Either way xen really should test (not certain how exactly) to see if it''s already in existence prior to using a mac that is dynamicly set. Not testing is playing russian roulette IMHO (as some including myself have found out already). Opinions? On Wed, 2004-10-13 at 15:17, Keir Fraser wrote:> > > > " This is *definitely* worth checking > > " out at this point, and can be fixed by picking your own MAC addresses. > > > > Yep. How big a space do you randomize for mac addrs? > > > > # grep -i AA:00:00:24:22:F3 a > > batch009 (172.16.12.19) at AA:00:00:24:22:F3 [ether] on eth0 > > batch075 (172.16.12.85) at AA:00:00:24:22:F3 [ether] on eth0 > > > > I think AA:00:00:00:00:00 - AA:00:00:7F:FF:FF (i.e., 8 million > addresses). But the random-number generator is crap. > > You''re definitely best off generating your own if creating any decent > number of VMs. > > -- Keir > > > ------------------------------------------------------- > This SF.net email is sponsored by: IT Product Guide on ITManagersJournal > Use IT products in your business? Tell us what you think of them. Give us > Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more > http://productguide.itmanagersjournal.com/guidepromo.tmpl > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/xen-devel------------------------------------------------------- This SF.net email is sponsored by: IT Product Guide on ITManagersJournal Use IT products in your business? Tell us what you think of them. Give us Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more http://productguide.itmanagersjournal.com/guidepromo.tmpl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
One option is to have the backend interfaces pick their MAC addresses from a well-known range: e.g., AA:00:FF:00:00:00 + <vif id>. These wouldn''t need to be unique -- they''d never be seen ''on the wire''. The random MAC allocator would never pick a MAC in that range, and xend could ensure that manually-selected MACs never fall in that range and print an error if they did. It''s a shame that the backend needs a MAC different from the frontend -- but if it doesn''t then the bridge code complains. :-( -- Keir> I temporarilly solved this by converting the IP to hexidecimal and > setting the first 2 bytes of the mac to DE:AD. Unfortunately xen > increments the 3rd if you set the mac. We need a way to set the mac on > both ends of the vif device. > > Maybe we can have things this way. options... > a: set the mac on one end and have a standard "increment" bit somewhere > higher up(say first byte). > b: set the macs on both ends > c: set a specific range of macs to use > > Either way xen really should test (not certain how exactly) to see if > it''s already in existence prior to using a mac that is dynamicly set. > Not testing is playing russian roulette IMHO (as some including myself > have found out already). > > Opinions? > > On Wed, 2004-10-13 at 15:17, Keir Fraser wrote: > > > > > > " This is *definitely* worth checking > > > " out at this point, and can be fixed by picking your own MAC addresses. > > > > > > Yep. How big a space do you randomize for mac addrs? > > > > > > # grep -i AA:00:00:24:22:F3 a > > > batch009 (172.16.12.19) at AA:00:00:24:22:F3 [ether] on eth0 > > > batch075 (172.16.12.85) at AA:00:00:24:22:F3 [ether] on eth0 > > > > > > > I think AA:00:00:00:00:00 - AA:00:00:7F:FF:FF (i.e., 8 million > > addresses). But the random-number generator is crap. > > > > You''re definitely best off generating your own if creating any decent > > number of VMs. > > > > -- Keir > > > > > > ------------------------------------------------------- > > This SF.net email is sponsored by: IT Product Guide on ITManagersJournal > > Use IT products in your business? Tell us what you think of them. Give us > > Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more > > http://productguide.itmanagersjournal.com/guidepromo.tmpl > > _______________________________________________ > > Xen-devel mailing list > > Xen-devel@lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/xen-devel >------------------------------------------------------- This SF.net email is sponsored by: IT Product Guide on ITManagersJournal Use IT products in your business? Tell us what you think of them. Give us Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more http://productguide.itmanagersjournal.com/guidepromo.tmpl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
On Wed, Oct 13, 2004 at 10:07:57PM -0500, Brian Wolfe wrote:> I temporarilly solved this by converting the IP to hexidecimal and > setting the first 2 bytes of the mac to DE:AD. Unfortunately xen > increments the 3rd if you set the mac. We need a way to set the mac on > both ends of the vif device. > > Opinions?Why don''t you simply convert an IP address a.b.c.d into AA:a:00:b:c:d and let Xen do its thing on the 3rd byte? I even doubt that doing AA:00:a:b:c:d would be a problem unless you really have IP addresses a.b.c.d and a.(b^1).c.d on the same switch... christian ------------------------------------------------------- This SF.net email is sponsored by: IT Product Guide on ITManagersJournal Use IT products in your business? Tell us what you think of them. Give us Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more http://productguide.itmanagersjournal.com/guidepromo.tmpl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
On Thu, Oct 14, 2004 at 09:30:51AM +0100, Christian Limpach wrote:> On Wed, Oct 13, 2004 at 10:07:57PM -0500, Brian Wolfe wrote: > > I temporarilly solved this by converting the IP to hexidecimal and > > setting the first 2 bytes of the mac to DE:AD. Unfortunately xen > > increments the 3rd if you set the mac. We need a way to set the mac on > > both ends of the vif device. > > > > Opinions? > > Why don''t you simply convert an IP address a.b.c.d into > AA:a:00:b:c:d and let Xen do its thing on the 3rd byte? > I even doubt that doing AA:00:a:b:c:d would be a problem > unless you really have IP addresses a.b.c.d and a.(b^1).c.dor actually a.b.c.d and (a^1).b.c.d...> on the same switch... > > christian > > > > ------------------------------------------------------- > This SF.net email is sponsored by: IT Product Guide on ITManagersJournal > Use IT products in your business? Tell us what you think of them. Give us > Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more > http://productguide.itmanagersjournal.com/guidepromo.tmpl > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/xen-devel >------------------------------------------------------- This SF.net email is sponsored by: IT Product Guide on ITManagersJournal Use IT products in your business? Tell us what you think of them. Give us Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more http://productguide.itmanagersjournal.com/guidepromo.tmpl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
So the xen side of the vif isn''t see on the wire except for a: the backend driver domain( usually domain-0) and that vif''s front and driver domain eh? (I got the front and back end locations right yes?) Interesting... Either way it goes, how would one go about verifying that the MAC is truly unique on it''s visible lan? I''d be i very interested in this as a safety precaution. Right now I can''t think of how to do the uniqueness test... 8-P if someone can give me an idea or two I could try to code it since it''s something that I want.(brain fried this morning) On Thu, 2004-10-14 at 03:21, Keir Fraser wrote:> One option is to have the backend interfaces pick their MAC addresses > from a well-known range: e.g., AA:00:FF:00:00:00 + <vif id>. > These wouldn''t need to be unique -- they''d never be seen ''on the > wire''. > > The random MAC allocator would never pick a MAC in that range, and > xend could ensure that manually-selected MACs never fall in that > range and print an error if they did. > > It''s a shame that the backend needs a MAC different from the frontend > -- but if it doesn''t then the bridge code complains. :-( > > -- Keir > > > I temporarilly solved this by converting the IP to hexidecimal and > > setting the first 2 bytes of the mac to DE:AD. Unfortunately xen > > increments the 3rd if you set the mac. We need a way to set the mac on > > both ends of the vif device. > > > > Maybe we can have things this way. options... > > a: set the mac on one end and have a standard "increment" bit somewhere > > higher up(say first byte). > > b: set the macs on both ends > > c: set a specific range of macs to use > > > > Either way xen really should test (not certain how exactly) to see if > > it''s already in existence prior to using a mac that is dynamicly set. > > Not testing is playing russian roulette IMHO (as some including myself > > have found out already). > > > > Opinions? > > > > On Wed, 2004-10-13 at 15:17, Keir Fraser wrote: > > > > > > > > " This is *definitely* worth checking > > > > " out at this point, and can be fixed by picking your own MAC addresses. > > > > > > > > Yep. How big a space do you randomize for mac addrs? > > > > > > > > # grep -i AA:00:00:24:22:F3 a > > > > batch009 (172.16.12.19) at AA:00:00:24:22:F3 [ether] on eth0 > > > > batch075 (172.16.12.85) at AA:00:00:24:22:F3 [ether] on eth0 > > > > > > > > > > I think AA:00:00:00:00:00 - AA:00:00:7F:FF:FF (i.e., 8 million > > > addresses). But the random-number generator is crap. > > > > > > You''re definitely best off generating your own if creating any decent > > > number of VMs. > > > > > > -- Keir > > > > > > > > > ------------------------------------------------------- > > > This SF.net email is sponsored by: IT Product Guide on ITManagersJournal > > > Use IT products in your business? Tell us what you think of them. Give us > > > Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more > > > http://productguide.itmanagersjournal.com/guidepromo.tmpl > > > _______________________________________________ > > > Xen-devel mailing list > > > Xen-devel@lists.sourceforge.net > > > https://lists.sourceforge.net/lists/listinfo/xen-devel > > > > > > ------------------------------------------------------- > This SF.net email is sponsored by: IT Product Guide on ITManagersJournal > Use IT products in your business? Tell us what you think of them. Give us > Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more > http://productguide.itmanagersjournal.com/guidepromo.tmpl > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/xen-devel------------------------------------------------------- This SF.net email is sponsored by: IT Product Guide on ITManagersJournal Use IT products in your business? Tell us what you think of them. Give us Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more http://productguide.itmanagersjournal.com/guidepromo.tmpl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
> So the xen side of the vif isn''t see on the wire except for a: the > backend driver domain( usually domain-0) and that vif''s front and driver > domain eh? (I got the front and back end locations right yes?)Noone ever sees the backend MAC address -- it is never written into any packet. We only need to give the backend a MAC address because we hook into the Linux networking code as a normal Ethernet interface, and normal Ethernet interfaces need a normal MAC address. :-) It needs to be unique because of sanity checks in the bridge that fail if it sees a remote address == a local address that it knows about.> Either way it goes, how would one go about verifying that the MAC is > truly unique on it''s visible lan? I''d be i very interested in this as a > safety precaution. Right now I can''t think of how to do the uniqueness > test... 8-P if someone can give me an idea or two I could try to code it > since it''s something that I want.(brain fried this morning)I''m pretty sure there''s no way of soliciting a response from an Ethernet host without using some higher-level protocol; probably IP or RARP. RARP is hardly ever used, but maybe if you know the IP subnet you could do a broadcast ping and collect the responses and look at their source MAC addresses? -- Keir> On Thu, 2004-10-14 at 03:21, Keir Fraser wrote: > > One option is to have the backend interfaces pick their MAC addresses > > from a well-known range: e.g., AA:00:FF:00:00:00 + <vif id>. > > These wouldn''t need to be unique -- they''d never be seen ''on the > > wire''. > > > > The random MAC allocator would never pick a MAC in that range, and > > xend could ensure that manually-selected MACs never fall in that > > range and print an error if they did. > > > > It''s a shame that the backend needs a MAC different from the frontend > > -- but if it doesn''t then the bridge code complains. :-( > > > > -- Keir > > > > > I temporarilly solved this by converting the IP to hexidecimal and > > > setting the first 2 bytes of the mac to DE:AD. Unfortunately xen > > > increments the 3rd if you set the mac. We need a way to set the mac on > > > both ends of the vif device. > > > > > > Maybe we can have things this way. options... > > > a: set the mac on one end and have a standard "increment" bit somewhere > > > higher up(say first byte). > > > b: set the macs on both ends > > > c: set a specific range of macs to use > > > > > > Either way xen really should test (not certain how exactly) to see if > > > it''s already in existence prior to using a mac that is dynamicly set. > > > Not testing is playing russian roulette IMHO (as some including myself > > > have found out already). > > > > > > Opinions? > > > > > > On Wed, 2004-10-13 at 15:17, Keir Fraser wrote: > > > > > > > > > > " This is *definitely* worth checking > > > > > " out at this point, and can be fixed by picking your own MAC addresses. > > > > > > > > > > Yep. How big a space do you randomize for mac addrs? > > > > > > > > > > # grep -i AA:00:00:24:22:F3 a > > > > > batch009 (172.16.12.19) at AA:00:00:24:22:F3 [ether] on eth0 > > > > > batch075 (172.16.12.85) at AA:00:00:24:22:F3 [ether] on eth0 > > > > > > > > > > > > > I think AA:00:00:00:00:00 - AA:00:00:7F:FF:FF (i.e., 8 million > > > > addresses). But the random-number generator is crap. > > > > > > > > You''re definitely best off generating your own if creating any decent > > > > number of VMs. > > > > > > > > -- Keir > > > > > > > > > > > > ------------------------------------------------------- > > > > This SF.net email is sponsored by: IT Product Guide on ITManagersJournal > > > > Use IT products in your business? Tell us what you think of them. Give us > > > > Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more > > > > http://productguide.itmanagersjournal.com/guidepromo.tmpl > > > > _______________________________________________ > > > > Xen-devel mailing list > > > > Xen-devel@lists.sourceforge.net > > > > https://lists.sourceforge.net/lists/listinfo/xen-devel > > > > > > > > > > > ------------------------------------------------------- > > This SF.net email is sponsored by: IT Product Guide on ITManagersJournal > > Use IT products in your business? Tell us what you think of them. Give us > > Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more > > http://productguide.itmanagersjournal.com/guidepromo.tmpl > > _______________________________________________ > > Xen-devel mailing list > > Xen-devel@lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/xen-devel >------------------------------------------------------- This SF.net email is sponsored by: IT Product Guide on ITManagersJournal Use IT products in your business? Tell us what you think of them. Give us Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more http://productguide.itmanagersjournal.com/guidepromo.tmpl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
On Thu, 2004-10-14 at 10:29, Keir Fraser wrote:> > So the xen side of the vif isn''t see on the wire except for a: the > > backend driver domain( usually domain-0) and that vif''s front and driver > > domain eh? (I got the front and back end locations right yes?) > > Noone ever sees the backend MAC address -- it is never written into > any packet. We only need to give the backend a MAC address because we > hook into the Linux networking code as a normal Ethernet interface, > and normal Ethernet interfaces need a normal MAC address. :-) > It needs to be unique because of sanity checks in the bridge that fail > if it sees a remote address == a local address that it knows about. > > > Either way it goes, how would one go about verifying that the MAC is > > truly unique on it''s visible lan? I''d be i very interested in this as a > > safety precaution. Right now I can''t think of how to do the uniqueness > > test... 8-P if someone can give me an idea or two I could try to code it > > since it''s something that I want.(brain fried this morning) > > I''m pretty sure there''s no way of soliciting a response from an > Ethernet host without using some higher-level protocol; probably IP or > RARP. RARP is hardly ever used, but maybe if you know the IP subnet > you could do a broadcast ping and collect the responses and look at > their source MAC addresses? >Hmm, not a bad idea.... :) I know brctl can be used to query for a list of MACs on it''s physical lan that it has cached. Maybe if xend on domain-0 did something similar using ARP and arp-reply packets that it sees on the bridges to vet it''s mac choices against, and issue a warning if it sees a dupe?> -- Keir<snip - backlog shortened > ------------------------------------------------------- This SF.net email is sponsored by: IT Product Guide on ITManagersJournal Use IT products in your business? Tell us what you think of them. Give us Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more http://productguide.itmanagersjournal.com/guidepromo.tmpl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
> It looks like xen0 or the bridge stuff is not getting some packets > destined for xenU. > > I traced an ssh connection from a machine running stock 2.4.25 > to a xenU domain by running tcpdump on xen0 and on the other machine. > > The trace on xen0 shows a 4 second gap where no ssh packets were received > for the xenU domain even though the other machine was sending retransmits. > Only the last retransmit appears in the xen0 dump. > > On xen0, tcpdump -i eth0 shows the 4 second gap between packetsgoing to xenU: If they''re not showing up here, then I think we can rule out anything to do with the bridge code as we''re looking before the bridge. In fact, its hard to see how they could be dropped by anything other than the driver, and you''d expect to see the dropped count bumped. What driver is this? Even if something (e.g. Xen) went wild and took the CPU away for 4 seconds, you''d wouldn''t expect to loose packets as the NIC would buffer them. I''ll place money that you''ve still got some duplicate MAC addresses out there... Ian ------------------------------------------------------- This SF.net email is sponsored by: IT Product Guide on ITManagersJournal Use IT products in your business? Tell us what you think of them. Give us Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more http://productguide.itmanagersjournal.com/guidepromo.tmpl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
I think the mailing list is sending duplicates. :-) K.> If they''re not showing up here, then I think we can rule out > anything to do with the bridge code as we''re looking before the > bridge. > > In fact, its hard to see how they could be dropped by anything > other than the driver, and you''d expect to see the dropped count > bumped. What driver is this? > > Even if something (e.g. Xen) went wild and took the CPU away for > 4 seconds, you''d wouldn''t expect to loose packets as the NIC > would buffer them. > > I''ll place money that you''ve still got some duplicate MAC > addresses out there... > > Ian------------------------------------------------------- This SF.net email is sponsored by: IT Product Guide on ITManagersJournal Use IT products in your business? Tell us what you think of them. Give us Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more http://productguide.itmanagersjournal.com/guidepromo.tmpl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Ian Pratt wrote:>>It looks like xen0 or the bridge stuff is not getting some packets >>destined for xenU. >> >>I traced an ssh connection from a machine running stock 2.4.25 >>to a xenU domain by running tcpdump on xen0 and on the other machine. >> >>The trace on xen0 shows a 4 second gap where no ssh packets were received >>for the xenU domain even though the other machine was sending retransmits. >>Only the last retransmit appears in the xen0 dump. >> >>On xen0, tcpdump -i eth0 shows the 4 second gap between packets >> >> >going to xenU: > > >If they''re not showing up here, then I think we can rule out >anything to do with the bridge code as we''re looking before the >bridge. > >In fact, its hard to see how they could be dropped by anything >other than the driver, and you''d expect to see the dropped count >bumped. What driver is this? > >Even if something (e.g. Xen) went wild and took the CPU away for >4 seconds, you''d wouldn''t expect to loose packets as the NIC >would buffer them. > >I''ll place money that you''ve still got some duplicate MAC >addresses out there... > >Ian > >I am seeing a similar thing with only one xenU domain running. The thing is that I am trying to run an ISCSI net boot. I have much bigger problems just now so I have not had a chance to look at what is going on with the dropped packets. But I am sure I do not have an issue with a duplicate mac address. I do now believe anybody sihpps with a mac address of aa:00:00:00:00:0X where X is 1,2 or 3. -- Alvin Starr || voice: (416)585-9971 Interlink Connectivity || fax: (416)785-3668 alvin@iplink.net || ------------------------------------------------------- This SF.net email is sponsored by: IT Product Guide on ITManagersJournal Use IT products in your business? Tell us what you think of them. Give us Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more http://productguide.itmanagersjournal.com/guidepromo.tmpl _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel