xen-3.0.3-94.el5_4.2 2.6.18-164.6.1.el5xen RHEL5.4 x86_64 I''ve got a dom0 that does nothing but have a DomU created. The DomU gets plenty of load. Over time, the dom0''s ipconntrack table fills up but not the DomU. Once it gets full I can restart iptables and it''s fine. The strange thing is this only happens on hosts I have provided (hardware and hosting) from one location. I''m not really sure what kind of boxes they are or if the architecture is a red herring. I sure would like to know what is going on, the network setup is bridged but the dom0 is the domU''s gateway host (don''t ask why I''m doing both) Any thoughts? -Mike _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
> > xen-3.0.3-94.el5_4.2 > 2.6.18-164.6.1.el5xen > RHEL5.4 x86_64 > > I''ve got a dom0 that does nothing but have a DomU created. The DomUgets> plenty of load. Over time, the dom0''s ipconntrack table fills up butnot> the DomU. Once it gets full I can restart iptables and it''s fine. > > The strange thing is this only happens on hosts I have provided(hardware> and hosting) from one location. I''m not really sure what kind ofboxes> they are or if the architecture is a red herring. > > I sure would like to know what is going on, the network setup isbridged> but the dom0 is the domU''s gateway host (don''t ask why I''m doing both) > > Any thoughts? >''cat /proc/net/ip_conntrack'' will tell you what''s in the conntrack database. Have a look in there and see if it''s what you expect... James _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Mon, 25 Jan 2010, James Harper wrote:> > > > xen-3.0.3-94.el5_4.2 > > 2.6.18-164.6.1.el5xen > > RHEL5.4 x86_64 > > > > I''ve got a dom0 that does nothing but have a DomU created. The DomU > gets > > plenty of load. Over time, the dom0''s ipconntrack table fills up but > not > > the DomU. Once it gets full I can restart iptables and it''s fine. > > > > The strange thing is this only happens on hosts I have provided > (hardware > > and hosting) from one location. I''m not really sure what kind of > boxes > > they are or if the architecture is a red herring. > > > > I sure would like to know what is going on, the network setup is > bridged > > but the dom0 is the domU''s gateway host (don''t ask why I''m doing both) > > > > Any thoughts? > > > > ''cat /proc/net/ip_conntrack'' will tell you what''s in the conntrack > database. Have a look in there and see if it''s what you expect... >net.ipv4.netfilter.ip_conntrack_count = 65536 net.ipv4.netfilter.ip_conntrack_max = 65536 Being full that''s what I''d expect, what I don''t understand is why they''re filling up. -Mike _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
> > > > ''cat /proc/net/ip_conntrack'' will tell you what''s in the conntrack > > database. Have a look in there and see if it''s what you expect... > > > > net.ipv4.netfilter.ip_conntrack_count = 65536 > net.ipv4.netfilter.ip_conntrack_max = 65536 > > Being full that''s what I''d expect, what I don''t understand is whythey''re> filling up. >That''s why you need to ''cat /proc/net/ip_conntrack'' and see what''s in there. It will tell you about all the connections it''s tracking. Could be full of SSH portscans. Maybe you have a spambot on your network? Could be anything, but you need to get an understanding of the actual connections, not just a count of them. There is also a tool in the netfilter suite that can do a live listing of any new connection that gets added and removed. James _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Mon, 25 Jan 2010, James Harper wrote:> > > > > > ''cat /proc/net/ip_conntrack'' will tell you what''s in the conntrack > > > database. Have a look in there and see if it''s what you expect... > > > > > > > net.ipv4.netfilter.ip_conntrack_count = 65536 > > net.ipv4.netfilter.ip_conntrack_max = 65536 > > > > Being full that''s what I''d expect, what I don''t understand is why > they''re > > filling up. > > > > That''s why you need to ''cat /proc/net/ip_conntrack'' and see what''s in > there. It will tell you about all the connections it''s tracking. Could > be full of SSH portscans. Maybe you have a spambot on your network? > Could be anything, but you need to get an understanding of the actual > connections, not just a count of them. > > There is also a tool in the netfilter suite that can do a live listing > of any new connection that gets added and removed. >Ok, that is a good indicator. I can see things contacting port 443, which is what should be on the domU. I''m also seeing lots of established connections that aren''t showing up in netstat. So it''s like the dom0 is tracking the domU''s iptables, but is not releasing them? -Mike _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Mon, Jan 25, 2010 at 7:00 AM, Mike McGrath <mmcgrath@redhat.com> wrote:> Ok, that is a good indicator. I can see things contacting port 443, which > is what should be on the domU. I''m also seeing lots of established > connections that aren''t showing up in netstat. So it''s like the dom0 is > tracking the domU''s iptables, but is not releasing them?Have you look at each domU''s conntrack count (assuming they also have iptables enabled)? Most likely if you add up all of them it''d match dom0''s count. If the load is what you expect (i.e. no portscan/attacks), and you don''t use dom0 as firewall (just a router), then perhaps you should simply just disable iptables on dom0. Another alternative is to increase max conntrack, or reduce conntrack timeouts on dom0. -- Fajar _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Mon, Jan 25, 2010, Fajar A. Nugraha wrote:>On Mon, Jan 25, 2010 at 7:00 AM, Mike McGrath <mmcgrath@xxxxxxxxxx> wrote: >> Ok, that is a good indicator. I can see things contacting port 443, which >> is what should be on the domU. I''m also seeing lots of established >> connections that aren''t showing up in netstat. So it''s like the dom0 is >> tracking the domU''s iptables, but is not releasing them?> Have you look at each domU''s conntrack count (assuming they also have > iptables enabled)? Most likely if you add up all of them it''d match > dom0''s count.> If the load is what you expect (i.e. no portscan/attacks), and you > don''t use dom0 as firewall (just a router), then perhaps you should > simply just disable iptables on dom0. Another alternative is to > increase max conntrack, or reduce conntrack timeouts on dom0.> -- > FajarThis whole conntrack design strikes me as a serious bug that can lead to DOS attacks, even assuming that the counter is 32 bits. And I''m not comfortable with dom0 "snooping"/recording traffic on domu, isolation wise. (Yeah, I know, anybody can run tcpdump or wireshark on bridged traffic, but this is all being recorded. At least it''s not world readable.) _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
"Fajar A. Nugraha" <fajar@fajar.net> writes:> On Mon, Jan 25, 2010 at 7:00 AM, Mike McGrath <mmcgrath@redhat.com> wrote: > >> Ok, that is a good indicator. I can see things contacting port 443, which >> is what should be on the domU. I''m also seeing lots of established >> connections that aren''t showing up in netstat. So it''s like the dom0 is >> tracking the domU''s iptables, but is not releasing them? > > Have you look at each domU''s conntrack count (assuming they also have > iptables enabled)? Most likely if you add up all of them it''d match > dom0''s count. > > If the load is what you expect (i.e. no portscan/attacks), and you > don''t use dom0 as firewall (just a router), then perhaps you should > simply just disable iptables on dom0. Another alternative is to > increase max conntrack, or reduce conntrack timeouts on dom0.Or zero out /proc/sys/net/bridge/bridge-nf-call-iptables on dom0. -- Feri. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Mon, Jan 25, 2010 at 5:08 PM, jim burns <jim_burn@bellsouth.net> wrote:> This whole conntrack design strikes me as a serious bug that can lead to DOS > attacks, even assuming that the counter is 32 bits. And I''m not comfortable > with dom0 "snooping"/recording traffic on domu, isolation wise. (Yeah, I know, > anybody can run tcpdump or wireshark on bridged traffic, but this is all being > recorded. At least it''s not world readable.)That depends on your design. On my system, dom0 does bridging. It doesn''t filter (nor track) domU''s connections. Thus I don''t have to worry about DOS in this case. It''s only normal that DOS attacks to domU can bring down the firewall as well. So if you''re worry about this you shouldn''t use dom0 as firewall. -- Fajar _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users