Hello folks, How do I do traffic accounting in Xen? How do I find out how much data is being used by each of the domUs. Since the only identification for the domU is the mac address, I can''t find any method for doing it unless xen supports it. Any help would be greatly appreciated. Thanks. -- :: Ligesh :: http://ligesh.com _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
If dom0 is Linux based then IPTables can be used. You can create a rule that matches the incoming or outgoing interface to that of one of the domUs. The rule doesn''t need an action. IPTables keeps track of the number of packets and bytes of packets matching a rule. There are two things that you need to keep in mind: 1) The values will roll-over at some point. I would recommend saving and then resetting the values every week or so. 2) Your distribution might not save the current chains and packet counts at shutdown/restart. You would need to modify your start up scripts as needed. -Paul On 8/19/06, Ligesh <myself@ligesh.com> wrote:> > Hello folks, > > How do I do traffic accounting in Xen? How do I find out how much data is being used by each of the domUs. Since the only identification for the domU is the mac address, I can''t find any method for doing it unless xen supports it. > > Any help would be greatly appreciated. > > Thanks. > > > -- > :: Ligesh :: http://ligesh.com > > > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
I''ve had nice success using a utility like vnstat and naming each dom-u''s vif. Vnstat stores each if''s data in a separate db, so its easy for a dom-u''s bandwidth consumption to follow it around as it migrates. I haven''t gotten it perfected yet, but I did have some major headaches trying to use iptables to account.. un-orderly (and unexpected) shutdowns via re-set or apc reboot can really cause problems. If you don''t tinker much and don''t often lose dom-0 , should be fine. http://humdi.net/vnstat/ to get a copy of vnstat. it''s apt-get-able as ''vnstat'' if you''re using Debian & friends. I''ve been tinkering with the idea of one small shared gfs system to go in-between dom-0 hosts just for centralizing vnstat and other things that need to roam with dom-u''s. Any sanity obtained from it is entirely up to your naming convention, however :) hth -Tim On Sun, 2006-08-20 at 06:13 +0000, Paul M. wrote:> If dom0 is Linux based then IPTables can be used. You can create a > rule that matches the incoming or outgoing interface to that of one of > the domUs. The rule doesn''t need an action. IPTables keeps track of > the number of packets and bytes of packets matching a rule. There are > two things that you need to keep in mind: > 1) The values will roll-over at some point. I would recommend saving > and then resetting the values every week or so. > 2) Your distribution might not save the current chains and packet > counts at shutdown/restart. You would need to modify your start up > scripts as needed. > -Paul > > On 8/19/06, Ligesh <myself@ligesh.com> wrote: > > > > Hello folks, > > > > How do I do traffic accounting in Xen? How do I find out how much data is being used by each of the domUs. Since the only identification for the domU is the mac address, I can''t find any method for doing it unless xen supports it. > > > > Any help would be greatly appreciated. > > > > Thanks. > > > > > > -- > > :: Ligesh :: http://ligesh.com > > > > > > > > _______________________________________________ > > Xen-users mailing list > > Xen-users@lists.xensource.com > > http://lists.xensource.com/xen-users > > > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users > >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Sun, Aug 20, 2006 at 06:13:40AM +0000, Paul M. wrote:> If dom0 is Linux based then IPTables can be used. You can create a > rule that matches the incoming or outgoing interface to that of one of > the domUs. The rule doesn''t need an action. IPTables keeps track of > the number of packets and bytes of packets matching a rule. There are > two things that you need to keep in mind: > 1) The values will roll-over at some point. I would recommend saving > and then resetting the values every week or so. > 2) Your distribution might not save the current chains and packet > counts at shutdown/restart. You would need to modify your start up > scripts as needed.Alternatively you can obtain values directly from /proc/net/dev. This is more lightweight than iptables but depending on what you want to do with the data afterwards may require more post-processing. ucd-snmp will do this for you if exposing the values via SNMP is what you''re after, or mrtg can use them directly via mrtg-ip-acct (in the mrtgutils package in Debian, at least). I''m sure other monitoring applications such as cacti can do similar. /proc/net/dev is easily parsable textual data so it should be easy to extract the information into whatever form you need. This is all in dom0, of course. Remember that you can name the virtual interfaces on the dom0 side whatever you want in the config file for the domUs. Dominic. -- Dominic Hargreaves | http://www.larted.org.uk/~dom/ PGP key 5178E2A5 from the.earth.li (keyserver,web,email) _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Hi Ligesh,> How do I do traffic accounting in Xen? How do I find out how much data > is being used by each of the domUs. Since the only identification for > the domU is the mac address, I can''t find any method for doing it unless > xen supports it.You would have to do it from Dom0. If you have control of the DomU''s, then any of the SNMP-based products will do a good job of this for you (MRTG, Cacti, RTG). You shouldn''t need to enable SNMP on the DomUs themselves if each DomU uses it''s own interface, or VLAN (making Dom0 basically like a switch) Hope this helps. -Alan _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
vnstat had some serious issues when i last used it. It was increasing the values of the data transfer of each adapters by a huge number each them they were shut off / turned back on. Even disabling the adapter was not having an effect on the same. I had tried to get support on the forums, however there was no response. Several posts on the forums showed majority of the people having the same issue. On 8/20/06, Tim Post <tim.post@netkinetics.net> wrote:> > I''ve had nice success using a utility like vnstat and naming each > dom-u''s vif. Vnstat stores each if''s data in a separate db, so its easy > for a dom-u''s bandwidth consumption to follow it around as it migrates. > > I haven''t gotten it perfected yet, but I did have some major headaches > trying to use iptables to account.. un-orderly (and unexpected) > shutdowns via re-set or apc reboot can really cause problems. > > If you don''t tinker much and don''t often lose dom-0 , should be fine. > > http://humdi.net/vnstat/ > > to get a copy of vnstat. it''s apt-get-able as ''vnstat'' if you''re using > Debian & friends. > > I''ve been tinkering with the idea of one small shared gfs system to go > in-between dom-0 hosts just for centralizing vnstat and other things > that need to roam with dom-u''s. Any sanity obtained from it is entirely > up to your naming convention, however :) > > hth > -Tim > > On Sun, 2006-08-20 at 06:13 +0000, Paul M. wrote: > > If dom0 is Linux based then IPTables can be used. You can create a > > rule that matches the incoming or outgoing interface to that of one of > > the domUs. The rule doesn''t need an action. IPTables keeps track of > > the number of packets and bytes of packets matching a rule. There are > > two things that you need to keep in mind: > > 1) The values will roll-over at some point. I would recommend saving > > and then resetting the values every week or so. > > 2) Your distribution might not save the current chains and packet > > counts at shutdown/restart. You would need to modify your start up > > scripts as needed. > > -Paul > > > > On 8/19/06, Ligesh <myself@ligesh.com> wrote: > > > > > > Hello folks, > > > > > > How do I do traffic accounting in Xen? How do I find out how much > data is being used by each of the domUs. Since the only identification for > the domU is the mac address, I can''t find any method for doing it unless xen > supports it. > > > > > > Any help would be greatly appreciated. > > > > > > Thanks. > > > > > > > > > -- > > > :: Ligesh :: http://ligesh.com > > > > > > > > > > > > _______________________________________________ > > > Xen-users mailing list > > > Xen-users@lists.xensource.com > > > http://lists.xensource.com/xen-users > > > > > > > _______________________________________________ > > Xen-users mailing list > > Xen-users@lists.xensource.com > > http://lists.xensource.com/xen-users > > > > > > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users >-- regards, Anand Gupta _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Did you install from source or use the debian package? Thanks -Tim On Fri, 2006-08-25 at 10:17 +0530, Anand Gupta wrote:> vnstat had some serious issues when i last used it. It was increasing > the values of the data transfer of each adapters by a huge number each > them they were shut off / turned back on. Even disabling the adapter > was not having an effect on the same. > > I had tried to get support on the forums, however there was no > response. Several posts on the forums showed majority of the people > having the same issue. > > On 8/20/06, Tim Post <tim.post@netkinetics.net> wrote: > I''ve had nice success using a utility like vnstat and naming > each > dom-u''s vif. Vnstat stores each if''s data in a separate db, so > its easy > for a dom-u''s bandwidth consumption to follow it around as it > migrates. > > I haven''t gotten it perfected yet, but I did have some major > headaches > trying to use iptables to account.. un-orderly (and > unexpected) > shutdowns via re-set or apc reboot can really cause problems. > > If you don''t tinker much and don''t often lose dom-0 , should > be fine. > > http://humdi.net/vnstat/ > > to get a copy of vnstat. it''s apt-get-able as ''vnstat'' if > you''re using > Debian & friends. > > I''ve been tinkering with the idea of one small shared gfs > system to go > in-between dom-0 hosts just for centralizing vnstat and other > things > that need to roam with dom-u''s. Any sanity obtained from it is > entirely > up to your naming convention, however :) > > hth > -Tim > > On Sun, 2006-08-20 at 06:13 +0000, Paul M. wrote: > > If dom0 is Linux based then IPTables can be used. You can > create a > > rule that matches the incoming or outgoing interface to that > of one of > > the domUs. The rule doesn''t need an action. IPTables keeps > track of > > the number of packets and bytes of packets matching a rule. > There are > > two things that you need to keep in mind: > > 1) The values will roll-over at some point. I would > recommend saving > > and then resetting the values every week or so. > > 2) Your distribution might not save the current chains and > packet > > counts at shutdown/restart. You would need to modify your > start up > > scripts as needed. > > -Paul > > > > On 8/19/06, Ligesh < myself@ligesh.com> wrote: > > > > > > Hello folks, > > > > > > How do I do traffic accounting in Xen? How do I find out > how much data is being used by each of the domUs. Since the > only identification for the domU is the mac address, I can''t > find any method for doing it unless xen supports it. > > > > > > Any help would be greatly appreciated. > > > > > > Thanks. > > > > > > > > > -- > > > :: Ligesh :: http://ligesh.com > > > > > > > > > > > > _______________________________________________ > > > Xen-users mailing list > > > Xen-users@lists.xensource.com > > > http://lists.xensource.com/xen-users > > > > > > > _______________________________________________ > > Xen-users mailing list > > Xen-users@lists.xensource.com > > http://lists.xensource.com/xen-users > > > > > > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users > > > > -- > regards, > > Anand Gupta_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
I installed from source. On 8/25/06, Tim Post <tim.post@netkinetics.net> wrote:> > Did you install from source or use the debian package? >-- regards, Anand Gupta _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
The thread is sort of old, I''ve since removed the previous posts.. just to re-cap, you had an issue when the vif was re-started and the count also re-setting? or you couldn''t get vnstat to log the vif? You did specify the vifname for each dom-u you were logging, right? I''ve yet to experience errors with it, but I''ve only been using it about 90 days and haven''t needed to re-start any. Thanks -Tim On Thu, 2006-09-21 at 21:25 +0530, Anand Gupta wrote:> I installed from source. > > On 8/25/06, Tim Post <tim.post@netkinetics.net> wrote: > Did you install from source or use the debian package? > > > -- > regards, > > Anand Gupta > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
When working with vnstat, i specified each domU vifname to log the traffic. Now if i stop and start any domU (i also start vnstat monitoring for that vifname before i stop the domU), most of the times, i see a huge number added to the traffic stats which the vnstat shows for that interface. I was able to replicate this on 3 different servers and all same came up with same problems in traffic reporting. On 9/22/06, Tim Post <tim.post@netkinetics.net> wrote:> > The thread is sort of old, I''ve since removed the previous posts.. just > to re-cap, you had an issue when the vif was re-started and the count > also re-setting? or you couldn''t get vnstat to log the vif? > > You did specify the vifname for each dom-u you were logging, right? I''ve > yet to experience errors with it, but I''ve only been using it about 90 > days and haven''t needed to re-start any. > > Thanks > -Tim > > On Thu, 2006-09-21 at 21:25 +0530, Anand Gupta wrote: > > I installed from source. > > > > On 8/25/06, Tim Post <tim.post@netkinetics.net> wrote: > > Did you install from source or use the debian package? > > > > > > -- > > regards, > > > > Anand Gupta > > _______________________________________________ > > Xen-users mailing list > > Xen-users@lists.xensource.com > > http://lists.xensource.com/xen-users > >-- regards, Anand Gupta _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users