asterisk users - Apr 2006 - (Semi-OT) QoS Question FTP Living with Asterisk

I have searched google and came up with too many options and packages 
that may or may not work for my needs, most articles seem to be for 
setting up routers.  Maybe someone on the list can give me some better 
insight.

I have monitoring turned on my "shift eight (tm)" (Asterisk ;-)) box
for
all calls.  We have over one hundred agents and tons of recordings in 
wav format.  I also have a cron job that runs a script to mux the in and 
out files and ftp them to a NAS device and it runs every five minutes. 

The NAS device and the * box are both directly connected to a Cisco 
Gigabit switch.  I have had complaints of calls fading in and out and 
also cutting off.  After reviewing the recordings, some of these 
complaints seem valid and I suspect the sheer bandwidth of the FTP 
traffic is causing the issues.  I also run nagios checks on the box and 
get ping warnings on a regular basis. 

My question is, how can I throttle the FTP (Standard with dist) 
transfers using out of the box CentOS4.3 (or any easy to use, low 
learning curve package)?  I thought about FTPing the files at less 
frequent intervals but that just makes the issue less frequent but last 
longer. 

I would like to accomplish throttling FTP on the Linux box with a 
solution that is not too elegant since this is a production machine in a 
busy call center.  If I cannot do it on the * box I guess my next step 
is to see if the Cisco Gigabit switch has any QoS functionality. 

Thanks,
Steve

It's a little crude but you can 

1: Use VLAN(ing) on the Cisco Switch to segment the traffic on an
addition 'LAN'.

2: Low Budget, Add a NIC on a separate network with the NAS.

3: Give me a bit, It'll come to me! :-)

SNIP!!

At 06:17 PM 4/29/2006, you wrote:
>My question is, how can I throttle the FTP (Standard with dist) 
>transfers using out of the box CentOS4.3 (or any easy to use, low 
>learning curve package)?  I thought about FTPing the files at less 
>frequent intervals but that just makes the issue less frequent but 
>last longer.
A really cheesy solution would be a second Ethernet card for the file 
transfers set to 10mb mode.

Ira

Ira wrote:
> At 06:17 PM 4/29/2006, you wrote:
>> My question is, how can I throttle the FTP (Standard with dist) 
>> transfers using out of the box CentOS4.3 (or any easy to use, low 
>> learning curve package)?  I thought about FTPing the files at less 
>> frequent intervals but that just makes the issue less frequent but 
>> last longer.
>
> A really cheesy solution would be a second Ethernet card for the file 
> transfers set to 10mb mode.
>
> Ira
Not sure if cheesy is the right word.  Sound solution may be a better 
adjective.  Adding two NICs, one to each machine and connecting them 
directly via crossover cable on a totally separate network may be my 
best solution.  No FTP traffic would even hit the NIC or the network 
used for VoIP and everything else.

Unless there is a setting in Linux somewhere (still holding out hope)

Thanks,
Steve

On Sat, 2006-04-29 at 21:17 -0400, Steve Totaro wrote:
> My question is, how can I throttle the FTP (Standard with dist) 
> transfers using out of the box CentOS4.3 (or any easy to use, low 
> learning curve package)?  I thought about FTPing the files at less 
> frequent intervals but that just makes the issue less frequent but last 
> longer. 
Is it the network/interrupt load or the CPU/RAM load that's causing the
issue?  If it's the later, seems like your SOL.  If it's the former, I
wonder if you could fiddle with traffic shaping in iptables to keep the
FTP traffic down.

Just a thought.

Paul
-- 
Paul Dugas, Computer Engineer                Dugas Enterprises, LLC
paul@dugas.cc        phone: 404-932-1355     522 Black Canyon Park
http://dugas.cc        fax: 866-751-6494     Canton, GA 30114 USA
--
On site at GDOT's W.Annex, 404-463-2860 x199
--
This e-mail and any attachments are confidential.  If you receive
this message in error or are not the intended recipient, you should
not retain, distribute, disclose or use any of this information and
you should destroy the e-mail and any attachments or copies.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url :
http://lists.digium.com/pipermail/asterisk-users/attachments/20060429/9e2e223f/attachment.pgp

Steve Totaro wrote:
> I have searched google and came up with too many options and packages 
> that may or may not work for my needs, most articles seem to be for 
> setting up routers.  Maybe someone on the list can give me some better 
> insight.
> 
> I have monitoring turned on my "shift eight (tm)" (Asterisk ;-))
box for
> all calls.  We have over one hundred agents and tons of recordings in 
> wav format.  I also have a cron job that runs a script to mux the in and 
> out files and ftp them to a NAS device and it runs every five minutes.
> The NAS device and the * box are both directly connected to a Cisco 
> Gigabit switch.  I have had complaints of calls fading in and out and 
> also cutting off.  After reviewing the recordings, some of these 
> complaints seem valid and I suspect the sheer bandwidth of the FTP 
> traffic is causing the issues.  I also run nagios checks on the box and 
> get ping warnings on a regular basis.
> My question is, how can I throttle the FTP (Standard with dist) 
> transfers using out of the box CentOS4.3 (or any easy to use, low 
> learning curve package)?  I thought about FTPing the files at less 
> frequent intervals but that just makes the issue less frequent but last 
> longer.
> I would like to accomplish throttling FTP on the Linux box with a 
> solution that is not too elegant since this is a production machine in a 
> busy call center.  If I cannot do it on the * box I guess my next step 
> is to see if the Cisco Gigabit switch has any QoS functionality.
> Thanks,
> Steve
> _______________________________________________
Steve,

	If you don't want to get too fancy, you should switch to using rsync 
(if possible) and use the "--bwlimit" option.  If you MUST use ftp,
try
using trickle:

http://monkey.org/~marius/pages/?page=trickle

	I haven't used it, but you should be able to call your FTP upload 
binary (whatever it may be) with it and force a lower transfer speed.

Let us know how it goes!

-- 
Kristian Kielhofner

Steve Totaro wrote:
> I have searched google and came up with too many options and packages 
> that may or may not work for my needs, most articles seem to be for 
> setting up routers.  Maybe someone on the list can give me some better 
> insight.
> 
> I have monitoring turned on my "shift eight (tm)" (Asterisk ;-))
box for
> all calls.  We have over one hundred agents and tons of recordings in 
> wav format.  I also have a cron job that runs a script to mux the in and 
> out files and ftp them to a NAS device and it runs every five minutes.
> The NAS device and the * box are both directly connected to a Cisco 
> Gigabit switch.  I have had complaints of calls fading in and out and 
> also cutting off.  After reviewing the recordings, some of these 
> complaints seem valid and I suspect the sheer bandwidth of the FTP 
> traffic is causing the issues.  I also run nagios checks on the box and 
> get ping warnings on a regular basis.
> My question is, how can I throttle the FTP (Standard with dist) 
> transfers using out of the box CentOS4.3 (or any easy to use, low 
> learning curve package)?  I thought about FTPing the files at less 
> frequent intervals but that just makes the issue less frequent but last 
> longer.
> I would like to accomplish throttling FTP on the Linux box with a 
> solution that is not too elegant since this is a production machine in a 
> busy call center.  If I cannot do it on the * box I guess my next step 
> is to see if the Cisco Gigabit switch has any QoS functionality.
> Thanks,
> Steve
Steve,

	Now for the fancy solutions:

1)  Try enabling NAPI interrupt handling for your ethernet card.  Some 
people report that it reduces interrupt load while increasing transfer 
speed by %85 - %100 (on the same processor, with the Intel e1000 driver 
and a good Intel NIC).  I haven't tried it yet, but that is what I have 
read...

2)  AstShape:

http://www.krisk.org/astlinux/misc/astshape

3)  AstShape and Cisco:

http://www.krisk.org/astlinux/misc/astshape

	Configure the Catalyst to map packets with IP TOS 0x10 and 0x18 into 
the second highest priority (the highest priority is reserved for 
"network control messages").  Either whatever the Cisco has or 802.1p
5
(the highest priority for "end user" traffic).

	At gigabit speeds QoS configured like this will probably just waste CPU 
time (on both the machine and the switch) while not being very 
effective.  Try my simple suggestions from my previous post first! :)

-- 
Kristian Kielhofner

On 30/04/06, Steve Totaro <stotaro@asteriskhelpdesk.com>
wrote:
> My question is, how can I throttle the FTP (Standard with dist)
> transfers using out of the box CentOS4.3 (or any easy to use, low
> learning curve package)?  I thought about FTPing the files at less
> frequent intervals but that just makes the issue less frequent but last
> longer.
It's a while since I've looked at it, but I seem to recall that
ProFTPD has options for bandwidth limiting per login - you could take
a look at that.

I just took a glance at the online docs - 'TransferRate' - 'The
TransferRate directive is used to set transfer rates limits on the
transfer of data. This directive allows for transfer rates to be set
in a wide variety of contexts, on a per-command basis, and for certain
subsets of users. Note that this limit only applies to a single
connection, and not to the overall transfer rate of the server.'

www.proftpd.org

Peter

--
Peter Bowyer
Email: peter@bowyer.org

Steve

 From what i read here and from what others have suggested i can only 
surmise that you tried almost everything besides the simplest thing .

out of the box CentOs installs proftpd (AFAIK ) , this ftp engine has a 
simple too called mod_shaper
which allows as you can assume shape traffic rates .

example of the  configuration is :
<IfModule mod_shaper.c>
    ShaperEngine on
    ShaperLog /var/log/ftpd/shaper.log
    ShaperTable /var/log/ftpd/shaper.tab
    # Enableing FXP
    AllowForeignAddress     on

    # An overall rate (in KB/s) must be set.  This line explicitly
    # sets both the download and upload rates to be the same.
***    ShaperAll downrate 100 uprate 100

This is the line to manipulate

    # Allow all system users to see shaper info
    #ShaperControlsACLs info allow user *

    # Allow FTP admins to alter settings both overall and per-session
    ShaperControlsACLs all,sess allow group ftpadm


  </IfModule>

you'll need to see if the module was installed in the initial 
installation , and if not - download it and compile it .


Assaf

Steve Totaro wrote:
> I have searched google and came up with too many options and packages 
> that may or may not work for my needs, most articles seem to be for 
> setting up routers.  Maybe someone on the list can give me some better 
> insight.
>
> I have monitoring turned on my "shift eight (tm)" (Asterisk ;-))
box
> for all calls.  We have over one hundred agents and tons of recordings 
> in wav format.  I also have a cron job that runs a script to mux the 
> in and out files and ftp them to a NAS device and it runs every five 
> minutes.
> The NAS device and the * box are both directly connected to a Cisco 
> Gigabit switch.  I have had complaints of calls fading in and out and 
> also cutting off.  After reviewing the recordings, some of these 
> complaints seem valid and I suspect the sheer bandwidth of the FTP 
> traffic is causing the issues.  I also run nagios checks on the box 
> and get ping warnings on a regular basis.
> My question is, how can I throttle the FTP (Standard with dist) 
> transfers using out of the box CentOS4.3 (or any easy to use, low 
> learning curve package)?  I thought about FTPing the files at less 
> frequent intervals but that just makes the issue less frequent but 
> last longer.
> I would like to accomplish throttling FTP on the Linux box with a 
> solution that is not too elegant since this is a production machine in 
> a busy call center.  If I cannot do it on the * box I guess my next 
> step is to see if the Cisco Gigabit switch has any QoS functionality.
> Thanks,
> Steve
> _______________________________________________
> --Bandwidth and Colocation provided by Easynews.com --
>
> Asterisk-Users mailing list
> To UNSUBSCRIBE or update options visit:
>   http://lists.digium.com/mailman/listinfo/asterisk-users
-- 
Assaf Flatto
Atelis IT Manager
Cellular: +972-54-5679230
e-mail: assaf@atelis.net