asterisk users - Feb 2005 - load balancing 20 asterisk servers

I've read several other emails and pages on the wiki but none give any
deffinate answers. if you have 20 asterisk servers each with 4 pri's, all
running RealTime Extensions and RealTime SIPBuddies from the same MySQL
server, what prevents you from putting all 20 servers behind a single load
balancer? That way all of your UA's can use the same IP to register to; vs
maintaining which customer is assigned to which machine.

perhaps its just that i am not that familiar with load balancers. i was
under the impression that a load balancer could/would send each recieved
packet to a different server.
this doesn't matter in the case of register requests since all asterisk
boxes share same SIP registry database.

but what about invite requests and the rtp stream? you would have a majorly
broken conversation if each packet in the rtp stream went to a different
asterisk box.

or are load balancers SIP aware? or is there some sort of session control
that the balancer is aware of and will send all packets in a "sip
session"
to the same asterisk box?

and then what about meet me conferences? if 10 UA's all dial a conference
DID number and all 10 get balanced to 10 different servers then they are all
sitting in seperate rooms right?

hints, opinions, facts...all welcome and appreciated.

-Matthew

Hi,

You may want to look into LVS (Linux Virtual Server). It allows load 
ballancing in a highly configurable way.
http://www.linuxvirtualserver.org/

We use it on our web and mail server to load ballance across multiple 
hosts. The way we have it configured
it will maintain a session for 15 minutes between a client and a 
specific server. So long as you have
qualify=yes in your configuration files, each client will continue to 
talk to the one server until they are turned off/
deactivated for at least 15 minutes (or whatever time period you 
configure into it). I've not tested LVS with
Asterisk, but it may be the right direction for you to take.

Cheers,
-Shaun

Matthew Boehm wrote:
>I've read several other emails and pages on the wiki but none give any
>deffinate answers. if you have 20 asterisk servers each with 4 pri's,
all
>running RealTime Extensions and RealTime SIPBuddies from the same MySQL
>server, what prevents you from putting all 20 servers behind a single load
>balancer? That way all of your UA's can use the same IP to register to;
vs
>maintaining which customer is assigned to which machine.
>
>perhaps its just that i am not that familiar with load balancers. i was
>under the impression that a load balancer could/would send each recieved
>packet to a different server.
>this doesn't matter in the case of register requests since all asterisk
>boxes share same SIP registry database.
>
>but what about invite requests and the rtp stream? you would have a majorly
>broken conversation if each packet in the rtp stream went to a different
>asterisk box.
>
>or are load balancers SIP aware? or is there some sort of session control
>that the balancer is aware of and will send all packets in a "sip
session"
>to the same asterisk box?
>
>and then what about meet me conferences? if 10 UA's all dial a
conference
>DID number and all 10 get balanced to 10 different servers then they are all
>sitting in seperate rooms right?
>
>hints, opinions, facts...all welcome and appreciated.
>
>-Matthew
>
>_______________________________________________
>Asterisk-Users mailing list
>Asterisk-Users@lists.digium.com
>http://lists.digium.com/mailman/listinfo/asterisk-users
>To UNSUBSCRIBE or update options visit:
>   http://lists.digium.com/mailman/listinfo/asterisk-users
>
>
>  
>

Inline...
> I've read several other emails and pages on the wiki but none give any
> deffinate answers. if you have 20 asterisk servers each with 4 pri's,
all
> running RealTime Extensions and RealTime SIPBuddies from the same MySQL
> server, what prevents you from putting all 20 servers behind a single load
> balancer? That way all of your UA's can use the same IP to register to;
vs
> maintaining which customer is assigned to which machine.
Load balancers vary rather dramatically in exactly how they fucntion.
Some work at layer-2, others at layer-3, and some at layers above.
Some include a small app that executes on each server to monitor
processor utilitization, etc, communicating key parameters to the
balancer. I've not tried any of these with *, but I'd have to guess
that selecting a specific model that properly handles udp sessions
(including variable length registrations) might require some resarch
that is a little more extensive then what the causual observer might
guess.
 
> perhaps its just that i am not that familiar with load balancers. i was
> under the impression that a load balancer could/would send each recieved
> packet to a different server.
That assumption is basically correct, however most balancers will 
maintain some sort of session-oriented function that will try to keep
the flow directed to the server for which it first assigned the traffic,
keeping in mind that it balances 'inbound' data flows not outbound
traffic.
> this doesn't matter in the case of register requests since all asterisk
> boxes share same SIP registry database.
I'd have to guess that registrations would be the tricky part of an
implementation simply because there are so many variations of that.
(Eg, some devices/systems register every minute while others every
hour, and about everything in between. From a load balancer perspective,
does the first registration look any different from the second and
follow-on registrations, and would the balancer consider those as
the same or different end points? Might that cause a flurry of other
system activities that have not been considered?)
 
> but what about invite requests and the rtp stream? you would have a majorly
> broken conversation if each packet in the rtp stream went to a different
> asterisk box.
No, there are parameters available to cause all packets associated with
a session to stay with the initial system and not try to load balance
on a per-packet basis. Some balancers refer to the parameter as a
sticky bit. However, careful thought has to be given to how the balancer
functions when an rtp session is _first_ initiated from an internal
* system verses a remote * system as an example.
 
> or are load balancers SIP aware? or is there some sort of session control
> that the balancer is aware of and will send all packets in a "sip
session"
> to the same asterisk box?
I have not heard of any balancer being sip-aware, and would suspect that
some of the nat-like issues probably apply to load balancers. It certainly
would _not_ be cool for the balancer to treat the rtp session setup as 
a new session when its tied directly to the sip negotitation process.
 
> and then what about meet me conferences? if 10 UA's all dial a
conference
> DID number and all 10 get balanced to 10 different servers then they are
all
> sitting in seperate rooms right?
Meetme (as well as other * functions) would certainly need to be
well thought out before considering a balancer. (Eg, where does the
customer's voicemail actually reside? How much inter-system traffic
would generated because various resources are scattered across
multiple systems such as meetme sessions, etc?)

There are obviously a lot of folks on the list that shot from the
hip with little or no practical experience. Load balancing will require
a little more well thought out engineering then that. I'm not sure
the actual realtime * implementation is at a point where these issues
can be addressed today.

You may want to consider a simpler aproach, why don't you balance the load
via DNS?
If you put in a zone file various A records for the same machine, but with
different
IP's, BIND will catch the trick and send a different IP (from the pool yo
defined) each
time a DNS request arrives. That's a simple way of doing that, it will
definively work
for termination, but you may have to think more who to cope with origiation
(outgoing
calls), since different clients will be connected to different servers.


--- asterisk-users-request@lists.digium.com wrote:

Hi,

You may want to look into LVS (Linux Virtual Server). It allows load 
ballancing in a highly configurable way.
http://www.linuxvirtualserver.org/

We use it on our web and mail server to load ballance across multiple 
hosts. The way we have it configured
it will maintain a session for 15 minutes between a client and a 
specific server. So long as you have
qualify=yes in your configuration files, each client will continue to 
talk to the one server until they are turned off/
deactivated for at least 15 minutes (or whatever time period you 
configure into it). I've not tested LVS with
Asterisk, but it may be the right direction for you to take.

Cheers,
-Shaun

Matthew Boehm wrote:
>I've read several other emails and pages on the wiki but none give any
>deffinate answers. if you have 20 asterisk servers each with 4 pri's,
all
>running RealTime Extensions and RealTime SIPBuddies from the same MySQL
>server, what prevents you from putting all 20 servers behind a single load
>balancer? That way all of your UA's can use the same IP to register to;
vs
>maintaining which customer is assigned to which machine.
>
>perhaps its just that i am not that familiar with load balancers. i was
>under the impression that a load balancer could/would send each recieved
>packet to a different server.
>this doesn't matter in the case of register requests since all asterisk
>boxes share same SIP registry database.
>
>but what about invite requests and the rtp stream? you would have a majorly
>broken conversation if each packet in the rtp stream went to a different
>asterisk box.
>
>or are load balancers SIP aware? or is there some sort of session control
>that the balancer is aware of and will send all packets in a "sip
session"
>to the same asterisk box?
>
>and then what about meet me conferences? if 10 UA's all dial a
conference
>DID number and all 10 get balanced to 10 different servers then they are all
>sitting in seperate rooms right?
>
>hints, opinions, facts...all welcome and appreciated.
>
>-Matthew
>
>_______________________________________________
>Asterisk-Users mailing list
>Asterisk-Users@lists.digium.com
>http://lists.digium.com/mailman/listinfo/asterisk-users
>To UNSUBSCRIBE or update options visit:
>   http://lists.digium.com/mailman/listinfo/asterisk-users
>
>
>  
>

		
__________________________________ 
Do you Yahoo!? 
All your favorites on one personal page – Try My Yahoo!
http://my.yahoo.com

> I've read several other emails and pages on the wiki but none give any
> deffinate answers. if you have 20 asterisk servers each with 4 pri's, 
> all
> running RealTime Extensions and RealTime SIPBuddies from the same MySQL
> server, what prevents you from putting all 20 servers behind a single 
> load
> balancer? That way all of your UA's can use the same IP to register 
> to; vs
> maintaining which customer is assigned to which machine.
> <snip>
I beleive what you're looking for is a scalable SIP proxy, like SER :)
That way, all clients registers to SER and SER redirects the caller to 
one of the asterisk boxes. Search the wiki at voip-info.org for 
"asterisk at large" :)

roy

--- asterisk-users-request@lists.digium.com wrote:

> The DNS approach does not handle single or multiple system failures,
> only very elementary load balancing over a lengthy period of time.
Are you shure of that? I'm aware that the load criteria is trickier, but
very possible.

If you use DDNS (dynamic DNS) using Bind 9. You have to run a health monitor
(like a tcp
or ping monitor) in one server (like the dns one) if one server dies, a script
removes
it's A record automagically from the pool, and even with a script that
monitors the load
you can dynamically add and remove the entry of each individual server in the
DNS server.
And you would not need to care if the load balancer sends the SIP stream to one
server
and the RTP stream to an other or in the case of outgoing connections or
whatever
extrange situation.

Use "nsupdate" utility for doing DDNS, it's really simple and
incredibly powerfull. Also
because all the requests are digitally signed, you will likely don't have
security
problems.

------------------------
> You may want to consider a simpler aproach, why don't you balance the
load via DNS?
> If you put in a zone file various A records for the same machine, but with
different
> IP's, BIND will catch the trick and send a different IP (from the pool
yo defined) each
> time a DNS request arrives. That's a simple way of doing that, it will
definively work
> for termination, but you may have to think more who to cope with origiation
(outgoing
> calls), since different clients will be connected to different servers.
> 
> 
> --- asterisk-users-request@lists.digium.com wrote:
> 
> We use it on our web and mail server to load ballance across multiple 
> hosts. The way we have it configured
> it will maintain a session for 15 minutes between a client and a 
> specific server. So long as you have
> qualify=yes in your configuration files, each client will continue to 
> talk to the one server until they are turned off/
> deactivated for at least 15 minutes (or whatever time period you 
> configure into it). I've not tested LVS with
> Asterisk, but it may be the right direction for you to take.
> 
> Cheers,
> -Shaun
> 
> Matthew Boehm wrote:
> 
> >I've read several other emails and pages on the wiki but none give
any
> >deffinate answers. if you have 20 asterisk servers each with 4
pri's, all
> >running RealTime Extensions and RealTime SIPBuddies from the same MySQL
> >server, what prevents you from putting all 20 servers behind a single
load
> >balancer? That way all of your UA's can use the same IP to register
to; vs
> >maintaining which customer is assigned to which machine.
> >
> >perhaps its just that i am not that familiar with load balancers. i was
> >under the impression that a load balancer could/would send each
recieved
> >packet to a different server.
> >this doesn't matter in the case of register requests since all
asterisk
> >boxes share same SIP registry database.
> >
> >but what about invite requests and the rtp stream? you would have a
majorly
> >broken conversation if each packet in the rtp stream went to a
different
> >asterisk box.
> >
> >or are load balancers SIP aware? or is there some sort of session
control
> >that the balancer is aware of and will send all packets in a "sip
session"
> >to the same asterisk box?
> >
> >and then what about meet me conferences? if 10 UA's all dial a
conference
> >DID number and all 10 get balanced to 10 different servers then they
are all
> >sitting in seperate rooms right?
> >
> >hints, opinions, facts...all welcome and appreciated.
> >
> >-Matthew



		
__________________________________ 
Do you Yahoo!? 
Meet the all-new My Yahoo! - Try it today! 
http://my.yahoo.com

> -----Original Message-----
> From: Rich Adamson [mailto:radamson@routers.com]
> You have fiber-seeking-backhoes in your area? Wow!
They're everywhere, man!  When I was in college an entire nearby town lost
all phone service for 24 hours due to a backhoe cutting a fiber optic cable.
3,000 people with no way of calling emergency services for an entire day.

As being said, the cost of HW based solutions is in many cases too expensive to
be
practical, leave alone to have a spare one to give you true high-availibility.
If you complain about DNS caching and timeouts not being respected, you can do a
fairly
easy thing.

As said in the follow-up, you can drop in a data center many tiny boxes.
When one box stops working or crashes you can bring down it's interface and
give a spare
box the same IP of the [now] defunct machine. If the box becomes irresponsive (a
real OS
crash) and you have IMPI 2.0 capable MoBo's (all intel server boards have
that, also some
other brands), you can remotely shutdown or reset the machine to avoid IP
clashes. Intel
has a command line utility (also the graphical console) to manage that, i've
using for a
while this and is absolutely wonderful.

Obiously, you will need a separate LAN with privates IP's to make much of
the
administration and the DB access, and use the public LAN only for
internet-related
things, so each box has it's own fixed private IP and only the public IP
changes.

Indeed, with IMPI 2.0 is possible to remotely power-up a machine (if you are
enviromentally concerned... or if the datacenter metters the electricity you
use), so you
don't need to be running all your spare servers waiting for a failure, maybe
only one and
have the others shut-down until needed.


--- asterisk-users-request@lists.digium.com wrote:
Message: 5
Date: Wed, 2 Feb 2005 21:57:42 -0600 (CST)
From: Joe Greco <jgreco@ns.sol.net>
Subject: Re: [Asterisk-Users] Re: load balancing 20 asterisk servers
To: asterisk-users@lists.digium.com
Message-ID: <200502030357.j133vgmY050836@aurora.sol.net>
Content-Type: text/plain; charset=us-ascii
> I'm trying to stay away from a software based load balancer cause what
> happens if that server fails?
> Its far less likely for a piece of dedicated hardware to fail than an
actual
> computer.
You really ought to open up one of those pieces of dedicated hardware
sometime and see what's inside.

Yep, it's software based.

Heck, many of the so-called pieces of dedicated hardware are in fact nothing
more than a fancy rack mount PC.  Open up something like a CacheFlow server
and you find an Intel server motherboard, some propietary software, and that
is about it.  Heck, go on eBay and pick yourself up some of those nice F5
BigIP ... rack mount PC's.

Some of the newer stuff is software based with some ASIC assistance for
SSL/compression.  I know that F5 has made an effort to not look like a PC
anymore, for example, and has integrated some switchlike capabilities in
their product.

Still, when it comes right down to it, the traffic direction logic in these
things is software based.

Incidentally: one of the /down/sides to these devices, aside from being
hellishly expensive, is that when it blows at 5:01PM on a Thursday evening
when Friday is Christmas, even if you have the best service contract, it
can be a trying experience to get service.  PC's have the distinct
advantage that you can actually plan to have spare parts available, and
on top of it, you can actually build high quality redundant equipment
fairly inexpensively.

AIC RMC2N-XP Chassis	$150
EMACS R2G-6350P Power	$300
SuperMicro P4SC8	$300
Intel P4-3.0 Prescott	$175
Memory		as desired
CF Adapter		$ 20
1GB CompactFlash Boot	$ 60
			----
			$1005

Toss in a monster passive heatsink and you have a system that isn't
particularly susceptible to the loss of any single moving part.

Of course, you have to be able to sysadmin your way out of a cardboard
box, so it's not like it's cost-free, but here's the thing:

If my hypothetical load balancer fails at 5:01PM on Xmas eve, I can:

1) Grab the cold spare I built because it's cheaper to do two of these
   than a single expensive HW based solution

2) Configure the hot spare I built into production (again because it's
   cheaper).

3) Grab a desktop PC and stick a few Intel GigE NIC's in it and go to
   town.

4) At least have a reasonable chance of figuring out some other way to
   fix things temporarily.

So.

What's really interesting is that even some "networking hardware"
is
actually just computing gear on steroids.  I recently saw a SMC 8624T
24-port gigE switch, and it appears to be a bunch of Broadcom GigE chips
with a CPU that runs some (can't recall which) embedded OS.  VxWorks?

... JG


====Miguel Ruiz Velasco

Version: OpenKeyServer v1.2
Comment: Extracted from belgium.keyserver.net
Signature: 0x59831109


		
__________________________________ 
Do you Yahoo!? 
Meet the all-new My Yahoo! - Try it today! 
http://my.yahoo.com

--- asterisk-users-request@lists.digium.com wrote:
> > DNS based load ballancing has it's place, as dose using an
> > application level switch.  
> > 
> > Say an earthquake takes out your California data center.
> > Shortly the DNS servers will notice and pull that center's
> > record.  However do to caches and all this is not fast
> > and users will notice.
> > 
> > What the switch does is route at the protocol level between
> > local machines.  You can take a machine off line and no one
> > will notice.  Works great until the big quake a backhoe
> > takes out a fiber cable ro there is a fire flood or who
> > knows what.
> 
> You have fiber-seeking-backhoes in your area? Wow!
Once in the data center where the company i worked had their equipment, a car
crashed in
the outside wall, and was just the place where 1 of the two whole floor power
panel were,
so half of the equipment crashed due to lack of energy, and also broke a fibre
patchpanel, so the available bandwidth was minuscule.
And in other ocasion, one repair gang in the city literally cutted a fibre cable
with a
street digger, and the repair lasted about 1 month.
Massive catastrophic failure due to external causes IS a possible scenario.
And yes, it IS a world class data-center.

====Miguel Ruiz Velasco

Version: OpenKeyServer v1.2
Comment: Extracted from belgium.keyserver.net
Signature: 0x59831109


		
__________________________________ 
Do you Yahoo!? 
Yahoo! Mail - Helps protect you from nasty viruses. 
http://promotions.yahoo.com/new_mail

Anybody with an idea how will I set me Asterisk to send call straight to my
extension with out playing demo-Congrat MENU.

> Anybody with an idea how will I set me Asterisk to send call straight to my
> extension with out playing demo-Congrat MENU.
Comething like this

exten => s,1,Answer
exten => s,2,Dial(IAX2/2001,20,tr)
exten => s,3,Voicemail(u2001)
exten => s,4,Voicemail(b2001)

Thaniks anyway but where and what files is this to be motify.

-----Original Message-----
From: asterisk-users-bounces@lists.digium.com
[mailto:asterisk-users-bounces@lists.digium.com]On Behalf Of
timebandit001@gmail.com
Sent: Friday, February 04, 2005 1:29 AM
To: Asterisk Users Mailing List - Non-Commercial Discussion
Subject: Re: [Asterisk-Users] First Call straight to my extension

> Anybody with an idea how will I set me Asterisk to send call straight to
my
> extension with out playing demo-Congrat MENU.
Comething like this

exten => s,1,Answer
exten => s,2,Dial(IAX2/2001,20,tr)
exten => s,3,Voicemail(u2001)
exten => s,4,Voicemail(b2001)
_______________________________________________
Asterisk-Users mailing list
Asterisk-Users@lists.digium.com
http://lists.digium.com/mailman/listinfo/asterisk-users
To UNSUBSCRIBE or update options visit:
   http://lists.digium.com/mailman/listinfo/asterisk-users