Fedora directory users - Feb 2007 - How to make application connect to multi-master set-up?

Hi,
   
  We have a weblogic app server based application which talks to RedHat LDAP
through weblogic''s security provider APIs.
   
  On Production env we have set-up master-master LDAP servers i.e. there are two
master LDAP servers. How should i make my application connect to these 2
servers?
   
  Should there be a front-end load balancer to which my application will make
request and then load balancer will handle requests to one of the LDAP servers?
   
  Also note that we have weblogic cluster in Prod with 2 instances. So should i
make each instance point to one LDAP server? But in that case how will automatic
failover work if one of the LDAP servers go down?
   
  Would be great if people can suggest any standard solution to deal with such
situations.
   
  regards,
  Ankur

 
---------------------------------
The fish are biting.
 Get more visitors on your site using Yahoo! Search Marketing.

Ankur Agarwal wrote:
> We have a weblogic app server based application which talks to RedHat 
> LDAP through weblogic''s security provider APIs.
>  
> On Production env we have set-up master-master LDAP servers i.e. there 
> are two master LDAP servers. How should i make my application connect 
> to these 2 servers?
>  
> Should there be a front-end load balancer to which my application will 
> make request and then load balancer will handle requests to one of the 
> LDAP servers?
>  
> Also note that we have weblogic cluster in Prod with 2 instances. So 
> should i make each instance point to one LDAP server? But in that case 
> how will automatic failover work if one of the LDAP servers go down?
>  
> Would be great if people can suggest any standard solution to deal 
> with such situations. 
>  
Typically this functionality is built in to the client (or in fact the 
LDAP client library).
Interposed proxies or load balancers are possible with LDAP but not 
normally needed.
I''m not sure about weblogic specifically but usually you get to 
configure two or more
LDAP servers, or you can use multiple A records in the DNS. I''d ask BEA
-- they
should be able to tell you.

Interesting question. Yes a hardware load balancers works perfectly. In the
configuration of a load balancer you create a virtual IP address and it will
split off the requests between two servers.

I accomplished rougly the same thing with linux-ha. I created two Virtual IP
addresses. Pointed half the apps at one and half the apps at the other. Both
of them will fail over to the other side.

What I want to do but did not have the time is use the Linux Virtual Server
Project. Ontop of linux-HA. In this way the VIRTUAL IP can float between the
two productions systems. That is really the best way to go. (and you save on
the cost of load balancing hardware)

There might be some ldap proxy software (sun makes something I think) that
can prooxy ldap and redirect it in a HA type way. I have not looked into
that. If you go with the linux virtual server project keep me in the loop.
Let me know how it turns out.

Edward

On 2/19/07, Ankur Agarwal <ankur_agwal@yahoo.com>
wrote:
>
> Hi,
>
> We have a weblogic app server based application which talks to RedHat LDAP
> through weblogic''s security provider APIs.
>
> On Production env we have set-up master-master LDAP servers i.e. there are
> two master LDAP servers. How should i make my application connect to these
2
> servers?
>
> Should there be a front-end load balancer to which my application will
> make request and then load balancer will handle requests to one of the LDAP
> servers?
>
> Also note that we have weblogic cluster in Prod with 2 instances. So
> should i make each instance point to one LDAP server? But in that case how
> will automatic failover work if one of the LDAP servers go down?
>
> Would be great if people can suggest any standard solution to deal with
> such situations.
>
> regards,
> Ankur
>
> ------------------------------
> The fish are biting.
>  Get more
visitors<http://us.rd.yahoo.com/evt=49679/*http://searchmarketing.yahoo.com/arp/sponsoredsearch_v2.php?o=US2140&cmp=Yahoo&ctv=Q107Tagline&s=Y&s2=EM&b=50>on
your site using Yahoo!
> Search Marketing.
>
>
> --
> Fedora-directory-users mailing list
> Fedora-directory-users@redhat.com
> https://www.redhat.com/mailman/listinfo/fedora-directory-users
>
>

That is a viable option as well. I am always partial to using network wide
solutions rather then client specific ones. We have had this problem before.
Some applications are capable us talking the active backup protocal some
applications are not. You get lulled into a false sence of security and you
think the application is completely load balanced. Then you realized some
other little app someone wrote somewhere is not configured the same way when
one server goes out.

Using multiple A records is an option as well. However the failover involves
making DNS changes. Based on your DNS settings that might take 20 minutes to
propagate. Plus the time to actually get a DNS change through.

Load balancers and other HA solutions have a faster, sometimes almost
instantanious recovery time. You manage them as a unit. There is no settings
confusion. Point everythign at ldap_vip.domain.com and go. They offer niec
options like 8 choices of algorithms Round robin, Weighted round robin,
least connection. etc.


Edward



On 2/19/07, David Boreham <david_list@boreham.org>
wrote:
>
> Ankur Agarwal wrote:
>
> > We have a weblogic app server based application which talks to RedHat
> > LDAP through weblogic''s security provider APIs.
> >
> > On Production env we have set-up master-master LDAP servers i.e. there
> > are two master LDAP servers. How should i make my application connect
> > to these 2 servers?
> >
> > Should there be a front-end load balancer to which my application will
> > make request and then load balancer will handle requests to one of the
> > LDAP servers?
> >
> > Also note that we have weblogic cluster in Prod with 2 instances. So
> > should i make each instance point to one LDAP server? But in that case
> > how will automatic failover work if one of the LDAP servers go down?
> >
> > Would be great if people can suggest any standard solution to deal
> > with such situations.
> >
>
> Typically this functionality is built in to the client (or in fact the
> LDAP client library).
> Interposed proxies or load balancers are possible with LDAP but not
> normally needed.
> I''m not sure about weblogic specifically but usually you get to
> configure two or more
> LDAP servers, or you can use multiple A records in the DNS. I''d
ask BEA
> -- they
> should be able to tell you.
>
>
>
>
>
> --
> Fedora-directory-users mailing list
> Fedora-directory-users@redhat.com
> https://www.redhat.com/mailman/listinfo/fedora-directory-users
>

Ankur Agarwal wrote:
>    
> We have a weblogic app server based application which talks to RedHat LDAP
through weblogic''s security provider APIs.
>    
> On Production env we have set-up master-master LDAP servers i.e. there are
two master LDAP servers. How should i make my application connect to these 2
servers?
>    
> Should there be a front-end load balancer to which my application will make
request and then load balancer will handle requests to one of the LDAP servers?
>    
> Also note that we have weblogic cluster in Prod with 2 instances. So should
i make each instance point to one LDAP server? But in that case how will
automatic failover work if one of the LDAP servers go down?
>    
> Would be great if people can suggest any standard solution to deal with
such situations.
> 
If you have multiple A records for the name in DNS, the client will 
receive all of them in a query and _can_ fail over quickly if the first 
choice does not respond.  A lot of web browsers handle this sensibly but 
  most other programs don''t.  The way to test it is to set up a DNS 
entry that contains one address that answers on the application''s port 
and one that doesn''t and see if the application works every time.  If 
you are writing your own client you can get failover a lot cheaper than 
using a dedicated load balancer - and this approach also handles 
connection failures between the client and one of the servers that a 
load balancer would not know about.

-- 
   Les Mikesell
    lesmikesell@gmail.com

Yes,

If you custom code your application you can set it up to handle multiple A
records. But you need to recode your applications each application has to be
reconfigured each time you add a server to the group.
As to the cost factor. Yes buying a load balancer might cost $2000.00. you
might be able to ebay one for $1000.(Linux Virtual Server is open source and
GPL but that is another story.)  How much does it really cost to recode your
applications, test, and redeploy? Probably a lot more work then $2000. Our
LDAP database is the corner stone of our company. We would have to recorde
10 applications to achieve our own round robin. And would only get some of
the features of a hardware load balancer.

In a real world deployment more sophisticated algorithms are sometimes
needed. For example say I have an older 2.0 GHZ web server and a newer
3.0GHZ server. A more sophisticated solution has more options. For
example:

Round Robin (you take one I take one)
Weighted Round Robin  (one server better then the other, ill take 3 your
take 2)
Least Connections (this server has LONG running ldap queries running on it.
Send queries to the other one.)
Weighted Least connection ( This server has long running LDAP queries but it
is more powerful then the other one )
Source IP ( Particular source IP is always routed to same destination server
)
Least Traffic (less TCP traffic, im busy your handle this one)
Weighted Lest traffic  (Im busy,but im alot better then you I will handle
this one )

The health checking is built in at the load balancer level. If the load
balancer detects a closed port that server is taken out of the group. Client
applications do not need extra intelligence. When you want to upgrade a
particular machine in the groop take it out of the configuration.

Google seems to be taking a hybrid approach. They likely use GEO-DNS,
mutliple A records. and hardware load balancing. Of course they are
multi-datacenter.

Non-authoritative answer:
Name:    www.l.google.com
Addresses:  216.239.37.99, 216.239.37.104

I would be willing to bet that  216.239.37.99 and 216.239.37.104 are
hardware load balancers.

All methods are viable. It just depends on what you want. Cold Failover,
High Availabilty (HA), or  Constant Availablity (CA).

For our deployment I have a two node LDAP system (multi master) If I drop
one of the nodes the IP floats to the other node within a few seconds. We
did not have to recode any application, just configure them with a floating
IP address. Some of our developers have built failover into their apps. I
think its just extra code that there is already a proven solution to. I am a
fan of mutli-master and true TCP load balancing, but thats just me.

Edward











On 2/19/07, Les Mikesell <lesmikesell@gmail.com>
wrote:
>
> Ankur Agarwal wrote:
> >
> > We have a weblogic app server based application which talks to RedHat
> LDAP through weblogic''s security provider APIs.
> >
> > On Production env we have set-up master-master LDAP servers i.e. there
> are two master LDAP servers. How should i make my application connect to
> these 2 servers?
> >
> > Should there be a front-end load balancer to which my application will
> make request and then load balancer will handle requests to one of the LDAP
> servers?
> >
> > Also note that we have weblogic cluster in Prod with 2 instances. So
> should i make each instance point to one LDAP server? But in that case how
> will automatic failover work if one of the LDAP servers go down?
> >
> > Would be great if people can suggest any standard solution to deal
with
> such situations.
> >
>
> If you have multiple A records for the name in DNS, the client will
> receive all of them in a query and _can_ fail over quickly if the first
> choice does not respond.  A lot of web browsers handle this sensibly but
>   most other programs don''t.  The way to test it is to set up a
DNS
> entry that contains one address that answers on the application''s
port
> and one that doesn''t and see if the application works every time. 
If
> you are writing your own client you can get failover a lot cheaper than
> using a dedicated load balancer - and this approach also handles
> connection failures between the client and one of the servers that a
> load balancer would not know about.
>
> --
>    Les Mikesell
>     lesmikesell@gmail.com
>
> --
> Fedora-directory-users mailing list
> Fedora-directory-users@redhat.com
> https://www.redhat.com/mailman/listinfo/fedora-directory-users
>

Eddie C wrote:
> If you custom code your application you can set it up to handle multiple A
> records. But you need to recode your applications each application has 
> to be
> reconfigured each time you add a server to the group.
No it doesn''t.  The client just needs to retry on all the IP addresses 
that the DNS request returns.  Add a server, add it''s address to DNS,
done.
> As to the cost factor. Yes buying a load balancer might cost $2000.00. you
> might be able to ebay one for $1000.(Linux Virtual Server is open source 
> and > GPL but that is another story.)
You can easily pay $30,000 and up for a load balancer.  Remember that it 
needs to be redundant and more reliable than the servers it balances to 
help any.  Then there is maintenance - and repeat for every site.  A 
good client software library routine would fix it for everyone.

 > How much does it really cost to recode  your
> applications, test, and redeploy?
Why do it any other way in the first place?  If you get alternative DNS 
addresses and the one you try first doesn''t accept your connection, why
shouldn''t every application do the sensible thing?  If IE can do it...

 > Probably a lot more work then $2000. Our
> LDAP database is the corner stone of our company. We would have to recorde
> 10 applications to achieve our own round robin.
Does this mean you don''t have a common library routine that connects to
the server?
> And would only get some of
> the features of a hardware load balancer.
And you get some the load balancer can''t provide.
> Google seems to be taking a hybrid approach. They likely use GEO-DNS,
> mutliple A records. and hardware load balancing. Of course they are
> multi-datacenter.
> 
> Non-authoritative answer:
> Name:    www.l.google.com
> Addresses:  216.239.37.99, 216.239.37.104
> 
> I would be willing to bet that  216.239.37.99 and 216.239.37.104 are
> hardware load balancers.
And you can bet that Google has spent hundreds of thousands on the 
balancing setup with DNS servers that are aware of the state of the 
servers behind a large number of local load balancers.
> For our deployment I have a two node LDAP system (multi master) If I drop
> one of the nodes the IP floats to the other node within a few seconds. We
> did not have to recode any application, just configure them with a floating
> IP address. Some of our developers have built failover into their apps. I
> think its just extra code that there is already a proven solution to. I 
> am a fan of mutli-master and true TCP load balancing, but thats just me.
I use hardware balancers too, but I recognize that most of what they do 
is cover up a problem of dumb clients that don''t know enough to try the
alternate address(es) that they already have.

-- 
   Les Mikesell
    lesmikesell@gmail.com