Halo, Please can you explain why this is advantage over a hardware load balancer. I fail to see advantage if anything it add in more point of failure, with several hundred thousand user, we can ill afford to mess around or add to complexity, sometime keeping it simple is simply way to be, when use qmail/vpopmail, we never had one failure or problem, ever, we very proud of this record so our users. if director service assign 60K user to each front end, how it handle if 5K simultaneous user login, but all 5K happen to be assign to that one machine, it do all work whilst other 7 server sit there do nothing negating what the LB is design for? Is it really worth it? Do we really need this, or just let foundry switch handle it as it does now. We also have 24 front end SMTP server, these deliver mail to netapp filer, all 24 plus 8 pop3 server and 2 webmail imap server all mount /vmail, so all access same maildir. it seem work very effective thus far and for many many year when we use qmail and vpopmail. Thaks
On 26.8.2010, at 23.54, Edward avanti wrote:> Please can you explain why this is advantage over a hardware load balancer.It guarantees that the same user is accessed via the same server. Hardware LB can at best assign the connections from the same IP to the same server (but not e.g. new mail deliveries, or if user has multiple clients like home/work/mobile, or simply accesses webmail at the same time as has a client open).> I fail to see advantage if anything it add in more point of failure, with > several hundred thousand user, we can ill afford to mess around or add to > complexity, sometime keeping it simple is simply way to be, when use > qmail/vpopmail, we never had one failure or problem, ever, we very proud of > this record so our users.Are you using Dovecot? For how long? Do you see any errors in your logs?> if director service assign 60K user to each front end, how it handle if 5K > simultaneous user login, but all 5K happen to be assign to that one machine, > it do all work whilst other 7 server sit there do nothing negating what the > LB is design for?The users are distributed based on the MD5 hash of their username and that gives a pretty good distribution of where the users go. Unless 5k of your users suddenly decide to coordinate such attack, I doubt you'll ever see anything even close to that happening. But sure, there is some variation. I don't have any real world numbers, but my guess is it's normally less than 20%. Also with some more work it would be possible to dynamically adjust how many new users are getting assigned to servers, but I wasn't planning on implementing that unless it becomes clear that it's needed.> Is it really worth it? Do we really need this, or just let foundry switch > handle it as it does now. > We also have 24 front end SMTP server, these deliver mail to netapp filer, > all 24 plus 8 pop3 server and 2 webmail imap server all mount /vmail, so all > access same maildir. it seem work very effective thus far and for many many > year when we use qmail and vpopmail.If you don't mind random Dovecot errors about index corruption I guess you're fine with how it works now. I guess your mails are delivered to maildirs by qmail? If you ever switch to Dovecot LDA you'll probably start getting more errors. And if you ever plan to switch to dbox format then at latest you'll need director.
On Fri, 2010-08-27 at 08:54 +1000, Edward avanti wrote:> Halo, > Please can you explain why this is advantage over a hardware load balancer.it is no advantage over a dedicated hardware solution, but director does not do the exact same thing.> I fail to see advantage if anything it add in more point of failure, withi agree with this and it is why we dont use it we use dovecots deliver with postfix and have noticed no problems, not to say there was none, but if so, we dont notice it. postfix looks up the user, it determines if it accepts the mail, if it does, it queues it for mailscanner to do its stuff, then gives it back to postfix, which is then told to give it to dovecots deliver, it makes no sense to me that it should then be sent to another machine just to be stored on a remote file server, the same remote file server the initial server assigned that conenction by a true load balancer has mounted and would store it to as well.... would be miuch easier to have deliver ignore the index file by an option, eliminating the corruption risks to the index file and just storing the darm thing. or am i only one who thinks mail systems do not need to be complex to run faultlessly, I think those who feel the need to make it very complex are not only looking for trouble, but further trying to justify their position to their employer that they are indispensable.> > if director service assign 60K user to each front end, how it handle if 5K > simultaneous user login, but all 5K happen to be assign to that one machine,that would be rare, but, technically speaking, if you are that large in user numbers, it is a possible scenario> Is it really worth it? Do we really need this, or just let foundry switch > handle it as it does now. > We also have 24 front end SMTP server, these deliver mail to netapp filer, > all 24 plus 8 pop3 server and 2 webmail imap server all mount /vmail, so all > access same maildir. it seem work very effective thus far and for many manySounds similar setup to us, smtp, pop3 and webmail all mounting /var/vmail/ on a FAS2050, I've asked if it can avoid touching the index files before (see a thread as recent as a few weeks back), Timo is just not interested, to much work apparently for so little users (although I never in all hte years ive been on this list, ever seen a poll taken/question asked to users - about it, plus, well, every single dovecot user is on this list, right? <sarcasm> anyway, mostly I guess although it has risks, it seems to work for everyone who uses NFS anyway and has done for very many years :) , maybe one day when Timo is so bored and cant think of anything to add, he will give us an option, or a dedicated deliver binary separate to normal deliver that does this) /rant ( but its nice to know im not the only one here who feels this way) Cheers -------------- next part -------------- A non-text attachment was scrubbed... Name: stock_smiley-1.png Type: image/png Size: 873 bytes Desc: not available URL: <http://dovecot.org/pipermail/dovecot/attachments/20100827/39d024e2/attachment-0002.png>
Noel, On 8/26/10 9:59 PM, "Noel Butler" <noel.butler at ausics.net> wrote:>> I fail to see advantage if anything it add in more point of failure, with > > i agree with this and it is why we dont use it > > we use dovecots deliver with postfix and have noticed no problems, not > to say there was none, but if so, we dont notice it.We might be a slightly larger install than you (60k users, mail on FAS 3170 Metrocluster), but we have noticed corruption issues and the director is definitely going to see use in our shop. We still use Sendmail+procmail for delivery, so no issue there... but we've got hordes of IMAP users that will leave a client running at home, at their desk, on their phone, and then will use Webmail on their laptop. Without the director, all of these sessions end up on different backend mailservers, and it's basically a crapshoot which Dovecot instance notices a new message first. NFS locking being what it is, odds are an index will get corrupted sooner or later, and when this happens the user's mail 'disappears' until Dovecot can reindex it. The users inevitably freak out and call the helpdesk, who tells them to close and reopen their mail client. Maybe you're small enough to not run into problems, or maybe your users just have lower expectations or a higher pain threshold than ours. Either way, it's unpleasant for everyone involved, and quite easy to solve with the director proxy. Timo has been saying for YEARS that you need user-node affinity if you're doing NFS, and now he's done something about it. If you've already got a load balancer, then just point the balancer at a pool of directors, and then point the directors at your existing mailserver pool. <shameless plug> For health monitoring on the directors, check out: http://github.com/brandond/poolmon </shameless plug> -Brad
On Fri, Aug 27, 2010 at 2:59 PM, Noel Butler <noel.butler at ausics.net> wrote:> On Fri, 2010-08-27 at 08:54 +1000, Edward avanti wrote: > > > Halo, > > Please can you explain why this is advantage over a hardware load > balancer. > > > it is no advantage over a dedicated hardware solution, but director does > not do the exact same thing. > > > > I fail to see advantage if anything it add in more point of failure, with > > > i agree with this and it is why we dont use it > > we use dovecots deliver with postfix and have noticed no problems, not > to say there was none, but if so, we dont notice it. > postfix looks up the user, it determines if it accepts the mail, if it > does, it queues it for mailscanner to do its stuff, then gives it back > to postfix, which is then told to give it to dovecots deliver, it makes >I have offlist discussion with Timo, he said help with I/O, you make good case, not more I/O intense than scanning mail, delivery just like router> no sense to me that it should then be sent to another machine just to be > stored on a remote file server, the same remote file server the initial > server assigned that conenction by a true load balancer has mounted and > would store it to as well.... would be miuch easier to have deliver > ignore the index file by an option, eliminating the corruption risks to > the index file and just storing the darm thing. or am i only one who > thinks mail systems do not need to be complex to run faultlessly, I > think those who feel the need to make it very complex are not only > looking for trouble, but further trying to justify their position to > their employer that they are indispensable. > >If operation is simple, is little to go wrong, when nothing go wrong, boss happy and my job safe> > > > > > if director service assign 60K user to each front end, how it handle if > 5K > > simultaneous user login, but all 5K happen to be assign to that one > machine, > > > that would be rare, but, technically speaking, if you are that large in > user numbers, it is a possible scenario > >We have 418K mailbox users> > > Is it really worth it? Do we really need this, or just let foundry switch > > handle it as it does now. > > We also have 24 front end SMTP server, these deliver mail to netapp > filer, > > all 24 plus 8 pop3 server and 2 webmail imap server all mount /vmail, so > all > > access same maildir. it seem work very effective thus far and for many > many > > > Sounds similar setup to us, smtp, pop3 and webmail all > mounting /var/vmail/ on a FAS2050, I've asked if it can avoid touching > the index files before (see a thread as recent as a few weeks back), > Timo is just not interested, to much work apparently for so little users >Oh my, so i waste time talking asking him for extra switch to deliver to ignore indexing, drat.> (although I never in all hte years ive been on this list, ever seen a > poll taken/question asked to users - about it, plus, well, every single > dovecot user is on this list, right? <sarcasm> anyway, mostly I guess > although it has risks, it seems to work for everyone who uses NFS anyway > and has done for very many years :) , maybe one day when Timo is so > bored and cant think of anything to add, he will give us an option, or a > dedicated deliver binary separate to normal deliver that does this) > > Maybe not many people here use time proven setup> /rant ( but its nice to know im not the only one here who feels this > way) > Cheers > >
On 27.8.2010, at 5.59, Noel Butler wrote:> I've asked if it can avoid touching > the index files before (see a thread as recent as a few weeks back),You can avoid touching indexes: protocol lda { mail_location = maildir:~/Maildir:INDEX=MEMORY } But you still have the problem of dovecot-uidlist file that gets updated. Well .. maybe you could do something ugly like: protocol lda { mail_location = maildir:~/Maildir:INDEX=MEMORY:CONTROL=/tmp/controls/%u } And then once in a while rm -rf /tmp/controls, but I don't know how badly that'll work out. I guess it's possible that LDA even goes and scans through the existing cur/ directory to build a new dovecot-uidlist.
> -----Original Message----- > From: Edward avanti > > have you been told where you might go lately and do with some part your > anatomy? > this Timo list, not you list, best remember this since you nobody this listSeriously? Grow up and/or take it off-list. -Brad