Hello all, I have a DL360 G4 1U server that does a wonderfull job with dovecot horde, Xmail and OpenLDAP for a company and serving about 40 acouunts. The machine is wonderful. I am very happy with it. However, I am running out of disk space. It has two times 76Gb Drives in RAID1 (disk mirroring) and the capacity has reached 82%.? I am starting of getting nervous. Does anyone know of a painless way to migrate the entire contents directly to another pair of 146Gb SCSI RAID1 disks ? I thought of downtime and using clonezilla, but my last experience with it was questionable. I remember having problems declaring disk re-sizing from the smaller capacity drives to the larger ones. CentOS 5.5 Manual install of : Mysql XMail (pop3/smtp) ASSP (anti spam) Apache / LAMP and last but by no means list : Dovecot Dovecot -n : # 1.2.16: /etc/dovecot/dovecot.conf # OS: Linux 2.6.18-194.17.4.el5 i686 CentOS release 5.5 (Final) ext3 base_dir: /var/run/dovecot/ log_path: /var/log/dovecot/dovecot.log info_log_path: /var/log/dovecot/dovecot-info.log ssl_parameters_regenerate: 48 verbose_ssl: yes login_dir: /var/run/dovecot//login login_executable: /usr/local/dovecot/libexec/dovecot/imap-login login_greeting: * Dovecot ready * login_max_processes_count: 96 mail_location: maildir:/var/MailRoot/domains/%d/%n/Maildir mail_plugins: zlib auth default: ? verbose: yes ? debug: yes ? debug_passwords: yes ? passdb: ? ? driver: passwd-file ? ? args: /etc/dovecot/passwd ? passdb: ? ? driver: pam ? userdb: ? ? driver: static ? ? args: uid=vmail gid=vmail home=/home/vmail/%u ? userdb: ? ? driver: passwd Any help would be appreciated or any ideas you might have. Regards, spyros ---- "I merely function as a channel that filters music through the chaos of noise" - Vangelis?
Hello Spyros, As "best practice" you never have the OS and the data/logs/user homes on the same partition or set of disks. If this is the case then your life is pretty easy: -simply create the new set of partitions -mount the new ones in a temporary location -rsync (or copy everything from old partitions) -Stop dovecot / all other daemons that might be using the data -mount the new ones in the place of old ones, mount the old ones in the place of new ones - rsync again (should be quick as not many things changed) - start all your deamons again :P If you do not have separate partitions maybe this is the perfect time to look into that... I would also look into btrfs... might be a good pick for your new partitions. best regards, Andrei> Hello all, > > I have a DL360 G4 1U server that does a wonderfull job with dovecot horde, > Xmail and OpenLDAP for a company and serving about 40 acouunts. > > The machine is wonderful. I am very happy with it. > However, I am running out of disk space. > It has two times 76Gb Drives in RAID1 (disk mirroring) and the capacity > has reached 82%.? > > I am starting of getting nervous. > > Does anyone know of a painless way to migrate the entire contents directly > to another pair of 146Gb SCSI RAID1 disks ? > > I thought of downtime and using clonezilla, but my last experience with it > was questionable. I remember having problems declaring disk re-sizing > from the smaller capacity drives to the larger ones. > > CentOS 5.5 > Manual install of : > > Mysql > XMail (pop3/smtp) > ASSP (anti spam) > Apache / LAMP > and last but by no means list : Dovecot > > Dovecot -n : > > # 1.2.16: /etc/dovecot/dovecot.conf > # OS: Linux 2.6.18-194.17.4.el5 i686 CentOS release 5.5 (Final) ext3 > base_dir: /var/run/dovecot/ > log_path: /var/log/dovecot/dovecot.log > info_log_path: /var/log/dovecot/dovecot-info.log > ssl_parameters_regenerate: 48 > verbose_ssl: yes > login_dir: /var/run/dovecot//login > login_executable: /usr/local/dovecot/libexec/dovecot/imap-login > login_greeting: * Dovecot ready * > login_max_processes_count: 96 > mail_location: maildir:/var/MailRoot/domains/%d/%n/Maildir > mail_plugins: zlib > auth default: > ? verbose: yes > ? debug: yes > ? debug_passwords: yes > ? passdb: > ? ? driver: passwd-file > ? ? args: /etc/dovecot/passwd > ? passdb: > ? ? driver: pam > ? userdb: > ? ? driver: static > ? ? args: uid=vmail gid=vmail home=/home/vmail/%u > ? userdb: > ? ? driver: passwd > > > Any help would be appreciated or any ideas you might have. > > Regards, > > spyros > > > > > > > ---- > "I merely function as a channel that filters > music through the chaos of noise" > - Vangelis? > > !DSPAM:50609d2c301831828332458! > >
Am 24.09.2012 19:42, schrieb Spyros Tsiolis:> Hello all, > > I have a DL360 G4 1U server that does a wonderfull job with dovecot horde, > Xmail and OpenLDAP for a company and serving about 40 acouunts. > > The machine is wonderful. I am very happy with it. > However, I am running out of disk space. > It has two times 76Gb Drives in RAID1 (disk mirroring) and the capacity > has reached 82%. > > I am starting of getting nervous. > > Does anyone know of a painless way to migrate the entire contents directly > to another pair of 146Gb SCSI RAID1 disks ? > > I thought of downtime and using clonezilla, but my last experience with it > was questionable. I remember having problems declaring disk re-sizing > from the smaller capacity drives to the larger ones. > > CentOS 5.5 > Manual install of : > > Mysql > XMail (pop3/smtp) > ASSP (anti spam) > Apache / LAMP > and last but by no means list : Dovecot > > Dovecot -n : > > # 1.2.16: /etc/dovecot/dovecot.conf > # OS: Linux 2.6.18-194.17.4.el5 i686 CentOS release 5.5 (Final) ext3 > base_dir: /var/run/dovecot/ > log_path: /var/log/dovecot/dovecot.log > info_log_path: /var/log/dovecot/dovecot-info.log > ssl_parameters_regenerate: 48 > verbose_ssl: yes > login_dir: /var/run/dovecot//login > login_executable: /usr/local/dovecot/libexec/dovecot/imap-login > login_greeting: * Dovecot ready * > login_max_processes_count: 96 > mail_location: maildir:/var/MailRoot/domains/%d/%n/Maildir > mail_plugins: zlib > auth default: > verbose: yes > debug: yes > debug_passwords: yes > passdb: > driver: passwd-file > args: /etc/dovecot/passwd > passdb: > driver: pam > userdb: > driver: static > args: uid=vmail gid=vmail home=/home/vmail/%u > userdb: > driver: passwd > > > Any help would be appreciated or any ideas you might have. > > Regards, > > spyros >rsync should do the job depending on your whole machine setup it might only be only umount old /home and mount new(bigger) /home after sync ,perhaps with tmp store elsewhere ( for sure you have to have a plan before doing..) but your dovecot is very outdated, i would recommend get up to new hard and software/os install, and then migrate to new machine> > > > > > ---- > "I merely function as a channel that filters > music through the chaos of noise" > - Vangelis >-- Best Regards MfG Robert Schetterer
This is one of those questions which is almost too easy if you are familiar with Linux. Trying not to sound like a d*ck, but is it an option to rent someone to help with admin jobs? For example, were it me then I would probably have setup some partitioning scheme with separate partitions for data and operating system? Possibly also using LVM? You have several options, mainly the choice of filesystem will dictate here, but quite possibly you can: 1) Pull the drives one by one and rebuild the raid after each. Keep the old drives since you can technically roll back onto them. Expand the partitions (scary without LVM) and then expand the filesystem on the partitions 2) Boot from a DVD/Flash on your favourite rescue distro (I like sysrecuecd). Create the new raid, copy the old to the new, remove the old drives, reboot from new. Possibly taking the time to repartition and move some data around while you do it (remember to update fstab) Both are fairly simple if you have done it once, but it would be well worth finding someone either local or who will log in via remote control and support you? Final thought: For the size of drives you are looking at, SSD drives are relatively inexpensive and likely comparable with the high end drives you are probably looking to buy? For 40 users I would hazard a guess you likely would be happy with inexpensive low end drives, but certainly a couple of small SSDs will blow away a spinning disk and give you a decent upgrade... Good luck Ed W On 24/09/2012 18:42, Spyros Tsiolis wrote:> Hello all, > > I have a DL360 G4 1U server that does a wonderfull job with dovecot horde, > Xmail and OpenLDAP for a company and serving about 40 acouunts. > > The machine is wonderful. I am very happy with it. > However, I am running out of disk space. > It has two times 76Gb Drives in RAID1 (disk mirroring) and the capacity > has reached 82%. > > I am starting of getting nervous. > > Does anyone know of a painless way to migrate the entire contents directly > to another pair of 146Gb SCSI RAID1 disks ? > > I thought of downtime and using clonezilla, but my last experience with it > was questionable. I remember having problems declaring disk re-sizing > from the smaller capacity drives to the larger ones. > > CentOS 5.5 > Manual install of : > > Mysql > XMail (pop3/smtp) > ASSP (anti spam) > Apache / LAMP > and last but by no means list : Dovecot > > Dovecot -n : > > # 1.2.16: /etc/dovecot/dovecot.conf > # OS: Linux 2.6.18-194.17.4.el5 i686 CentOS release 5.5 (Final) ext3 > base_dir: /var/run/dovecot/ > log_path: /var/log/dovecot/dovecot.log > info_log_path: /var/log/dovecot/dovecot-info.log > ssl_parameters_regenerate: 48 > verbose_ssl: yes > login_dir: /var/run/dovecot//login > login_executable: /usr/local/dovecot/libexec/dovecot/imap-login > login_greeting: * Dovecot ready * > login_max_processes_count: 96 > mail_location: maildir:/var/MailRoot/domains/%d/%n/Maildir > mail_plugins: zlib > auth default: > verbose: yes > debug: yes > debug_passwords: yes > passdb: > driver: passwd-file > args: /etc/dovecot/passwd > passdb: > driver: pam > userdb: > driver: static > args: uid=vmail gid=vmail home=/home/vmail/%u > userdb: > driver: passwd > > > Any help would be appreciated or any ideas you might have. > > Regards, > > spyros > > > > > > > ---- > "I merely function as a channel that filters > music through the chaos of noise" > - Vangelis
Hello Spyros, Oupss... the DL360 G4 has only 2 bays and no external SCSI/SATA connector... so the solution below does not really apply to you :( Andrei> Hello Spyros, > > As "best practice" you never have the OS and the data/logs/user homes on > the same partition or set of disks. > > If this is the case then your life is pretty easy: > -simply create the new set of partitions > -mount the new ones in a temporary location > -rsync (or copy everything from old partitions) > -Stop dovecot / all other daemons that might be using the data > -mount the new ones in the place of old ones, mount the old ones in the > place of new ones > - rsync again (should be quick as not many things changed) > - start all your deamons again :P > > If you do not have separate partitions maybe this is the perfect time to > look into that... > > I would also look into btrfs... might be a good pick for your new > partitions. > > best regards, > Andrei > >> Hello all, >> >> I have a DL360 G4 1U server that does a wonderfull job with dovecot >> horde, >> Xmail and OpenLDAP for a company and serving about 40 acouunts. >> >> The machine is wonderful. I am very happy with it. >> However, I am running out of disk space. >> It has two times 76Gb Drives in RAID1 (disk mirroring) and the capacity >> has reached 82%.? >> >> I am starting of getting nervous. >> >> Does anyone know of a painless way to migrate the entire contents >> directly >> to another pair of 146Gb SCSI RAID1 disks ? >> >> I thought of downtime and using clonezilla, but my last experience with >> it >> was questionable. I remember having problems declaring disk re-sizing >> from the smaller capacity drives to the larger ones. >> >> CentOS 5.5 >> Manual install of : >> >> Mysql >> XMail (pop3/smtp) >> ASSP (anti spam) >> Apache / LAMP >> and last but by no means list : Dovecot >> >> Dovecot -n : >> >> # 1.2.16: /etc/dovecot/dovecot.conf >> # OS: Linux 2.6.18-194.17.4.el5 i686 CentOS release 5.5 (Final) ext3 >> base_dir: /var/run/dovecot/ >> log_path: /var/log/dovecot/dovecot.log >> info_log_path: /var/log/dovecot/dovecot-info.log >> ssl_parameters_regenerate: 48 >> verbose_ssl: yes >> login_dir: /var/run/dovecot//login >> login_executable: /usr/local/dovecot/libexec/dovecot/imap-login >> login_greeting: * Dovecot ready * >> login_max_processes_count: 96 >> mail_location: maildir:/var/MailRoot/domains/%d/%n/Maildir >> mail_plugins: zlib >> auth default: >> ? verbose: yes >> ? debug: yes >> ? debug_passwords: yes >> ? passdb: >> ? ? driver: passwd-file >> ? ? args: /etc/dovecot/passwd >> ? passdb: >> ? ? driver: pam >> ? userdb: >> ? ? driver: static >> ? ? args: uid=vmail gid=vmail home=/home/vmail/%u >> ? userdb: >> ? ? driver: passwd >> >> >> Any help would be appreciated or any ideas you might have. >> >> Regards, >> >> spyros >> >> >> >> >> >> >> ---- >> "I merely function as a channel that filters >> music through the chaos of noise" >> - Vangelis? >> >> >> >> > > > > !DSPAM:5060a006309197419291868! > >
Am 24.09.2012 20:07, schrieb Michescu Andrei:> Hello Spyros, > > Oupss... the DL360 G4 has only 2 bays and no external SCSI/SATA > connector... so the solution below does not really apply to you :( > > Andreidepends how long downtime is acceptable i.e go rsync tmp over usb storage, or simple tmp nfs mounts to other servers are thinkable to minimize downtime should be no big problem its only one server with maildir and less mailboxes and data but thats all much off topic with dovecot -- Best Regards MfG Robert Schetterer
On 24/09/2012 19:07, Ed W wrote:> This is one of those questions which is almost too easy if you are > familiar with Linux. Trying not to sound like a d*ck, but is it an > option to rent someone to help with admin jobs? For example, were it > me then I would probably have setup some partitioning scheme with > separate partitions for data and operating system? Possibly also using > LVM?That came out wrong... What I meant to say was something more like "if you were to employ someone locally they would probably give you a whole bunch of ideas on how you could adjust the setup of the server to be more future proof. It would be worth working with someone just to get that right. For example, here are some ideas that occur to me that you could use ..." Sorry, should re-read my words before hitting send Ed
Spyros Tsiolis skrev den 24-09-2012 19:42:> Any help would be appreciated or any ideas you might have.try google "centos cloud server" if you would like to do it local, use all 4 drives with 2 raid1 in the same controller if possible, then use sysrescue cd to tar it all over to the other raid1 while its down no matter how, it will be downtime i am not using centos here so i cant be more specifik http://www.sysresccd.org
On 09/24/2012 01:42 PM, Spyros Tsiolis wrote:> Hello all, > > I have a DL360 G4 1U server that does a wonderfull job with dovecot horde, > Xmail and OpenLDAP for a company and serving about 40 acouunts. > > The machine is wonderful. I am very happy with it. > However, I am running out of disk space. > It has two times 76Gb Drives in RAID1 (disk mirroring) and the capacity > has reached 82%. > > I am starting of getting nervous. > > Does anyone know of a painless way to migrate the entire contents directly > to another pair of 146Gb SCSI RAID1 disks ? > > I thought of downtime and using clonezilla, but my last experience with it > was questionable. I remember having problems declaring disk re-sizing > from the smaller capacity drives to the larger ones.We've done this on the same hardware. You can pick up these servers for cheap; just buy an extra one. Take the new machine, throw two big disks in it, and install Gentoo. Rsync the important stuff. Make sure all of the services are working on the new machine. When you're ready to make the switch, disable external networking on the current live server. Rsync everything again, and then turn the old server off. Add its IP address to the new server. Maybe kick your router's ARP cache to expedite the change. It should only cause a minute or two of downtime.
Spyros Tsiolis schreef:> Hello all, > > I have a DL360 G4 1U server that does a wonderfull job with dovecot horde, > Xmail and OpenLDAP for a company and serving about 40 acouunts. > > The machine is wonderful. I am very happy with it. > However, I am running out of disk space. > It has two times 76Gb Drives in RAID1 (disk mirroring) and the capacity > has reached 82%. > > I am starting of getting nervous. > > Does anyone know of a painless way to migrate the entire contents directly > to another pair of 146Gb SCSI RAID1 disks ? > > I thought of downtime and using clonezilla, but my last experience with it > was questionable. I remember having problems declaring disk re-sizing > from the smaller capacity drives to the larger ones. > > CentOS 5.5 > Manual install of : > > Mysql > XMail (pop3/smtp) > ASSP (anti spam) > Apache / LAMP > and last but by no means list : Dovecot > > Dovecot -n : > > # 1.2.16: /etc/dovecot/dovecot.conf > # OS: Linux 2.6.18-194.17.4.el5 i686 CentOS release 5.5 (Final) ext3 > base_dir: /var/run/dovecot/ > log_path: /var/log/dovecot/dovecot.log > info_log_path: /var/log/dovecot/dovecot-info.log > ssl_parameters_regenerate: 48 > verbose_ssl: yes > login_dir: /var/run/dovecot//login > login_executable: /usr/local/dovecot/libexec/dovecot/imap-login > login_greeting: * Dovecot ready * > login_max_processes_count: 96 > mail_location: maildir:/var/MailRoot/domains/%d/%n/Maildir > mail_plugins: zlib > auth default: > verbose: yes > debug: yes > debug_passwords: yes > passdb: > driver: passwd-file > args: /etc/dovecot/passwd > passdb: > driver: pam > userdb: > driver: static > args: uid=vmail gid=vmail home=/home/vmail/%u > userdb: > driver: passwd > > > Any help would be appreciated or any ideas you might have. > > Regards, > > spyros > > > > > > > ---- > "I merely function as a channel that filters > music through the chaos of noise" > - VangelisA solution you could try is add a new server to the mix. Leave the OS on the DL360 and put the mail data on a second machine. Then use NFS to mount the new data store. If you use a ZFS solution, expanding your data store is a no brainer. 146 GB drives are expensive, for 40 users SATA drives can do the job without any problem. gr Johan Hendriks
On 9/25/2012 12:29 AM, Spyros Tsiolis wrote:> My clients don't invest in hardware. I live in Greece. Things are really > bad right now. Not that if they were better they (the client) would invest > in their infrastructure....> Thank you very much Stan for you kind reply. > > I reply to you in private since I was told to stop the thread > on the dovecot list because it's off-topic.I'm going to bring it back on topic. To recap the issue: 1. 73GB volume, ~82% full. Volume contains OS, spool, and mailboxes 2. Two 146GB drives on hand that are impossible to employ in either a live or offline migration due to hardware limitations--drive slots (0.82*73GB=60GB)-5GB(OS)= ~55GB/40= ~1.375GB/user mailbox (This math assumes the box has already been scoured and things like source files and other temporary stuff has already been removed, along with anything and everything not necessary to the primary functions of this box) 1.375GB/user suggests these users seem to horde old emails, or simply don't keep their mailboxes tidy. Normally I'd guess that a lot of space in a situation like this is taken up by full text search indexes, but you don't appear to have these enabled. I notice in your 'dovecot -n' output that you are already employing zlib, but to what degree is currently unknown to the list. Are you compressing maildirs with a nightly script? If not, and you're compressing on an ad hoc basis, realtime compression during write could save you quite a bit of additional space, extending the operating life of your current drives. Of course a simple request to your users to clean up their mailboxes would surely help as well, if that's a possibility--some users would flog one for making such a suggestion. If you're not fully taking advantage of compression currently, you may want to consider upgrading to Dovecot 2.x which allows transparent compression to dbox and maildir storage. You may want to migrate to mdbox message store format while still using zlib, and enable single instance attachment storage (SIS). As the name suggests, SIS saves an attachment only once, regardless how many user emails in which it exists. See: http://wiki2.dovecot.org/Plugins/Zlib I'd point you to the docs for SIS but I'm unable to find them in the Wiki. Surely someone else can provide the settings and/or update the wiki. -- Stan
On 9/24/2012 7:42 PM, Spyros Tsiolis wrote:> Hello all, > > I have a DL360 G4 1U server that does a wonderfull job with dovecot horde, > Xmail and OpenLDAP for a company and serving about 40 acouunts. > > The machine is wonderful. I am very happy with it. > However, I am running out of disk space. > It has two times 76Gb Drives in RAID1 (disk mirroring) and the capacity > has reached 82%. > > I am starting of getting nervous. > > Does anyone know of a painless way to migrate the entire contents directly > to another pair of 146Gb SCSI RAID1 disks ? > > I thought of downtime and using clonezilla, but my last experience with it > was questionable. I remember having problems declaring disk re-sizing > from the smaller capacity drives to the larger ones. > > CentOS 5.5 > Manual install of : > > Mysql > XMail (pop3/smtp) > ASSP (anti spam) > Apache / LAMP > and last but by no means list : Dovecot<SNIP> It really depends on the raid you have. is it software or hardware raid? if it's software raid it will be more then simple to do it while taking one drive out put a new one in and use let say FINNIX linux from cd\dvd\usb and manage the whole partitioning copying etc from another OS while not harming anything in the old HDDs. you will might need to setup a new partitions manually on the new drive but just make a plan try it on some small VM to make sure the steps you are doing are fine for centos 5.5 and go for it. rsync is your friend!! in this case. someone mentioned here Gentoo which one installation of the system can give you very big backgorund on manual partitioning chrooting and other basic stuff that can help you in the process. Regards, Eliezer> Any help would be appreciated or any ideas you might have. > > Regards, > > spyros > > > > > > > ---- > "I merely function as a channel that filters > music through the chaos of noise" > - Vangelis