Hello, sorry for the long email, it's a little hard to explain this issue. The gist of it is that the Ubuntu version of parted allowed me to do something which perhaps should not be allowed i.e. creating partitions on a 2.7TB drive when the partition table is not *gpt* but *msdos*. I am trying to configure 2 identical servers, both are Dell Poweredge 2970 machines with 6 disks in them configured as a RAID 5 with one hotspare, and both give me 2.726TB of space after the RAID 5 is configured. There are slight differences between the BIOS versions and Firmware versions of the LSI disk controller, etc but I'm not sure that matters in this case. Now, I setup server "A" a few months ago and for some reason that I don't remember now I resorted to using a Ubuntu 64-bit LiveCD to create the partitions. Since the disk is larger than 2TB, I had to use 'parted' to create the partitions. So I happily created the partitions I wanted which are below: / 50 GB /var 20 GB /data the remainder (a large partition) Now, after a few months I forgot all about the Ubuntu LiveCD and tried to setup server "B" using the CentOS 5.3 x86_64 CD. However the installer immediately complained that "this disk in using a GPT partition table and this computer cannot boot using GPT" and it keeps saying this no matter what I do. I've tried creating a separate /boot partition, using LVMs for everything, etc but nothing works, even "dd" did not give me much luck, although perhaps I should try deleting the "end" of the disk rather than the beginning? I have now noticed that when press Ctrl-Alt-F2 on server "B" during the CentOS install and attempt to use 'parted' it says that the "label" type is "gpt". It allows me to create new partitions, etc but it's no help because the installer keeps complaining that the machine will not boot with a GPT partition table. So I need to really use a "msdos" partition type to boot this machine successfully, but the CentOS version of parted will not allow me to do that. The additional mystery is that if I check server "A" which I partitioned a few months ago using Ubuntu, the "label" type is "msdos"!! How is that possible? In addition if I use the CentOS CD and try to use parted on server "A" now, it gives the following error: --------------------------- # parted GNU Parted 1.8.1 Using /dev/sda Welcome to GNU Parted! Type 'help' to view a list of commands. (parted) p Error: msdos labels do not support devices that have more than 4294967295 sectors. --------------------------- BUT if I reboot this same server (server A) again using the Ubuntu LiveCD parted works just fine! It prints the label type as "msdos", and it prints the above partition table correctly and gives me the right size for it and everything. So what is going on here? Is the Ubuntu parted somehow buggy and allowing me to do something dangerous that I will regret later, or can I just ignore the label setting in parted and continue to setup Server "B" the same as "A" and hope for the best? I would appreciate any insights.
2010/2/23 Khusro Jaleel <mailing-lists at kerneljack.com>:> Hello, sorry for the long email, it's a little hard to explain this issue. The gist of it is that the Ubuntu version of parted allowed me to do something which perhaps should not be allowed i.e. creating partitions on a 2.7TB drive when the partition table is not *gpt* but *msdos*. > > I am trying to configure 2 identical servers, both are Dell Poweredge 2970 machines with 6 disks in them configured as a RAID 5 with one hotspare, and both give me 2.726TB of space after the RAID 5 is configured. There are slight differences between the BIOS versions and Firmware versions of the LSI disk controller, etc but I'm not sure that matters in this case. > > Now, I setup server "A" a few months ago and for some reason that I don't remember now I resorted to using a Ubuntu 64-bit LiveCD to create the partitions. Since the disk is larger than 2TB, I had to use 'parted' to create the partitions. So I happily created the partitions I wanted which are below:I think it is not possible to create partitions lager than 1.7TB without gpt partition table. Of course you can create small boot partition and some lvm partitions and combine them to one big lvm volume or use another drive for booting and use gpt partition table for big disk.. -- Eero
On Tue, 23 Feb 2010 at 3:38pm, Khusro Jaleel wrote> Now, after a few months I forgot all about the Ubuntu LiveCD and tried > to setup server "B" using the CentOS 5.3 x86_64 CD. However the > installer immediately complained that "this disk in using a GPT > partition table and this computer cannot boot using GPT" and it keeps > saying this no matter what I do. I've tried creating a separate /boot > partition, using LVMs for everything, etc but nothing works, even "dd" > did not give me much luck, although perhaps I should try deleting the > "end" of the disk rather than the beginning?There are 2 facts at play here: 1) Any device larger than 2TB must use a GPT disklabel. 2) You cannot boot from a device with a GPT disklabel. None of the tricks you mention above will work. What you need to do is use the RAID card BIOS to divide the array into multiple devices. Most decent RAID cards will either "auto-carve" arrays into <2TB chunks or let you create a small "boot-drive". The latter is preferable, IMO. If your RAID card doesn't offer such an option, then you'll need to either remove some disks from the array to use as boot drives or add more drives to the system.> The additional mystery is that if I check server "A" which I partitioned > a few months ago using Ubuntu, the "label" type is "msdos"!! How is that > possible? In addition if I use the CentOS CD and try to use parted on > server "A" now, it gives the following error:Weird things happen when trying to boot from GPT labeled devices, including all sorts of data-loss scenarios. -- Joshua Baker-LePain QB3 Shared Cluster Sysadmin UCSF
On Feb 23, 2010, at 9:38 AM, Khusro Jaleel wrote:> I am trying to configure 2 identical servers, both are Dell Poweredge 2970 machines with 6 disks in them configured as a RAID 5 with one hotspare, and both give me 2.726TB of space after the RAID 5 is configured. There are slight differences between the BIOS versions and Firmware versions of the LSI disk controller, etc but I'm not sure that matters in this case.I had this exact issue with the exact hardware and setup you describe above.> So what is going on here?I believe that the issue is with the Anaconda.> Is the Ubuntu parted somehow buggy and allowing me to do something dangerous that I will regret later, or can I just ignore the label setting in parted and continue to setup Server "B" the same as "A" and hope for the best? > > I would appreciate any insights.Here is what I ended up doing... In the hardware RAID controller I setup a "virtual drive" of 50GB and another virtual drive made from the remaining space. Here is a nice ASCII description of the RAID setup. 0 - 750GB SATA --- Global Hot spare 1 - 750GB SATA ----\ 2 - 750GB SATA -----\ 3 - 750GB SATA ------> RAID 5 == ~2.7 TB 4 - 750GB SATA -----/ 5 - 750GB SATA ----/ VD = Virtual Disk VD 0 (sda) - 50GB (boot volume [/boot, /, /home, /tmp, etc.]) VD 1 (sdb) - <remaining space> ("data" storage, /var) After saving those changes within the RAID controller I then booted the CentOS 5.4 x86_64 installer and installed away. Hope that helps. - tim -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2415 bytes Desc: not available URL: <http://lists.centos.org/pipermail/centos/attachments/20100223/150fc615/attachment.p7s>
Thanks for your replies, just to clear things up, here is what I am seeing. If I reboot server "A" with the Ubuntu LiveCD, I get: ---------------------------------------- # parted /dev/sda p Model: DELL PERC 5/i (scsi) Disk /dev/sda: 2998GB Sector size (logical/physical): 512B/512B Partition Table: msdos Number Start End Size Type File system Flags 1 32.3kB 53.7GB 53.7GB primary ext3 2 53.7GB 62.3GB 8595MB primary linux-swap 3 62.3GB 83.8GB 21.5GB primary ext3 4 83.8GB 2199GB 2115GB primary xfs # fdisk -l Disk /dev/sda: 2998.4 GB, 2998424043520 bytes 255 heads, 63 sectors/track, 364537 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Disk identifier: 0x852b68e5 Device Boot Start End Blocks Id System /dev/sda1 1 6528 52436128+ 83 Linux /dev/sda2 6529 7573 8393962+ 82 Linux swap / Solaris /dev/sda3 7574 10185 20980890 83 Linux /dev/sda4 10186 267349 2065669830 83 Linux ---------------------------------------- Now when I try this with CentOS, I get: ---------------------------------------- Error: msdos labels do not support devices that have more than 4294967295 sectors. ----------------------------------------- straight away. I understand what you guys are saying about GPT and not being able to boot off it, etc but how did I end up in this situation? And is this dangerous? I am thinking that if this is possible, why not try and setup the second server the same way? But it just feels wrong that Ubuntu allows this and if CentOS does not, there must be a good reason.
At Tue, 23 Feb 2010 16:11:48 +0000 CentOS mailing list <centos at centos.org> wrote:> > Thanks for your replies, just to clear things up, here is what I am seeing. > > If I reboot server "A" with the Ubuntu LiveCD, I get: > ---------------------------------------- > # parted /dev/sda p > > Model: DELL PERC 5/i (scsi) > Disk /dev/sda: 2998GB > Sector size (logical/physical): 512B/512B > Partition Table: msdos > > Number Start End Size Type File system Flags > 1 32.3kB 53.7GB 53.7GB primary ext3 > 2 53.7GB 62.3GB 8595MB primary linux-swap > 3 62.3GB 83.8GB 21.5GB primary ext3 > 4 83.8GB 2199GB 2115GB primary xfs > > # fdisk -l > > Disk /dev/sda: 2998.4 GB, 2998424043520 bytes > 255 heads, 63 sectors/track, 364537 cylinders > Units = cylinders of 16065 * 512 = 8225280 bytes > Disk identifier: 0x852b68e5 > > Device Boot Start End Blocks Id System > /dev/sda1 1 6528 52436128+ 83 Linux > /dev/sda2 6529 7573 8393962+ 82 Linux swap / Solaris > /dev/sda3 7574 10185 20980890 83 Linux > /dev/sda4 10186 267349 2065669830 83 Linux > ---------------------------------------- > > Now when I try this with CentOS, I get: > ---------------------------------------- > Error: msdos labels do not support devices that have more than 4294967295 sectors. > ----------------------------------------- > > straight away. I understand what you guys are saying about GPT and > not being able to boot off it, etc but how did I end up in this > situation? And is this dangerous? > > I am thinking that if this is possible, why not try and setup the > second server the same way? But it just feels wrong that Ubuntu allows > this and if CentOS does not, there must be a good reason.I guessing one of these things is going on: A) Ubuntu has *patched* versions of parted and fdisk that disable their error checking (!). B) Ubuntu has new versions of parted and fdisk that are more liberal than the (older) versions shipped with CentOS. What does parted -v and fdisk -v display under the Ubuntu LiveCD? BTW which version of Ubuntu are you using? Hardy (8.4) or Karma (9.1)?> > > _______________________________________________ > CentOS mailing list > CentOS at centos.org > http://lists.centos.org/mailman/listinfo/centos > >-- Robert Heller -- 978-544-6933 Deepwoods Software -- Download the Model Railroad System http://www.deepsoft.com/ -- Binaries for Linux and MS-Windows heller at deepsoft.com -- http://www.deepsoft.com/ModelRailroadSystem/
On 23 Feb 2010, at 18:02, Robert Heller wrote:> I guessing one of these things is going on: > > A) Ubuntu has *patched* versions of parted and fdisk that disable their > error checking (!). > > B) Ubuntu has new versions of parted and fdisk that are more liberal > than the (older) versions shipped with CentOS. > > What does > > parted -v > and > fdisk -vYes, that is possible. The versions are: # parted -v: 1.8.8 # fdisk -v: util-linux-ng 2.14.2
Khusro Jaleel
2010-Mar-08 12:53 UTC
[CentOS] which FS is appropriate for a 2TB-sized partition?
Thanks to all of you for your help, and especially Tim Shubitz who faced the same problem and his solution worked perfectly for me. However, now that I have properly created a GPT partition of size 2.7TB, which filesystem is best on it? This filesystem will be used to store backups of various other linux systems, so the files will be mostly small, however some systems do host big movie files and sometimes SVN dumps, and DB dumps can get a little big. I am going to be using rsnapshot to do the backups, so perhaps I should be careful about the number of inodes I create and try to maximise them? I am thinking of using XFS, but am not sure. I seem to have heard in the past that one should avoid EXT3 on such huge filesystems, but I can't find a reference or proper justification for it. JFS is another option but then some mailing list threads online say it has lost data for them so I'm a bit confused as to what is best to use in my scenario. As for XFS I have read that a UPS is necessary and this is not a problem since these machines are already connected to a UPS (and that UPS has a backup as well). Any help appreciated, thanks, Khusro