CentOS virt - Apr 2015 - CentOS Images on AWS with partitions on /dev/xvda1 are awkwared to resize

I'm staring at the free CentOS images on AWS, and seeing that whoever
set those up elected to use a partition for /dev/xvda1 rather than
taking advantage of Amazon's tendency to use "/dev/xvda",
"/dev/xvdb",
etc. for each disk and use those directly as a file system.

The result is that if you elect to allocate a larger base disk image,
for example allocating 50 Gig to allow local home directories or space
for "mock" or for bulky logs, and don't spend the time to select
and
allocate new disk images, it's awkward to simply expand the "/"
partition. And with only 8 Gig allocated in the latest CentOS 6 images
that I see in AWS, it's possible to get pretty pressed for space
pretty quickly.

Now, AWS published guidelines on manipulating partition size, and
expanding a matching filesystem, but they're very clear to "unmount
the parition before you touch it!!!" That's a bit difficult to unmount
with a "/" partition, and they understandably don't have the kind
of
"boot from CD and work from the console" setup I'd normally use
for
that kind of work.

So: why did the creators of that CentOS AMI elect to use such a small
/ partition? And how dangerous is it, with the system essentially
idle, to use "parted" to expand the "/dev/xvda1" parition
and then use
"resize2fs" to expand the "/" file system while the system
is alive?

Note that, because I'm a complete weasel, I know at least one way
around this: add a second disk, copy the OS to *that*, set grub to
boot from the second disk, reboot from that, paritition the first disk
as desired, copy the OS back, reset grub to boot from the first disk,
and pray. I've had good success with the approach in the past, and
have rebuilt rougly 15,000 Linux systems this way. But the work
predates CentOS, and I dont't want to go through that again.

So, has anyone resized "/" successfully and gracefully on AWS CentOS
instances?

                             Nico Kadel-Garcia

This is not really a problem at all.
when you launch your image for the first time, you can specify a larger / 
volume size and cloud-init-tools will take care of the rest.
This is well documented in the AWS userguides.

-- Kelly Prescott

On Wed, 29 Apr 2015, Nico Kadel-Garcia wrote:
> I'm staring at the free CentOS images on AWS, and seeing that whoever
> set those up elected to use a partition for /dev/xvda1 rather than
> taking advantage of Amazon's tendency to use "/dev/xvda",
"/dev/xvdb",
> etc. for each disk and use those directly as a file system.
>
> The result is that if you elect to allocate a larger base disk image,
> for example allocating 50 Gig to allow local home directories or space
> for "mock" or for bulky logs, and don't spend the time to
select and
> allocate new disk images, it's awkward to simply expand the
"/"
> partition. And with only 8 Gig allocated in the latest CentOS 6 images
> that I see in AWS, it's possible to get pretty pressed for space
> pretty quickly.
>
> Now, AWS published guidelines on manipulating partition size, and
> expanding a matching filesystem, but they're very clear to
"unmount
> the parition before you touch it!!!" That's a bit difficult to
unmount
> with a "/" partition, and they understandably don't have the
kind of
> "boot from CD and work from the console" setup I'd normally
use for
> that kind of work.
>
> So: why did the creators of that CentOS AMI elect to use such a small
> / partition? And how dangerous is it, with the system essentially
> idle, to use "parted" to expand the "/dev/xvda1"
parition and then use
> "resize2fs" to expand the "/" file system while the
system is alive?
>
> Note that, because I'm a complete weasel, I know at least one way
> around this: add a second disk, copy the OS to *that*, set grub to
> boot from the second disk, reboot from that, paritition the first disk
> as desired, copy the OS back, reset grub to boot from the first disk,
> and pray. I've had good success with the approach in the past, and
> have rebuilt rougly 15,000 Linux systems this way. But the work
> predates CentOS, and I dont't want to go through that again.
>
> So, has anyone resized "/" successfully and gracefully on AWS
CentOS instances?
>
>                             Nico Kadel-Garcia
> _______________________________________________
> CentOS-virt mailing list
> CentOS-virt at centos.org
> http://lists.centos.org/mailman/listinfo/centos-virt
>

to follow-up, I will give an example.
Here is the listing for the official centos AMI:

IMAGE   ami-96a818fe    aws-marketplace/CentOS 7 x86_64 (2014_09_29) EBS 
HVM-b7ee8a69-ee97-4a49-9e68-afaee216db2e-ami-d2a117ba.2 
aws-marketplace available       public  [marketplace: 
aw0evgkw8e5c1q413zgy5pjce]        x86_64  machineebs      hvm     xen
BLOCKDEVICEMAPPING      EBS     /dev/sda1               snap-591037fd   8 
false   standard                Not Encrypted
as you can see the block device mapping is by default set to 
BLOCKDEVICEMAPPING      EBS     /dev/sda1               snap-591037fd   8
false   standard                Not Encrypted

it is a standard volume, not encrypted, and 8 GB
my modification consists in adding this to my run command for my ami 
launch:
  -b /dev/sda1=snap-591037fd:20:false:gp2

I set the drive the same, the snapshot the same, and I give it 20GB 
instead of 8, I also use the gp2 type instead of the standard as well as 
telling it not to delete the volume when the instance terminates.

Hope this helps.
kp




On Wed, 29 Apr 2015, Nico Kadel-Garcia wrote:
> I'm staring at the free CentOS images on AWS, and seeing that whoever
> set those up elected to use a partition for /dev/xvda1 rather than
> taking advantage of Amazon's tendency to use "/dev/xvda",
"/dev/xvdb",
> etc. for each disk and use those directly as a file system.
>
> The result is that if you elect to allocate a larger base disk image,
> for example allocating 50 Gig to allow local home directories or space
> for "mock" or for bulky logs, and don't spend the time to
select and
> allocate new disk images, it's awkward to simply expand the
"/"
> partition. And with only 8 Gig allocated in the latest CentOS 6 images
> that I see in AWS, it's possible to get pretty pressed for space
> pretty quickly.
>
> Now, AWS published guidelines on manipulating partition size, and
> expanding a matching filesystem, but they're very clear to
"unmount
> the parition before you touch it!!!" That's a bit difficult to
unmount
> with a "/" partition, and they understandably don't have the
kind of
> "boot from CD and work from the console" setup I'd normally
use for
> that kind of work.
>
> So: why did the creators of that CentOS AMI elect to use such a small
> / partition? And how dangerous is it, with the system essentially
> idle, to use "parted" to expand the "/dev/xvda1"
parition and then use
> "resize2fs" to expand the "/" file system while the
system is alive?
>
> Note that, because I'm a complete weasel, I know at least one way
> around this: add a second disk, copy the OS to *that*, set grub to
> boot from the second disk, reboot from that, paritition the first disk
> as desired, copy the OS back, reset grub to boot from the first disk,
> and pray. I've had good success with the approach in the past, and
> have rebuilt rougly 15,000 Linux systems this way. But the work
> predates CentOS, and I dont't want to go through that again.
>
> So, has anyone resized "/" successfully and gracefully on AWS
CentOS instances?
>
>                             Nico Kadel-Garcia
> _______________________________________________
> CentOS-virt mailing list
> CentOS-virt at centos.org
> http://lists.centos.org/mailman/listinfo/centos-virt
>

On Wed, Apr 29, 2015 at 11:24 PM, Kelly Prescott <kprescott at coolip.net>
wrote:
> This is not really a problem at all.
> when you launch your image for the first time, you can specify a larger /
> volume size and cloud-init-tools will take care of the rest.
> This is well documented in the AWS userguides.
>
> -- Kelly Prescott
I just had that discussion with an experienced AWS user, who hadn't
noticed that the documentation at
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/storage_expand_partition.html
says the following, very clearly:

      Unmount the partition if it is mounted. Run the umount command
with the value of MOUNTPOINT from the lsblk command. In this example,
the MOUNTPOINT value for the partition is /mnt.

Notice the "unmount the parition part".  This is not feasible with a
"/" partition unless you've booted from separate media and have
console access or remote SSH access with that separate media. I'm
experienced enough to actually configure a second disk and ensure
booting from *that* to re-arrange the primary disk's filesystems, but
I'm looking for a better way.

I'm also staring at the "cloud-init" docs you mentioned at
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AmazonLinuxAMIBasics.html#CloudInit.
I see nothing about resizing the base OS image size, especially the
"/" partition. Do you see something relevant?

                                Nico Kadel-Garcia

On Wed, Apr 29, 2015 at 11:33 PM, Kelly Prescott <kprescott at coolip.net>
wrote:
> to follow-up, I will give an example.
> Here is the listing for the official centos AMI:
>
> IMAGE   ami-96a818fe    aws-marketplace/CentOS 7 x86_64 (2014_09_29) EBS
> HVM-b7ee8a69-ee97-4a49-9e68-afaee216db2e-ami-d2a117ba.2 aws-marketplace
> available       public  [marketplace: aw0evgkw8e5c1q413zgy5pjce]
> x86_64  machineebs      hvm     xen
> BLOCKDEVICEMAPPING      EBS     /dev/sda1               snap-591037fd   8
> false   standard                Not Encrypted
> as you can see the block device mapping is by default set to
> BLOCKDEVICEMAPPING      EBS     /dev/sda1               snap-591037fd   8
> false   standard                Not Encrypted
>
> it is a standard volume, not encrypted, and 8 GB
> my modification consists in adding this to my run command for my ami
launch:
>  -b /dev/sda1=snap-591037fd:20:false:gp2
>
> I set the drive the same, the snapshot the same, and I give it 20GB instead
> of 8, I also use the gp2 type instead of the standard as well as telling it
> not to delete the volume when the instance terminates.
>
> Hope this helps.
> kp
Perhaps so, and I appreciate the pointer. I can try to work with that
to integrate command line based deployment and get this option.

So you're working from the command line tools in the EPEL
'cloud-init'
package, not the AWS GUI? Because when I tried expanding the size of
the base disk image in the GUI, I wound up with an an 8 Gig default
/dev/xvda1 on a 20 Gig /dev/xvda. That's why I was looking at "how do
I resize this thing safel?"

Unfortunately, it doesn't help a lot with what I already have built,
but could be useful going forward.