I want to generate or extend a large file in an ext4 filesystem allocating space (i.e. not creating a sparse file) but not actually writing any data. I realise that this will result in the file containing the contents of whatever was there on the disk before, which is a possible security problem in some circumstances, but it isn't a problem here. Ideally what I'd like is a "make unsparse" bit of code. I'm happy for this to use the libraries, and work on an unmounted fs (indeed that is probably better). Supplementary question: can I assume that if a non-sparse file is on disk and never opened, and never unlinked, then the sectors used to to store that file's data will never change irrespective of other operations on the ext4 filesystem? IE nothing is shuffling where ext4 files are stored. -- Alex Bligh
On Sun, Oct 31, 2010 at 11:12:41 +0100, Alex Bligh <alex at alex.org.uk> wrote:> I want to generate or extend a large file in an ext4 filesystem allocating > space (i.e. not creating a sparse file) but not actually writing any data. > I realise that this will result in the file containing the contents of > whatever was there on the disk before, which is a possible security problem > in some circumstances, but it isn't a problem here.There isn't going to be a way to do that through the file system, because as you note it is a security problem. What is the high level thing you are trying to accomplish here? Modifying the filesystem offline seems risky and maybe there is a safer way to accomplish your goals.> Supplementary question: can I assume that if a non-sparse file is on disk > and never opened, and never unlinked, then the sectors used to to store > that file's data will never change irrespective of other operations on the > ext4 filesystem? IE nothing is shuffling where ext4 files are stored.I think SSDs will move stuff around at a very low level. They would look like they are at the same place to stuff access the device like a disk, but physically would be stored in a different hardware location. With normal disks, you'd only see this if the device got a read error, but was able to successfully read a marginal sector and remap it to a spare sector. But again, stuff talking to the disk will see it at the same address.
On Sun, Oct 31, 2010 at 11:12:41AM +0100, Alex Bligh wrote:> I want to generate or extend a large file in an ext4 filesystem allocating > space (i.e. not creating a sparse file) but not actually writing any data.Well, some metadata will have to be written, but not data. shouldn't posix_fallocate(3) and/or fallocate(2) do that? I haven't got EXT4 around ATM, but IIRC it should work on it too. On XFS it seems to work too: # time fallocate -l 3000000000 /stuff/tmp/bla fallocate -l 3000000000 /stuff/tmp/bla 0,00s user 0,00s system 0% cpu 0,402 total # du -h /stuff/tmp/bla 2,8G /stuff/tmp/bla # du -bh /stuff/tmp/bla 2,8G /stuff/tmp/bla # rm -f /stuff/tmp/bla fallocate(1) is from util-linux on my Debian Squeeze Oppose that to dramatically slower dd(1), which fills them with zeros explicitely: # time dd if=/dev/zero of=/stuff/tmp/bla count=30000 bs=100000 time dd if=/dev/zero of=/stuff/tmp/bla count=30000 bs=100000 30000+0 records in 30000+0 records out 3000000000 bytes (3,0 GB) copied, 31,2581 s, 96,0 MB/s dd if=/dev/zero of=/stuff/tmp/bla count=30000 bs=100000 0,00s user 3,41s system 10% cpu 31,341 total # du -h /stuff/tmp/bla 2,8G /stuff/tmp/bla -- Opinions above are GNU-copylefted.