Hi, First, sorry if this isn''t the place to get this kind of help... If not, I appreciate some link , forum, where I can try get some answers... My problem: * Using btrfs + compression , flush of 60 MB/s take 4 minutes.... (on this 4 minutes they keep constatly I/O of +- 4MB/s no disks) (flush from Informix database) The enviroment : * Virtualized environment * OpenSuse 12.1 64bits, running over VmWare ESXi 5 * Btrfs version : btrfsprogs-0.19-43.1.2.x86_64 * Kernel : Linux jdivm06 3.1.10-1.16-desktop #1 SMP PREEMPT Wed Jun 27 05:21:40 UTC 2012 (d016078) x86_64 x86_64 x86_64 GNU/Linux * The file system is used to keep the Informix database data (chunks). * 8 cores (intel i7) , so, all btrfs threads able to work parallel. The file system mount : root@jdivm06:/proc/sys/fs# mount |grep ifx /dev/sdb1 on /ifxdados type btrfs (rw,nosuid,nodev,noexec,noatime,compress=zlib,space_cache) My question, what I believed will help to avoid this long flush : * Have some way to force this flush all in memory cache and then use the btrfs background process to flush to disk ... Security and recover aren''t a priority for now, because this is part of a database bulkload ...after finish , integrity will be desirable (not a obligation, since this is a test environment) For now, performance is the mainly requirement... A plus : root@jdivm06:/proc/sys/fs# cat /proc/sys/vm/dirty_ratio 50 root@jdivm06:/proc/sys/fs# cat /proc/sys/vm/dirty_background_ratio 10 Thanks Cesar -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Mon, Oct 1, 2012 at 8:27 PM, Cesar Inacio Martins <cesar_inacio_martins@yahoo.com.br> wrote:> My problem: > * Using btrfs + compression , flush of 60 MB/s take 4 minutes.... > (on this 4 minutes they keep constatly I/O of +- 4MB/s no disks) > (flush from Informix database)> * OpenSuse 12.1 64bits, running over VmWare ESXi 5 > * Btrfs version : btrfsprogs-0.19-43.1.2.x86_64 > * Kernel : Linux jdivm06 3.1.10-1.16-desktop #1 SMP PREEMPT Wed Jun 27> My question, what I believed will help to avoid this long flush : > * Have some way to force this flush all in memory cache and then use the > btrfs background process to flush to disk ... > Security and recover aren''t a priority for now, because this is part of a > database bulkload ...after finish , integrity will be desirable (not a > obligation, since this is a test environment) > > For now, performance is the mainly requirement...I suggest you start by reading http://www.mail-archive.com/linux-btrfs@vger.kernel.org/msg18827.html After that, PROBABLY start your database by preloading libeatmydata to disable fsync completely. On a side note, zfs has "sync" property, which when set to "disabled", have pretty much the same effect as libeatmydata. -- Fajar -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
> I suggest you start by reading > http://www.mail-archive.com/linux-btrfs@vger.kernel.org/msg18827.html > > After that, PROBABLY start your database by preloading libeatmydata to > disable fsync completely.Which will cure the sympthoms, not the issue itself - I remember the same advice was given for Reiser4 back then ;) Usually for non-toy use-cases data is too valueable to just disable fsync. - Clemens 2012/10/1 Fajar A. Nugraha <list@fajar.net>:> On Mon, Oct 1, 2012 at 8:27 PM, Cesar Inacio Martins > <cesar_inacio_martins@yahoo.com.br> wrote: > >> My problem: >> * Using btrfs + compression , flush of 60 MB/s take 4 minutes.... >> (on this 4 minutes they keep constatly I/O of +- 4MB/s no disks) >> (flush from Informix database) > > >> * OpenSuse 12.1 64bits, running over VmWare ESXi 5 >> * Btrfs version : btrfsprogs-0.19-43.1.2.x86_64 >> * Kernel : Linux jdivm06 3.1.10-1.16-desktop #1 SMP PREEMPT Wed Jun 27 > > >> My question, what I believed will help to avoid this long flush : >> * Have some way to force this flush all in memory cache and then use the >> btrfs background process to flush to disk ... >> Security and recover aren''t a priority for now, because this is part of a >> database bulkload ...after finish , integrity will be desirable (not a >> obligation, since this is a test environment) >> >> For now, performance is the mainly requirement... > > > I suggest you start by reading > http://www.mail-archive.com/linux-btrfs@vger.kernel.org/msg18827.html > > After that, PROBABLY start your database by preloading libeatmydata to > disable fsync completely. > > On a side note, zfs has "sync" property, which when set to "disabled", > have pretty much the same effect as libeatmydata. > > -- > Fajar > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html-- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Tue, Oct 2, 2012 at 3:16 AM, Clemens Eisserer <linuxhippy@gmail.com> wrote:>> I suggest you start by reading >> http://www.mail-archive.com/linux-btrfs@vger.kernel.org/msg18827.html >> >> After that, PROBABLY start your database by preloading libeatmydata to >> disable fsync completely. > > Which will cure the sympthoms, not the issue itself - I remember the > same advice was given for Reiser4 back then ;) > Usually for non-toy use-cases data is too valueable to just disable fsync.The OP DID say he doesn''t really care about security, recovery, nor integrity (or at least, it''s not an obligatiion) :D Other than trying latest -rc and using libeatmydata, I can''t see what else can be done to improve current db performance on btrfs. As the list archive shows, zfs is currently MUCH more suitable for that. -- Fajar -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html