Hi. Here you can find patches with changes to gmirror(8) and graid3(8): people.freebsd.org/~pjd/patches/gmirror.7.patch people.freebsd.org/~pjd/patches/graid3.patch The patches does the following: - Significant synchronization speed improvement. Now many parallel synchronization I/O requests can be used instead of only one before. Many people requested this. - Close race between regular and synchronization requests (I wasn't able to trigger it with one sync request, but with many parallel requests it's real). - Reimplement locking. I moved softc synchronization from the topology lock to per-device sx lock. I'd like to ask gmirror/graid3 users to test those patches as much as possible, because I want to commit them to the HEAD and RELENG_6 branches. Because of locking changes it will be really good if you can turn on INVARIANTS, INVARIANT_SUPPORT options and eventually DIAGNOSTIC in your kernel. Thanks in advance! -- Pawel Jakub Dawidek wheel.pl pjd@FreeBSD.org FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 187 bytes Desc: not available Url : lists.freebsd.org/pipermail/freebsd-stable/attachments/20060306/a14d6309/attachment.bin
On Mon, Mar 06, 2006 at 11:28:44PM +0100, Pawel Jakub Dawidek wrote:> Hi. > > Here you can find patches with changes to gmirror(8) and graid3(8): > > people.freebsd.org/~pjd/patches/gmirror.7.patch > people.freebsd.org/~pjd/patches/graid3.patchHi Pawel, I've been experiencing lockups with gmirror, ATA/SATA on both i386 and amd64, under severe I/O (very heavily loaded Postgres DB). This has been on several different machines (remotely located, with no possibility of breaking into the debugger). Do you think these patches are worth testing in my case ? Phil
On Mon, Mar 06, 2006 at 04:45:06PM -0600, Larry Rosenman wrote: +> Pawel Jakub Dawidek wrote: +> > Hi. +> > +> > Here you can find patches with changes to gmirror(8) and graid3(8): +> > +> > people.freebsd.org/~pjd/patches/gmirror.7.patch +> > people.freebsd.org/~pjd/patches/graid3.patch +> > +> > The patches does the following: +> > - Significant synchronization speed improvement. Now many parallel +> > synchronization I/O requests can be used instead of only one before. +> > Many people requested this. +> > - Close race between regular and synchronization requests (I wasn't +> > able to trigger it with one sync request, but with many parallel +> > requests it's real). +> > - Reimplement locking. I moved softc synchronization from the topology +> > lock to per-device sx lock. +> > +> > I'd like to ask gmirror/graid3 users to test those patches as much as +> > possible, because I want to commit them to the HEAD and RELENG_6 +> > branches. +> > +> > Because of locking changes it will be really good if you can turn on +> > INVARIANTS, INVARIANT_SUPPORT options and eventually DIAGNOSTIC in +> > your kernel. +> > +> > Thanks in advance! +> +> is there any chance at all of making a way to do a kernel dump onto a +> gmirror'd device? +> +> or at least exposing the 'b' slices of the disks so to allow a dump to them? I'm CCing the list you removed, because I'm seeing this question not the first time. In theory it is possible, but in practise its hard. Here are the problems: - Kernel is dumped without GEOM knowledge, so when gmirror is configured on top of da0 and da1, let's say it will decide to dump on da0. After reboot savecore will try to read the dump from a mirror provider. If we have round-robin balance algorithm it will get trash, because there is a kernel dump on da0, but not on da1. So basically we should setup 'perfer' balance algorithm to always read from one disk only. - 'prefer' balance algorithm reads from the component with the higest priority if it is in an active state (is UP and it's not beeing synchronized). When it is synchronized which component should I choose on dumpon(8) call? If I choose the first active component, synchronization can be finished before kernel is dumped. If I choose component with the higest priority even if it is synchronized, kernel can be dumped before synchronization is finished, so on boot 'prefer' balance algorithm can read from the wrong (active) component. To handle this I'd need to remember if kerneldump was requested and update component to dump when synchronization will finish or when component will be disconnected, etc. Such automatic control will break possibility to change dump device with dumpon(8) command once it was configured to dump on gmirror's provider: 1. dumpon /dev/mirror/foo 2. dumpon /dev/da2s1b 3. da0 is disconnected from mirror/foo, so gmirror changes automatically to dump on da1. -- Pawel Jakub Dawidek wheel.pl pjd@FreeBSD.org FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 187 bytes Desc: not available Url : lists.freebsd.org/pipermail/freebsd-stable/attachments/20060307/8e3e2e72/attachment.bin
At 05:28 PM 06/03/2006, Pawel Jakub Dawidek wrote:>Hi. > >Here you can find patches with changes to gmirror(8) and graid3(8): > > people.freebsd.org/~pjd/patches/gmirror.7.patch > people.freebsd.org/~pjd/patches/graid3.patchHi, Against stable, this patch fails in places. This is against RELENG_6 as of today patch < graid3.patch Hmm... Looks like a unified diff to me... The text leading up to this was: -------------------------- |Index: g_raid3.c |==================================================================|RCS file: /usr/repo/src/sys/geom/raid3/g_raid3.c,v |retrieving revision 1.54 |diff -u -p -r1.54 g_raid3.c |--- g_raid3.c 22 Feb 2006 10:21:05 -0000 1.54 |+++ g_raid3.c 7 Mar 2006 13:25:14 -0000 -------------------------- Patching file g_raid3.c using Plan A... Hunk #1 failed at 62. Hunk #2 succeeded at 85 (offset -5 lines). Hunk #3 succeeded at 92 (offset -5 lines). Hunk #4 succeeded at 117 (offset -5 lines). Hunk #5 succeeded at 170 (offset -5 lines). Hunk #6 succeeded at 278 (offset -5 lines). Hunk #7 succeeded at 318 (offset -5 lines). Hunk #8 succeeded at 348 (offset -5 lines). Hunk #9 succeeded at 399 (offset -5 lines). Hunk #10 succeeded at 453 (offset -5 lines). Hunk #11 succeeded at 520 (offset -5 lines). Hunk #12 succeeded at 532 (offset -5 lines). Hunk #13 succeeded at 547 (offset -5 lines). Hunk #14 succeeded at 569 (offset -5 lines). Hunk #15 failed at 599. Hunk #16 succeeded at 640 (offset -5 lines). Hunk #17 succeeded at 655 with fuzz 1 (offset -5 lines). Hunk #18 succeeded at 675 (offset -14 lines). Hunk #19 succeeded at 739 (offset -5 lines). Hunk #20 succeeded at 780 (offset 10 lines). Hunk #21 failed at 805. Hunk #22 failed at 825. Hunk #23 failed at 855. Hunk #24 failed at 864. Hunk #25 succeeded at 825 with fuzz 1 (offset -43 lines). Hunk #26 succeeded at 980 (offset 36 lines). Hunk #27 succeeded at 928 (offset -43 lines). Hunk #28 succeeded at 1096 (offset 36 lines). Hunk #29 succeeded at 1116 (offset -50 lines). Hunk #30 succeeded at 1331 (offset 14 lines). Hunk #31 succeeded at 1325 (offset -50 lines). Hunk #32 succeeded at 1530 (offset 14 lines). Hunk #33 succeeded at 1521 (offset -50 lines). Hunk #34 succeeded at 1603 (offset 14 lines). Hunk #35 succeeded at 1556 (offset -50 lines). Hunk #36 succeeded at 1738 (offset 14 lines). Hunk #37 succeeded at 1687 with fuzz 1 (offset -52 lines). Hunk #38 succeeded at 1850 (offset 14 lines). Hunk #39 succeeded at 1794 (offset -52 lines). Hunk #40 failed at 1840. Hunk #41 succeeded at 1925 (offset 14 lines). Hunk #42 failed at 1945. Hunk #43 succeeded at 1887 (offset -53 lines). Hunk #44 succeeded at 1962 (offset 14 lines). Hunk #45 succeeded at 1913 (offset -53 lines). Hunk #46 succeeded at 1994 (offset 14 lines). Hunk #47 failed at 2007. Hunk #48 succeeded at 1945 (offset -58 lines). Hunk #49 failed at 1955. Hunk #50 failed at 1992. Hunk #51 succeeded at 2112 (offset 45 lines). Hunk #52 succeeded at 2032 (offset -58 lines). Hunk #53 succeeded at 2208 (offset 45 lines). Hunk #54 succeeded at 2124 (offset -58 lines). Hunk #55 succeeded at 2249 (offset 45 lines). Hunk #56 succeeded at 2166 (offset -58 lines). Hunk #57 succeeded at 2285 (offset 45 lines). Hunk #58 succeeded at 2273 (offset -58 lines). Hunk #59 succeeded at 2604 (offset 45 lines). Hunk #60 succeeded at 2813 (offset -56 lines). Hunk #61 failed at 2844. Hunk #62 failed at 2854. Hunk #63 succeeded at 3039 (offset 62 lines). Hunk #64 failed at 3059. Hunk #65 succeeded at 2997 (offset -56 lines). Hunk #66 succeeded at 3135 (offset 62 lines). Hunk #67 succeeded at 3027 (offset -56 lines). Hunk #68 succeeded at 3222 (offset 62 lines). Hunk #69 succeeded at 3113 (offset -56 lines). Hunk #70 succeeded at 3245 (offset 62 lines). Hunk #71 succeeded at 3162 (offset -56 lines). Hunk #72 succeeded at 3292 (offset 62 lines). Hunk #73 succeeded at 3211 (offset -57 lines). Hunk #74 succeeded at 3364 (offset 62 lines). Hunk #75 succeeded at 3315 (offset -57 lines). Hunk #76 succeeded at 3446 (offset 62 lines). 14 out of 76 hunks failed--saving rejects to g_raid3.c.rej Hmm... The next patch looks like a unified diff to me... The text leading up to this was: -------------------------- |Index: g_raid3.h |==================================================================|RCS file: /usr/repo/src/sys/geom/raid3/g_raid3.h,v |retrieving revision 1.15 |diff -u -p -r1.15 g_raid3.h |--- g_raid3.h 11 Feb 2006 17:42:31 -0000 1.15 |+++ g_raid3.h 7 Mar 2006 13:25:16 -0000 -------------------------- Patching file g_raid3.h using Plan A... Hunk #1 succeeded at 109 (offset -1 lines). Hunk #2 succeeded at 169 (offset -1 lines). Hunk #3 succeeded at 197 (offset -1 lines). Hmm... The next patch looks like a unified diff to me... The text leading up to this was: -------------------------- |Index: g_raid3_ctl.c |==================================================================|RCS file: /usr/repo/src/sys/geom/raid3/g_raid3_ctl.c,v |retrieving revision 1.12 |diff -u -p -r1.12 g_raid3_ctl.c |--- g_raid3_ctl.c 1 Feb 2006 12:06:01 -0000 1.12 |+++ g_raid3_ctl.c 6 Mar 2006 19:50:13 -0000 -------------------------- Patching file g_raid3_ctl.c using Plan A... Hunk #1 succeeded at 51. Hunk #2 succeeded at 60. Hunk #3 succeeded at 75. Hunk #4 succeeded at 112. Hunk #5 succeeded at 163. Hunk #6 succeeded at 227. Hunk #7 succeeded at 240. Hunk #8 succeeded at 262. Hunk #9 succeeded at 286. Hunk #10 succeeded at 311. Hunk #11 succeeded at 342. Hunk #12 succeeded at 372. Hunk #13 succeeded at 381. Hunk #14 succeeded at 396. Hunk #15 succeeded at 459. Hunk #16 succeeded at 469. Hunk #17 succeeded at 494. Hunk #18 succeeded at 503. Hunk #19 succeeded at 518. Hunk #20 succeeded at 534. Hunk #21 succeeded at 545. Hunk #22 succeeded at 574. Hunk #23 succeeded at 587. done
On Wed, Mar 08, 2006 at 01:50:42PM -0500, Mike Tancsa wrote: +> At 05:28 PM 06/03/2006, Pawel Jakub Dawidek wrote: +> >Hi. +> > +> >Here you can find patches with changes to gmirror(8) and graid3(8): +> > +> > people.freebsd.org/~pjd/patches/gmirror.7.patch +> > people.freebsd.org/~pjd/patches/graid3.patch +> +> Hi, +> Against stable, this patch fails in places. This is against RELENG_6 as of today graid3 from HEAD and RELENG_6 differ. Can you apply the patches to HEAD version and just copy entire sys/geom/raid3 from HEAD to RELENG_6? -- Pawel Jakub Dawidek wheel.pl pjd@FreeBSD.org FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 187 bytes Desc: not available Url : lists.freebsd.org/pipermail/freebsd-stable/attachments/20060308/a0149106/attachment.bin
At 02:39 PM 08/03/2006, Pawel Jakub Dawidek wrote:>On Wed, Mar 08, 2006 at 01:50:42PM -0500, Mike Tancsa wrote: >+> At 05:28 PM 06/03/2006, Pawel Jakub Dawidek wrote: >+> >Hi. >+> > >+> >Here you can find patches with changes to gmirror(8) and graid3(8): >+> > >+> > people.freebsd.org/~pjd/patches/gmirror.7.patch >+> > people.freebsd.org/~pjd/patches/graid3.patch >+> >+> Hi, >+> Against stable, this patch fails in places. This is against >RELENG_6 as of today > >graid3 from HEAD and RELENG_6 differ. Can you apply the patches to HEAD >version and just copy entire sys/geom/raid3 from HEAD to RELENG_6?Yes, done. I had saved the test results from the last set of tests and added the new results. tancsa.com/raid3.html There is both a speed improvement and regression in reading depending on the test size. I repeated the 2G and 3G test twice and there was very little change between tests, but it doesnt seem to make sense why 4G would be faster than 2G. Also, this test if you recall, used to lock up the box with preemption in the kernel. The version in HEAD, with the above patch works just fine in RELENG_6 using these simple tests and does not lockup. ---Mike
Pawel Jakub Dawidek wrote:> Hi. > > Here you can find patches with changes to gmirror(8) and graid3(8): > > people.freebsd.org/~pjd/patches/gmirror.7.patch > people.freebsd.org/~pjd/patches/graid3.patch >Will these patches make it in to 5.5 ?