zfs discuss - Apr 2007 - SPARC: no cache synchronize

Hello zfs-discuss,

bash-3.00# uname -a
SunOS nfs-10-1.srv 5.10 Generic_125100-04 sun4u sparc SUNW,Sun-Fire-V440

zil_disable set to 1
Disks are over FCAL from 3510.

bash-3.00# dtrace -n
fbt::*SYNCHRONIZE*:entry''{printf("%Y",walltimestamp);}''
dtrace: description ''fbt::*SYNCHRONIZE*:entry'' matched 4
probes

[no output]

Why?


ps. of course there''re writes to the pool



-- 
Best regards,
 Robert                          mailto:rmilkowski at task.gda.pl
                                     http://milek.blogspot.com

rmilkowski at task.gda.pl said:
> bash-3.00# uname -a SunOS nfs-10-1.srv 5.10 Generic_125100-04 sun4u sparc
> SUNW,Sun-Fire-V440
> 
> zil_disable set to 1 Disks are over FCAL from 3510.
> 
> bash-3.00# dtrace -n
fbt::*SYNCHRONIZE*:entry''{printf("%Y",walltimestamp);}''
> dtrace: description ''fbt::*SYNCHRONIZE*:entry'' matched 4
probes
> 
> [no output]
> 
> Why? 
Robert,

I saw this happen in some of my tests.  It was quite odd in that one
pool would have no sync''s show up, and another pool on the same machine
(and on the same array) would show sync''s happening.

The explanation we came up with was that the code remembers if at some
point a sync-cache request failed (e.g. "invalid operation" or some
such
error), after which it will no longer issue sync requests (until the next
server reboot, of course).

We never did figure out why the array might''ve given an error on one
set
of LUN''s and not on another.  Perhaps a queue overrun or some other
condition happened during earlier tests.

Nowadays we have the array configured to ignore the sync requests, and
indeed we never see them in dtrace anymore (on any pool/LUN).  Presumably
the array responds "invalid operation" for every such request now.

Regards,

Marion

Hello Robert,

Monday, April 23, 2007, 1:20:28 AM, you wrote:

RM> Hello zfs-discuss,

RM> bash-3.00# uname -a
RM> SunOS nfs-10-1.srv 5.10 Generic_125100-04 sun4u sparc SUNW,Sun-Fire-V440

RM> zil_disable set to 1
RM> Disks are over FCAL from 3510.

RM> bash-3.00# dtrace -n
RM>
fbt::*SYNCHRONIZE*:entry''{printf("%Y",walltimestamp);}''
RM> dtrace: description ''fbt::*SYNCHRONIZE*:entry'' matched
4 probes

RM> [no output]

RM> Why?


RM> ps. of course there''re writes to the pool

Thanks to Roch for suggesting uscsi_strategy.



bash-3.00# dtrace -n
fbt::*sd_uscsi_strategy:entry,fbt::*SYNCHRONIZE*:entry''{printf("%Y",walltimestamp);}''
dtrace: description
''fbt::*sd_uscsi_strategy:entry,fbt::*SYNCHRONIZE*:entry''
matched 6 probes
CPU     ID                    FUNCTION:NAME
  2  83446         ssd_uscsi_strategy:entry 2007 Apr 23 16:07:39
  1  83446         ssd_uscsi_strategy:entry 2007 Apr 23 16:10:26
  0  43849          sd_uscsi_strategy:entry 2007 Apr 23 16:10:55
  2  43849          sd_uscsi_strategy:entry 2007 Apr 23 16:11:46
  3  83446         ssd_uscsi_strategy:entry 2007 Apr 23 16:12:24
  3  43849          sd_uscsi_strategy:entry 2007 Apr 23 16:12:54
  0  43849          sd_uscsi_strategy:entry 2007 Apr 23 16:13:30
  2  83446         ssd_uscsi_strategy:entry 2007 Apr 23 16:14:07
  0  83446         ssd_uscsi_strategy:entry 2007 Apr 23 16:14:16
  0  83446         ssd_uscsi_strategy:entry 2007 Apr 23 16:14:19
^C

I should be getting those cache flushes every 5s - right?
The question is why I observer different behavior?
There''re definitely both reads and writes all the time.
And on that hosts we''re observing performance problems so maybe
it''s
somehow related?

Also sync() can take several minutes to complete...

Do you have any idea?



bash-3.00#
bash-3.00# zpool iostat 1
               capacity     operations    bandwidth
pool         used  avail   read  write   read  write
----------  -----  -----  -----  -----  -----  -----
f3-1         494G   730G  1.22K    405  5.04M  5.07M
misc        20.6M  52.0G      0      0     21      0
----------  -----  -----  -----  -----  -----  -----
f3-1         494G   730G  2.14K     11  8.05M  1.49M
misc        20.6M  52.0G      0      0      0      0
----------  -----  -----  -----  -----  -----  -----
f3-1         494G   730G  2.03K      6  11.9M   888K
misc        20.6M  52.0G      0      0      0      0
----------  -----  -----  -----  -----  -----  -----
f3-1         494G   730G  2.19K     10  11.6M  1.36M
misc        20.6M  52.0G      0      0      0      0
----------  -----  -----  -----  -----  -----  -----
f3-1         494G   730G  2.03K      1  7.47M   253K
misc        20.6M  52.0G      0      0      0      0
----------  -----  -----  -----  -----  -----  -----
f3-1         494G   730G  1.89K     13  11.8M  1.73M
misc        20.6M  52.0G      0      0      0      0
----------  -----  -----  -----  -----  -----  -----
f3-1         494G   730G  2.30K      8  22.6M  1.11M
misc        20.6M  52.0G      0      0      0      0
----------  -----  -----  -----  -----  -----  -----
^C
bash-3.00#



-- 
Best regards,
 Robert                            mailto:rmilkowski at task.gda.pl
                                       http://milek.blogspot.com