zfs discuss - Feb 2009 - Adding a SDcard as zfs cache (L2ARC?) on a laptop?

I''ve been searching without success if this has been done or even
discussed previously.

Would it be possible now or in the future to use an SD Card on a laptop as a
cache for ZFS?

I''ve been trying to do this with OpenSolaris 1108 on a Dell XPS M1530
laptop. The SD Card is presented as /dev/rdsk/c8t0d0p0 under rmformat.

When using ''pfexec zpool add rpool cache c8t0d0'' I get:

cannot label ''c8t0d0'': EFI labeled devices are not supported
on root pools.

Cheers, JP
-- 
This message posted from opensolaris.org

On Wed, Feb 4, 2009 at 9:31 AM, Jean-Paul Rivet <jrivet at bigpond.net.au>
wrote:
> I''ve been searching without success if this has been done or even
discussed previously.
>
> Would it be possible now or in the future to use an SD Card on a laptop as
a cache for ZFS?
> When using ''pfexec zpool add rpool cache c8t0d0'' I get:
>
> cannot label ''c8t0d0'': EFI labeled devices are not
supported on root pools.
>From that error message I believe the problem is not on "laptop"
or
"SD Card". It''s because you''re trying to add cache
to root pool.

Try adding it to another pool. It should work.

PS: is your SD card REALLY faster than your HD? If yes, what brand/model?

Regards,

Fajar

> >From that error message I believe the problem is not
> on "laptop" or
> "SD Card". It''s because you''re trying to add
cache to
> root pool.
> 
> Try adding it to another pool. It should work.
Thanks for the response. Being a laptop I don''t have other pools to
choose from - just the rpool:)

So I guess the rule is this will work with other pools as long as its *not* the
root pool?
 
> PS: is your SD card REALLY faster than your HD? If
> yes, what brand/model?
No idea. I have 4GB SanDisk Extreme III lying around and thought I''d
throw it in and see what happens. This HD is a 5400rpm model, not a 7200rpm one.

Originally I was thinking Intel Turbo Memory but believe this managed by the
BIOS and would not be exposed to Solaris as a device. If anyone can confirm or
deny this also I''d appreciate it.

Cheers, JP
-- 
This message posted from opensolaris.org

Jean-Paul Rivet wrote:
> I''ve been searching without success if this has been done or even
discussed previously.
>
> Would it be possible now or in the future to use an SD Card on a laptop as
a cache for ZFS?
>
> I''ve been trying to do this with OpenSolaris 1108 on a Dell XPS
M1530 laptop. The SD Card is presented as /dev/rdsk/c8t0d0p0 under rmformat.
>
> When using ''pfexec zpool add rpool cache c8t0d0'' I get:
>
> cannot label ''c8t0d0'': EFI labeled devices are not
supported on root pools.
>   
It looks like it''s doing a somewhat misguided check which is
appropriate
for a device in the root pool which the BIOS needs to be able to boot 
from, but probably not for the cache device.

Anyway, you could try simply creating standard FDISK/Solaris/vtoc 
partitioning on the SD card, with all the free space contained in one 
slice, and give that slice to ZFS.

-- 
Andrew

> Anyway, you could try simply creating standard
> FDISK/Solaris/vtoc 
> partitioning on the SD card, with all the free space
> contained in one 
> slice, and give that slice to ZFS.
This is what I''ve done so far.

fdisk - 

 Total disk size is 1943 cylinders
             Cylinder size is 4096 (512 byte) blocks

                                               Cylinders
      Partition   Status    Type          Start   End   Length    %
      =========   ======    ============  =====   ===   ======   ==          1  
Solaris2          1  1942    1942    100

partition> p
Volume:  cache
Current partition table (original):
Total disk cylinders available: 1940 + 2 (reserved cylinders)

Part      Tag    Flag     Cylinders        Size            Blocks
  0 unassigned    wm       0               0         (0/0/0)          0
  1 unassigned    wm       0               0         (0/0/0)          0
  2 unassigned    wu       1 - 1939        3.79GB    (1939/0/0) 7942144
  3 unassigned    wm       0               0         (0/0/0)          0
  4 unassigned    wm       0               0         (0/0/0)          0
  5 unassigned    wm       0               0         (0/0/0)          0
  6 unassigned    wm       0               0         (0/0/0)          0
  7 unassigned    wm       0               0         (0/0/0)          0
  8       boot    wu       0 -    0        2.00MB    (1/0/0)       4096
  9 unassigned    wm       0               0         (0/0/0)          0

partition> 

I then tried numerous variations of the command and kept getting errors like
this one:

$ pfexec zpool add rpool cache /dev/dsk/c8t0d0s2
cannot add to ''rpool'': invalid argument for this pool
operation
$

Eventually I tried this one:

$ pfexec zpool add rpool cache /dev/rdsk/c8t0d0

I think its working but aren''t sure because it still hasn''t
finished after 1hr15. The HD seems to be getting hit fairly hard and iostat
shows:

zpool iostat -v rpool
               capacity     operations    bandwidth
pool         used  avail   read  write   read  write
----------  -----  -----  -----  -----  -----  -----
rpool       49.1G  14.4G     11     56   747K  3.88M
  c4t0d0s0  49.1G  14.4G     11     56   747K  3.88M
----------  -----  -----  -----  -----  -----  -----

I''ll let it run through the night and see what happens.

Cheers, JP
-- 
This message posted from opensolaris.org

Jean-Paul,

Our goofy disk formatting is tripping you...

Put the disk space of c8t0d0 in c8t0d0s0 and try the
zpool add syntax again. If you need help with the
format syntax, let me know.

This command syntax should have complained:

pfexec zpool add rpool cache /dev/rdsk/c8t0d0

See the zpool syntax below for pointers.

Cindy

# zpool add rpool cache c0t1d0s0
# zpool iostat -v rpool
                capacity     operations    bandwidth
pool         used  avail   read  write   read  write
----------  -----  -----  -----  -----  -----  -----
rpool       11.4G  22.4G      0      0  5.86K     73
   c0t0d0s0  11.4G  22.4G      0      0  5.86K     73
cache           -      -      -      -      -      -
   c0t1d0s0  47.8M  33.9G      0     53  18.1K  6.38M
----------  -----  -----  -----  -----  -----  -----


Jean-Paul Rivet wrote:
>>Anyway, you could try simply creating standard
>>FDISK/Solaris/vtoc 
>>partitioning on the SD card, with all the free space
>>contained in one 
>>slice, and give that slice to ZFS.
> 
> 
> This is what I''ve done so far.
> 
> fdisk - 
> 
>  Total disk size is 1943 cylinders
>              Cylinder size is 4096 (512 byte) blocks
> 
>                                                Cylinders
>       Partition   Status    Type          Start   End   Length    %
>       =========   ======    ============  =====   ===   ======   ==>    
1                 Solaris2          1  1942    1942    100
> 
> partition> p
> Volume:  cache
> Current partition table (original):
> Total disk cylinders available: 1940 + 2 (reserved cylinders)
> 
> Part      Tag    Flag     Cylinders        Size            Blocks
>   0 unassigned    wm       0               0         (0/0/0)          0
>   1 unassigned    wm       0               0         (0/0/0)          0
>   2 unassigned    wu       1 - 1939        3.79GB    (1939/0/0) 7942144
>   3 unassigned    wm       0               0         (0/0/0)          0
>   4 unassigned    wm       0               0         (0/0/0)          0
>   5 unassigned    wm       0               0         (0/0/0)          0
>   6 unassigned    wm       0               0         (0/0/0)          0
>   7 unassigned    wm       0               0         (0/0/0)          0
>   8       boot    wu       0 -    0        2.00MB    (1/0/0)       4096
>   9 unassigned    wm       0               0         (0/0/0)          0
> 
> partition> 
> 
> I then tried numerous variations of the command and kept getting errors
like this one:
> 
> $ pfexec zpool add rpool cache /dev/dsk/c8t0d0s2
> cannot add to ''rpool'': invalid argument for this pool
operation
> $
> 
> Eventually I tried this one:
> 
> $ pfexec zpool add rpool cache /dev/rdsk/c8t0d0
> 
> I think its working but aren''t sure because it still
hasn''t finished after 1hr15. The HD seems to be getting hit fairly hard
and iostat shows:
> 
> zpool iostat -v rpool
>                capacity     operations    bandwidth
> pool         used  avail   read  write   read  write
> ----------  -----  -----  -----  -----  -----  -----
> rpool       49.1G  14.4G     11     56   747K  3.88M
>   c4t0d0s0  49.1G  14.4G     11     56   747K  3.88M
> ----------  -----  -----  -----  -----  -----  -----
> 
> I''ll let it run through the night and see what happens.
> 
> Cheers, JP

> Put the disk space of c8t0d0 in c8t0d0s0 and try the
> zpool add syntax again. If you need help with the
> format syntax, let me know.
> 
> This command syntax should have complained:
> 
> pfexec zpool add rpool cache /dev/rdsk/c8t0d0
> 
> See the zpool syntax below for pointers.
> 
> Cindy
I''ve tried the following and get:

~$ pfexec zpool add rpool cache c8t0d0s0
cannot open ''/dev/dsk/c8t0d0s0'': No such device or address
~$ 

Expected because s0 is defined as 0 bytes in the partition table I presume?

~$ pfexec zpool add rpool cache c8t0d0s2
cannot add to ''rpool'': invalid argument for this pool
operation
~$ 

Should this have worked?

Have I partitioned the SD card correctly and am I trying to allocate the correct
slice to zfs?

Thanks for your responses far.
 
partition> p
Volume:  cache
Current partition table (original):
Total disk cylinders available: 1940 + 2 (reserved cylinders)

Part      Tag    Flag     Cylinders        Size            Blocks
  0 unassigned    wm       0               0         (0/0/0)          0
  1 unassigned    wm       0               0         (0/0/0)          0
  2 unassigned    wu       1 - 1939        3.79GB    (1939/0/0) 7942144
  3 unassigned    wm       0               0         (0/0/0)          0
  4 unassigned    wm       0               0         (0/0/0)          0
  5 unassigned    wm       0               0         (0/0/0)          0
  6 unassigned    wm       0               0         (0/0/0)          0
  7 unassigned    wm       0               0         (0/0/0)          0
  8       boot    wu       0 -    0        2.00MB    (1/0/0)       4096
  9 unassigned    wm       0               0         (0/0/0)          0

partition> 

Cheers, JP
> # zpool add rpool cache c0t1d0s0
> # zpool iostat -v rpool
>                 capacity     operations    bandwidth
> d  avail   read  write   read  write
> ----------  -----  -----  -----  -----  -----  -----
> rpool       11.4G  22.4G      0      0  5.86K     73
>    c0t0d0s0  11.4G  22.4G      0      0  5.86K     73
> he           -      -      -      -      -      -
>    c0t1d0s0  47.8M  33.9G      0     53  18.1K  6.38M
> -------  -----  -----  -----  -----  -----  -----
>
-- 
This message posted from opensolaris.org

Jean-Paul,

Regarding your comments here:

Expected because s0 is defined as 0 bytes in the partition table I presume?

Yes, you need to put the disk space into s0 by using the format
utility. Use the modify option from format''s partition menu is
probably the easiest way. Email me directly if you need the
steps.

 >
 > ~$ pfexec zpool add rpool cache c8t0d0s2
 > cannot add to ''rpool'': invalid argument for this pool
operation

 > ~$
 >
 > Should this have worked?

No, because s2 has a special meaning with a VTOC (SMI) label.

After you fix the disk partitions, everything should just work. :-)

Cindy


Jean-Paul Rivet wrote:
>>Put the disk space of c8t0d0 in c8t0d0s0 and try the
>>zpool add syntax again. If you need help with the
>>format syntax, let me know.
>>
>>This command syntax should have complained:
>>
>>pfexec zpool add rpool cache /dev/rdsk/c8t0d0
>>
>>See the zpool syntax below for pointers.
>>
>>Cindy
> 
> 
> I''ve tried the following and get:
> 
> ~$ pfexec zpool add rpool cache c8t0d0s0
> cannot open ''/dev/dsk/c8t0d0s0'': No such device or
address
> ~$ 
> 
> Expected because s0 is defined as 0 bytes in the partition table I presume?
> 
> ~$ pfexec zpool add rpool cache c8t0d0s2
> cannot add to ''rpool'': invalid argument for this pool
operation
> ~$ 
> 
> Should this have worked?
> 
> Have I partitioned the SD card correctly and am I trying to allocate the
correct slice to zfs?
> 
> Thanks for your responses far.
>  
> partition> p
> Volume:  cache
> Current partition table (original):
> Total disk cylinders available: 1940 + 2 (reserved cylinders)
> 
> Part      Tag    Flag     Cylinders        Size            Blocks
>   0 unassigned    wm       0               0         (0/0/0)          0
>   1 unassigned    wm       0               0         (0/0/0)          0
>   2 unassigned    wu       1 - 1939        3.79GB    (1939/0/0) 7942144
>   3 unassigned    wm       0               0         (0/0/0)          0
>   4 unassigned    wm       0               0         (0/0/0)          0
>   5 unassigned    wm       0               0         (0/0/0)          0
>   6 unassigned    wm       0               0         (0/0/0)          0
>   7 unassigned    wm       0               0         (0/0/0)          0
>   8       boot    wu       0 -    0        2.00MB    (1/0/0)       4096
>   9 unassigned    wm       0               0         (0/0/0)          0
> 
> partition> 
> 
> Cheers, JP
> 
> 
>># zpool add rpool cache c0t1d0s0
>># zpool iostat -v rpool
>>                capacity     operations    bandwidth
>>d  avail   read  write   read  write
>>----------  -----  -----  -----  -----  -----  -----
>>rpool       11.4G  22.4G      0      0  5.86K     73
>>   c0t0d0s0  11.4G  22.4G      0      0  5.86K     73
>>he           -      -      -      -      -      -
>>   c0t1d0s0  47.8M  33.9G      0     53  18.1K  6.38M
>>-------  -----  -----  -----  -----  -----  -----
>>

Did anyone ever have success with this?

I''m trying to add a usb flash device as rpool cache, and am hitting the
same problem, even after working through the SMI/EFI label and other issues
above.

root at asura:~# zpool add rpool cache /dev/dsk/c6t0d0s0
invalid vdev specification
use ''-f'' to override the following errors:
/dev/dsk/c6t0d0s0 overlaps with /dev/dsk/c6t0d0s2
root at asura:~# zpool add -f rpool cache /dev/dsk/c6t0d0s0
cannot add to ''rpool'': invalid argument for this pool
operation
-- 
This message posted from opensolaris.org

> Did anyone ever have success with this?
> I''m trying to add a usb flash device as rpool cache, and am
hitting the same problem,
> even after working through the SMI/EFI label and other issues above.
I played with adding a USB stick as L2ARC a few versions ago of SXCE, pre 104.

At the time, I got the same error message, but if I booted to
"failsafe" it would allow me to add the USB device as rpool cache. 
Unfortunately, on normal boot up, the device name had changed on the USB device
so ZFS complained.  I don''t recall how I got around that problem.

I found that the USB implementation on Solaris was weak enough that the USB
drive, which consistently read 20 MB/s or so under Linux was hovering around 5
MB/s in Solaris.  Only under extremely random reads did the flash drive help. 
In all other cases, it actually slowed down the system.  I assume that a better
USB implementation or a different storage interface would have made it a much
better experience.

Good luck,
Marty
-- 
This message posted from opensolaris.org