Hello the idea introducd by Chris Greer to use servers as solid state disks kept my brain busy the last days. Perhaps it makes sense to put L2ARC devices in memory as well to increase the in-memory-part of a database in excess of the capacity of a single server. I already wrote about the idea in my blog at http://www.c0t0d0s0.org/archives/4919-L2ARC-on-ramdisks.html so i will just copy-and-paste the article here: "I thought a little bit about the idea of transforming server into solid state disks. The idea in the mail of Chris Greer on zfs-discuss was to use mirrored iSCSI shared ramdisks as a storage for the seperated ZILs. But i think you could use the concept as well for L2ARC as well - e.g. for large databases. One of the sizing rules of databases: More main memory never hurts. Nothing helps the performance of a database more than even more memory. The rule of "main memory never hurts" is based on the fact, that a hard disk has only a few IOPS compared with the main memory and hard drive access massively hurts the performance of your database. But obviously the size of memory is limited, albeit the this limit is quite high with systems with memory sizes in the range of 512 GB on 4 rack units. But how can you get more memory into your database system, when all DIMM slots are filled with the biggest available DIMMS. I had an idea while cooking tea this evening while i thought about a discussion with a colleague: Let?s assume an architecture based on a X4600 as a head in front of four X4600 each fully maxed to 512GB. All the nodes are connected with Infiniband. The first X4600 is your normal database server (for example mysql or LarryBase). You put your data into an ZFS storage pool. This storage pool is augumented with L2ARC devices. But now comes the plot twist. Let?s use the 512GB X4600 as huge ramdisks (yes, i know, every engineers heart will crying now) speaking via iSER (no TCP/IP, just RDMA) at 20 GBit/s to the central database node. This would give you a cache in the size of almost 2 TB plus the cache on the database server itself.. By using L2ARC you could use the memory as database caches of other systems without using a database doing a combination of the memory resources by other means, for example the CacheFusion stuff of Oracle. You don?t have to fuse the caches of other databases servers. The other servers are caches. You don?t have to partition the databases. It would be interesting how such an system would perform in comparision to a Oracle RAC or other memory implementations. Anybody out there willing to test this ... my Infiniband switches are in the laundry at the moment ;-)" I hope this idea is not complete nonsense ... Regards Joerg -- Joerg Moellenkamp Tel: (+49 40) 25 15 23 - 460 Senior Systems Engineer Fax: (+49 40) 25 15 23 - 425 Sun Microsystems GmbH Mobile: (+49 172) 83 18 433 Nagelsweg 55 mailto:joerg.moellenkamp at sun.com D-20097 Hamburg http://www.sun.de Sitz der Gesellschaft: Sun Microsystems GmbH Sonnenallee 1 D-85551 Kirchheim-Heimstetten Amtsgericht M?nchen: HRB 161028 Gesch?ftsf?hrer: Thomas Schr?der Wolfgang Engels Dr. Roland B?mer Vorsitzender des Aufsichtsrates: Martin H?ring -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20081014/9cae44fb/attachment.html>