Hi, Trying to make the most of a limited budget. I need fast I/O for operations under 4MB, and high availability of VMs in an Ovirt cluster. I Have 3 nodes running Ovirt and want to rebuild them with hardware for converging storage. Should I use 2 960GB SSDs in RAID1 in each node, replica 3? Or can I get away with 1 larger SSD per node, JBOD, replica 3? Is a flash-backed Raid required for JBOD, and should it be 1gb, 2, or 4gb flash? Storage network will be 10gbe. Enterprise SSDs and Flash-backed raid is very expensive, so I want to ensure the investment will provide best value in terms of capacity, performance, and availability. Thanks, Vincent -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20180404/b34608c5/attachment.html>
Based on your message, it sounds like your total usable capacity requirement is around <1TB. With a modern SSD, you'll get something like 40k theoretical IOPs for 4k I/O size. You don't mention budget. What is your budget? You mention "4MB operations", where is that requirement coming from? On Wed, Apr 4, 2018 at 12:41 PM, Vincent Royer <vincent at epicenergy.ca> wrote:> Hi, > > Trying to make the most of a limited budget. I need fast I/O for > operations under 4MB, and high availability of VMs in an Ovirt cluster. > > I Have 3 nodes running Ovirt and want to rebuild them with hardware for > converging storage. > > Should I use 2 960GB SSDs in RAID1 in each node, replica 3? > > Or can I get away with 1 larger SSD per node, JBOD, replica 3? > > Is a flash-backed Raid required for JBOD, and should it be 1gb, 2, or 4gb > flash? > > Storage network will be 10gbe. > > Enterprise SSDs and Flash-backed raid is very expensive, so I want to > ensure the investment will provide best value in terms of capacity, > performance, and availability. > > Thanks, > > Vincent > > _______________________________________________ > Gluster-users mailing list > Gluster-users at gluster.org > http://lists.gluster.org/mailman/listinfo/gluster-users >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20180404/b43a01c1/attachment.html>
thanks for your reply, Yes the VMs are very small and provide only a single service, I would prefer a total of 2TB but 1TB to start is sufficient. Ideally I want a scheme that is easy to expand by dropping an extra disk in each node. When all slots are full, add another node. our current setup accesses a storage share via NFS, most read/write operations under load are under 4MB. There isn't any long sequential I/O. Currently we have 2 nodes, I am specing the 3rd and adding necessary components to the existing. Budget around $20k for the upgrade. On Wed, Apr 4, 2018 at 12:49 PM, Alex Chekholko <alex at calicolabs.com> wrote:> Based on your message, it sounds like your total usable capacity > requirement is around <1TB. With a modern SSD, you'll get something like > 40k theoretical IOPs for 4k I/O size. > > You don't mention budget. What is your budget? You mention "4MB > operations", where is that requirement coming from? > > On Wed, Apr 4, 2018 at 12:41 PM, Vincent Royer <vincent at epicenergy.ca> > wrote: > >> Hi, >> >> Trying to make the most of a limited budget. I need fast I/O for >> operations under 4MB, and high availability of VMs in an Ovirt cluster. >> >> I Have 3 nodes running Ovirt and want to rebuild them with hardware for >> converging storage. >> >> Should I use 2 960GB SSDs in RAID1 in each node, replica 3? >> >> Or can I get away with 1 larger SSD per node, JBOD, replica 3? >> >> Is a flash-backed Raid required for JBOD, and should it be 1gb, 2, or 4gb >> flash? >> >> Storage network will be 10gbe. >> >> Enterprise SSDs and Flash-backed raid is very expensive, so I want to >> ensure the investment will provide best value in terms of capacity, >> performance, and availability. >> >> Thanks, >> >> Vincent >> >> _______________________________________________ >> Gluster-users mailing list >> Gluster-users at gluster.org >> http://lists.gluster.org/mailman/listinfo/gluster-users >> > >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20180404/85091eb6/attachment.html>
> > > Is a flash-backed Raid required for JBOD, and should it be 1gb, 2, or 4gb > flash? >Is anyone able to clarify this requirement for me? -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20180409/0b7677ce/attachment.html>
Your question is difficult to parse. Typically RAID and JBOD are mutually exclusive. By "flash-backed", do you mean a battery backup unit (BBU) on your RAID controller? On Mon, Apr 9, 2018 at 8:49 AM, Vincent Royer <vincent at epicenergy.ca> wrote:> >> Is a flash-backed Raid required for JBOD, and should it be 1gb, 2, or 4gb >> flash? >> > > Is anyone able to clarify this requirement for me? > > > _______________________________________________ > Gluster-users mailing list > Gluster-users at gluster.org > http://lists.gluster.org/mailman/listinfo/gluster-users >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20180409/4e236038/attachment.html>
On 09/04/18 16:49, Vincent Royer wrote:> > > Is a flash-backed Raid required for JBOD, and should it be 1gb, 2, > or 4gb flash? > >RAID and JBOD are completely different things. JBODs are just that, bunches of disks, and they don't have any cache above them in hardware. If you're going to use ZFS under Gluster, look at the ZFS docs first. Short answer is no. If Gluster passes sync writes down to the lower level FS as sync, and you decide to use a ZFS SLOG device, usually an SSD, it should have power fail protection capacitors. Do *not* use a RAID controller for ZFS, use a decent HBA instead, so ZFS can access the disks directly. Using a RAID controller is not only wasteful it's setting yourself up for a world of pain when a ZFS VDEV member device fails. Don't do it. -- This message is intended only for the addressee and may contain confidential information. Unless you are that person, you may not disclose its contents or use it in any way and are requested to delete the message along with any attachments and notify us immediately. This email is not intended to, nor should it be taken to, constitute advice. The information provided is correct to our knowledge & belief and must not be used as a substitute for obtaining tax, regulatory, investment, legal or any other appropriate advice. "Transact" is operated by Integrated Financial Arrangements Ltd. 29 Clement's Lane, London EC4N 7AE. Tel: (020) 7608 4900 Fax: (020) 7608 5300. (Registered office: as above; Registered in England and Wales under number: 3727592). Authorised and regulated by the Financial Conduct Authority (entered on the Financial Services Register; no. 190856). -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20180409/0e6dad5a/attachment.html>