Hello list!
I''m currently storing Xen sparse image files for VM''s on
remote attached storage devices, and looking to increase the amount of IO
available on it. I''ve ran across MaxIQ SSD read acceleration caching,
and was wondering if this is a smart move, given the way we use the sparse image
files.
My fear is that the MaxIQ will cache entire image files, instead of the most
commonly accessed files *INSIDE* the VM. Can anyone confirm or deny my theory?
I''ve talked with Adaptec and this is what they told me:
--------
It''s hard to say exactly. Keep in mind that a RAID card isn''t
aware of files, only blocks. So depending on how the blocks are accessed, the
MaxIQ product may or may not help you. My first impression with 30GB files is
that it may not be very helpful.
--------
So, that being said, right now we are accessing the image files using the
''file'' mode:
<disk type=''file'' device=''disk''>
<driver name=''file''/>
<source file=''/distributed/vms/29609/disk.img''/>
<target dev=''xvda'' bus=''xen''/>
</disk>
Will the MaxIQ work in the way I want it to, or will I have to change the way we
access the images? Or will it end up trying to cache entire images onto the SSD?
Has anyone had experience with this type of application?
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users
That''s simple: caching only if data exists. All ''holes'' in spare file does not exists. They are just opinion of filesystem, but on block level there is no sparse. One more: MaxIQ cache most reed data. Do you often read holes from sparse? В Чтв, 08/07/2010 в 16:08 -0500, Andy Pace пишет:> Hello list! > > > > I’m currently storing Xen sparse image files for VM’s on remote > attached storage devices, and looking to increase the amount of IO > available on it. I’ve ran across MaxIQ SSD read acceleration caching, > and was wondering if this is a smart move, given the way we use the > sparse image files. > > > > My fear is that the MaxIQ will cache entire image files, instead of > the most commonly accessed files *INSIDE* the VM. Can anyone confirm > or deny my theory? > > > > I’ve talked with Adaptec and this is what they told me: > > > > -------- > > It''s hard to say exactly. Keep in mind that a RAID card isn''t aware of > files, only blocks. So depending on how the blocks are accessed, the > MaxIQ product may or may not help you. My first impression with 30GB > files is that it may not be very helpful. > > -------- > > > > So, that being said, right now we are accessing the image files using > the ‘file’ mode: > > > > <disk type=''file'' device=''disk''> > > <driver name=''file''/> > > <source file=''/distributed/vms/29609/disk.img''/> > > <target dev=''xvda'' bus=''xen''/> > > </disk> > > > > Will the MaxIQ work in the way I want it to, or will I have to change > the way we access the images? Or will it end up trying to cache entire > images onto the SSD? Has anyone had experience with this type of > application? > > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Right, but the data does exist. What I'm wondering is if it will treat the sparse image file as 1 file, or will it cache on a block basis.> Do you often read holes from sparse?I'm not sure how to answer that. -----Original Message----- From: xen-users-bounces@lists.xensource.com [mailto:xen-users-bounces@lists.xensource.com] On Behalf Of George Shuklin Sent: Thursday, July 08, 2010 4:36 PM To: xen-users@lists.xensource.com Subject: Re: [Xen-users] Xen Sparse Image Files and MaxIQ SSD That's simple: caching only if data exists. All 'holes' in spare file does not exists. They are just opinion of filesystem, but on block level there is no sparse. One more: MaxIQ cache most reed data. Do you often read holes from sparse? В Чтв, 08/07/2010 в 16:08 -0500, Andy Pace пишет:> Hello list! > > > > I’m currently storing Xen sparse image files for VM’s on remote > attached storage devices, and looking to increase the amount of IO > available on it. I’ve ran across MaxIQ SSD read acceleration caching, > and was wondering if this is a smart move, given the way we use the > sparse image files. > > > > My fear is that the MaxIQ will cache entire image files, instead of > the most commonly accessed files *INSIDE* the VM. Can anyone confirm > or deny my theory? > > > > I’ve talked with Adaptec and this is what they told me: > > > > -------- > > It's hard to say exactly. Keep in mind that a RAID card isn't aware of > files, only blocks. So depending on how the blocks are accessed, the > MaxIQ product may or may not help you. My first impression with 30GB > files is that it may not be very helpful. > > -------- > > > > So, that being said, right now we are accessing the image files using > the ‘file’ mode: > > > > <disk type='file' device='disk'> > > <driver name='file'/> > > <source file='/distributed/vms/29609/disk.img'/> > > <target dev='xvda' bus='xen'/> > > </disk> > > > > Will the MaxIQ work in the way I want it to, or will I have to change > the way we access the images? Or will it end up trying to cache entire > images onto the SSD? Has anyone had experience with this type of > application? > > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
What data exist in zero-length file? In sparsed file ''holes'' areas does not exists on lower levels, this just ''imagination'' of filesystem. The areas with data are threating normally and would cached to MaxIQ only if reed often. In other words, MaxIQ works on block level. And any block level have no idea about sparsed files or filesystem or any other swaps created on block device. Read requests to empty area will not come to block level. Read requests to filled area and write requests to any area will come as they come for any other files. Those two things just have no any connection. (like your ethernet card does not care about ssh key length in any way). But sparsed files in product have a great performance penalty - in my tests any write operations to sparsed areas reduce performance about twice. Most visible test is filesystem creation. And in long terms many sparsed files on same storage means higher fragmentation (and more random io operations, and performance loss). So, disk space is cheap, filesystems are resizable, so no reason for using sparse in production environment. N.B. For SSD this can be other question, I''m talking about old pain HDD (and even MaxIQ does not change it principally, it just an emulation of big memory cache). В Чтв, 08/07/2010 в 16:56 -0500, Andy Pace пишет:> Right, but the data does exist. What I''m wondering is if it will treat the sparse image file as 1 file, or will it cache on a block basis. > > > Do you often read holes from sparse? > > I''m not sure how to answer that. > > -----Original Message----- > From: xen-users-bounces@lists.xensource.com [mailto:xen-users-bounces@lists.xensource.com] On Behalf Of George Shuklin > Sent: Thursday, July 08, 2010 4:36 PM > To: xen-users@lists.xensource.com > Subject: Re: [Xen-users] Xen Sparse Image Files and MaxIQ SSD > > That''s simple: caching only if data exists. > > All ''holes'' in spare file does not exists. They are just opinion of filesystem, but on block level there is no sparse. > > One more: MaxIQ cache most reed data. Do you often read holes from sparse? > > > В Чтв, 08/07/2010 в 16:08 -0500, Andy Pace пишет: > > Hello list! > > > > > > > > I’m currently storing Xen sparse image files for VM’s on remote > > attached storage devices, and looking to increase the amount of IO > > available on it. I’ve ran across MaxIQ SSD read acceleration caching, > > and was wondering if this is a smart move, given the way we use the > > sparse image files. > > > > > > > > My fear is that the MaxIQ will cache entire image files, instead of > > the most commonly accessed files *INSIDE* the VM. Can anyone confirm > > or deny my theory? > > > > > > > > I’ve talked with Adaptec and this is what they told me: > > > > > > > > -------- > > > > It''s hard to say exactly. Keep in mind that a RAID card isn''t aware of > > files, only blocks. So depending on how the blocks are accessed, the > > MaxIQ product may or may not help you. My first impression with 30GB > > files is that it may not be very helpful. > > > > -------- > > > > > > > > So, that being said, right now we are accessing the image files using > > the ‘file’ mode: > > > > > > > > <disk type=''file'' device=''disk''> > > > > <driver name=''file''/> > > > > <source file=''/distributed/vms/29609/disk.img''/> > > > > <target dev=''xvda'' bus=''xen''/> > > > > </disk> > > > > > > > > Will the MaxIQ work in the way I want it to, or will I have to change > > the way we access the images? Or will it end up trying to cache entire > > images onto the SSD? Has anyone had experience with this type of > > application? > > > > > > _______________________________________________ > > Xen-users mailing list > > Xen-users@lists.xensource.com > > http://lists.xensource.com/xen-users > > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
I think we're getting a bit off topic. We have not noticed any significant performance *DECREASE* using sparse image files. We're getting gigabit line speed writes (130MB/s). I welcome anyone elses comments or feedback in regards to the MaxIQ SSD question. -----Original Message----- From: xen-users-bounces@lists.xensource.com [mailto:xen-users-bounces@lists.xensource.com] On Behalf Of George Shuklin Sent: Thursday, July 08, 2010 6:37 PM To: xen-users@lists.xensource.com Subject: RE: [Xen-users] Xen Sparse Image Files and MaxIQ SSD What data exist in zero-length file? In sparsed file 'holes' areas does not exists on lower levels, this just 'imagination' of filesystem. The areas with data are threating normally and would cached to MaxIQ only if reed often. In other words, MaxIQ works on block level. And any block level have no idea about sparsed files or filesystem or any other swaps created on block device. Read requests to empty area will not come to block level. Read requests to filled area and write requests to any area will come as they come for any other files. Those two things just have no any connection. (like your ethernet card does not care about ssh key length in any way). But sparsed files in product have a great performance penalty - in my tests any write operations to sparsed areas reduce performance about twice. Most visible test is filesystem creation. And in long terms many sparsed files on same storage means higher fragmentation (and more random io operations, and performance loss). So, disk space is cheap, filesystems are resizable, so no reason for using sparse in production environment. N.B. For SSD this can be other question, I'm talking about old pain HDD (and even MaxIQ does not change it principally, it just an emulation of big memory cache). В Чтв, 08/07/2010 в 16:56 -0500, Andy Pace пишет:> Right, but the data does exist. What I'm wondering is if it will treat the sparse image file as 1 file, or will it cache on a block basis. > > > Do you often read holes from sparse? > > I'm not sure how to answer that. > > -----Original Message----- > From: xen-users-bounces@lists.xensource.com > [mailto:xen-users-bounces@lists.xensource.com] On Behalf Of George > Shuklin > Sent: Thursday, July 08, 2010 4:36 PM > To: xen-users@lists.xensource.com > Subject: Re: [Xen-users] Xen Sparse Image Files and MaxIQ SSD > > That's simple: caching only if data exists. > > All 'holes' in spare file does not exists. They are just opinion of filesystem, but on block level there is no sparse. > > One more: MaxIQ cache most reed data. Do you often read holes from sparse? > > > В Чтв, 08/07/2010 в 16:08 -0500, Andy Pace пишет: > > Hello list! > > > > > > > > I’m currently storing Xen sparse image files for VM’s on remote > > attached storage devices, and looking to increase the amount of IO > > available on it. I’ve ran across MaxIQ SSD read acceleration > > caching, and was wondering if this is a smart move, given the way we > > use the sparse image files. > > > > > > > > My fear is that the MaxIQ will cache entire image files, instead of > > the most commonly accessed files *INSIDE* the VM. Can anyone confirm > > or deny my theory? > > > > > > > > I’ve talked with Adaptec and this is what they told me: > > > > > > > > -------- > > > > It's hard to say exactly. Keep in mind that a RAID card isn't aware > > of files, only blocks. So depending on how the blocks are accessed, > > the MaxIQ product may or may not help you. My first impression with > > 30GB files is that it may not be very helpful. > > > > -------- > > > > > > > > So, that being said, right now we are accessing the image files > > using the ‘file’ mode: > > > > > > > > <disk type='file' device='disk'> > > > > <driver name='file'/> > > > > <source file='/distributed/vms/29609/disk.img'/> > > > > <target dev='xvda' bus='xen'/> > > > > </disk> > > > > > > > > Will the MaxIQ work in the way I want it to, or will I have to > > change the way we access the images? Or will it end up trying to > > cache entire images onto the SSD? Has anyone had experience with > > this type of application? > > > > > > _______________________________________________ > > Xen-users mailing list > > Xen-users@lists.xensource.com > > http://lists.xensource.com/xen-users > > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users