Gluster users - Jan 2010 - Gluster+NFS as VM datastore for ESXi - issues

Hi all,

I've been trying out Gluster Platform, so some info will be from the
management interface. If additional data from logs is required, please let me
know which logfiles.

When using a Gluster NFS volume as a ESXi datastore, creation of a disk file
(vmdk) takes a considerable amount of time (with unfsd using 10-50% cpu) and
then crashes unfsd. A reboot of the system seems necessary to restore proper
operation. On a mirror volume this resulted in a severely corrupted volume (red
status icon), a regular volume was still seen as ok (green status icon) after
the reboot. Accessing a directory on this volume (through the ESXi datastore
browser) again results in a unfsd crash. None of these crashes result in errors
logged to the Log Manager, I noticed unfsd had crashed by looking at
/var/log/messages.

Has anyone successfully used Gluster+NFS as a datastore for ESXi? Which settings
have been used for this setup?

How can I enable useful logging for unfsd, without being swamped in debug
messages?

Also of interest: ESXi mentions that the datastore is responsible for deciding
whether a disk file will be thin provisioned (pre-allocated or spare). It seems
that unfsd is configured to pre-allocate disk files (not 100% sure about this),
which is wasteful in regards of diskspace and might cause or considerably
aggravate fore mentioned issue. Does anyone know how this behavior can be
modified?


Sincerely,

Milo de Vries
Managed Hosting | Sys Admin

info.nl?| full service internet agency
Sint Antoniesbreestraat 16 | 1011 HB Amsterdam
t. +31 20 530 9111
f. +31 20 530 9101
Milo at info.nl
http://info.nl

??Please consider the environment - do you really need to print this email?

Hi Milo,

 * replies inline *

On Thu, Jan 28, 2010 at 7:59 PM, Milo de Vries <Milo at info.nl> wrote:
> Hi all,
>
> I've been trying out Gluster Platform, so some info will be from the
> management interface. If additional data from logs is required, please let
> me know which logfiles.
>
When using a Gluster NFS volume as a ESXi datastore, creation of a disk
file
> (vmdk) takes a considerable amount of time (with unfsd using 10-50% cpu)
and
> then crashes unfsd. A reboot of the system seems necessary
We have seen crashes with the 3.0.0 release with NFS reexport, we will soon
release an update for that version where you will not face any issues with
NFS reexport.

> to restore proper operation. On a mirror volume this resulted in a severely
> corrupted volume (red status icon), a regular volume was still seen as ok
> (green status icon) after the reboot. Accessing a directory on this volume
> (through the ESXi datastore browser) again results in a unfsd crash. None
of
> these crashes result in errors logged to the Log Manager, I noticed unfsd
> had crashed by looking at /var/log/messages.
>
> Has anyone successfully used Gluster+NFS as a datastore for ESXi? Which
> settings have been used for this setup?
>
We already have fixed this issue, please wait for an update through our GSN
which will have the fix.

Regards
Harshavardhana
>
> How can I enable useful logging for unfsd, without being swamped in debug
> messages?
>
> Also of interest: ESXi mentions that the datastore is responsible for
> deciding whether a disk file will be thin provisioned (pre-allocated or
> spare). It seems that unfsd is configured to pre-allocate disk files (not
> 100% sure about this), which is wasteful in regards of diskspace and might
> cause or considerably aggravate fore mentioned issue. Does anyone know how
> this behavior can be modified?
>
>
> Sincerely,
>
> Milo de Vries
> Managed Hosting | Sys Admin
>
> info.nl | full service internet agency
> Sint Antoniesbreestraat 16 | 1011 HB Amsterdam
> t. +31 20 530 9111
> f. +31 20 530 9101
> Milo at info.nl
> http://info.nl
>
> ? Please consider the environment - do you really need to print this email?
>
>
>
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
>