Gluster users - Jun 2015 - [Nfs-ganesha-devel] Problems in /usr/libexec/ganesha/dbus-send.sh and ganesha dbus interface when disabling exports from gluster

On 06/17/2015 10:57 PM, Alessandro De Salvo wrote:
> Hi,
> when disabling exports from gluster 3.7.1, by using gluster vol set
<volume> ganesha.enable off, I always get the following error:
>
> Error: Dynamic export addition/deletion failed. Please see log file for
details
>
> This message is produced by the failure of
/usr/libexec/ganesha/dbus-send.sh, and in fact if I manually perform the command
to remove the share I see:
you got it wrong. '/usr/libexec/ganesha/dbus-send.sh' is used by 
Gluster-CLI to unexport the volume "gluster volume set <volname> 
ganesha.enable off" which rightly deletes the export file too while 
un-exporting the volume.
>
> # dbus-send --print-reply --system --dest=org.ganesha.nfsd
/org/ganesha/nfsd/ExportMgr org.ganesha.nfsd.exportmgr.RemoveExport uint16:2
> Error org.freedesktop.DBus.Error.NoReply: Message did not receive a reply
(timeout by message bus)
>
> So, there is a timeout and it fails completely.
Check if nfs-ganesha is still running. There was a bug in unexporting 
the volume. Its been fixed recently in V2.3-dev, yet to be back-ported 
to V2.2-stable branch.
https://review.gerrithub.io/#/c/236129/

Thanks,
Soumya
> In this case I think there is a bug in /usr/libexec/ganesha/dbus-send.sh,
since it blindly deletes the share config if the RemoveExport fails (function
check_cmd_status()), but leaves the %include inside ganesha.conf as in the
check_cmd_status() there is a runaway condition and the other removal statements
are then not executed. I believe the logic should be fixed here, otherwise even
a restart of the service will fail due to the bad configuration.
>
> What?s more worrying is the problem with the dbus. Issuing a DisplayExport
before the RemoveExport apparently fixes the problem, so something like this
always works:
>
> # dbus-send --print-reply --system --dest=org.ganesha.nfsd
/org/ganesha/nfsd/ExportMgr org.ganesha.nfsd.exportmgr.DisplayExport uint16:2
> # dbus-send --print-reply --system --dest=org.ganesha.nfsd
/org/ganesha/nfsd/ExportMgr org.ganesha.nfsd.exportmgr.RemoveExport uint16:2
>
> So, it?s like the DisplayExport is forcing someway a refresh that is needed
by the RemoveExport. Any idea why?
> I?m using the latest version of ganesha 2.2.0, i.e. 2.2.0-3.
> Thanks,
>
> 	Alessandro
>
> PS: sorry for reporting so many things in a few days :-)
>
>
>
>
------------------------------------------------------------------------------
>
>
>
> _______________________________________________
> Nfs-ganesha-devel mailing list
> Nfs-ganesha-devel at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel
>

Hi Soumya,
> Il giorno 17/giu/2015, alle ore 19:41, Soumya Koduri <skoduri at
redhat.com> ha scritto:
> 
> 
> 
> On 06/17/2015 10:57 PM, Alessandro De Salvo wrote:
>> Hi,
>> when disabling exports from gluster 3.7.1, by using gluster vol set
<volume> ganesha.enable off, I always get the following error:
>> 
>> Error: Dynamic export addition/deletion failed. Please see log file for
details
>> 
>> This message is produced by the failure of
/usr/libexec/ganesha/dbus-send.sh, and in fact if I manually perform the command
to remove the share I see:
> you got it wrong. '/usr/libexec/ganesha/dbus-send.sh' is used by
Gluster-CLI to unexport the volume "gluster volume set <volname>
ganesha.enable off" which rightly deletes the export file too while
un-exporting the volume.
Indeed, but the code is the following, only showing the two important functions:


function check_cmd_status()
{
        if [ "$1" != "0" ]
                 then
                 rm -rf $GANESHA_DIR/exports/export.$VOL.conf
                 exit 1
        fi
}

function dynamic_export_remove()
{
        removed_id=`cat $GANESHA_DIR/exports/export.$VOL.conf |\
grep Export_Id | cut -d " " -f8`
        check_cmd_status `echo $?`
        dbus-send --print-reply --system \
--dest=org.ganesha.nfsd /org/ganesha/nfsd/ExportMgr \
org.ganesha.nfsd.exportmgr.RemoveExport uint16:$removed_id
        check_cmd_status `echo $?`
        sed -i /$VOL.conf/d $CONF
        rm -rf $GANESHA_DIR/exports/export.$VOL.conf

}


So, if the dbus-send fails the function check_cmd_status() just deletes
$GANESHA_DIR/exports/export.$VOL.conf and then quits the whole script, so the
last two lines (the sed and the rm) are never executed in
dynamic_export_remove()
> 
>> 
>> # dbus-send --print-reply --system --dest=org.ganesha.nfsd
/org/ganesha/nfsd/ExportMgr org.ganesha.nfsd.exportmgr.RemoveExport uint16:2
>> Error org.freedesktop.DBus.Error.NoReply: Message did not receive a
reply (timeout by message bus)
>> 
>> So, there is a timeout and it fails completely.
> Check if nfs-ganesha is still running. There was a bug in unexporting the
volume. Its been fixed recently in V2.3-dev, yet to be back-ported to
V2.2-stable branch.
> https://review.gerrithub.io/#/c/236129/
Ah, I see! Yes, ganesha was still alive, otherwise the message would have been
different (something like ganesha is unknown by messagebus), and I have also
checked with service nfs-ganesha status. However, probably due to the bug you?re
mentioning, without issuing e.g. a DisplayExport I?m always getting timeouts.
Thanks,

	Alessandro
> 
> Thanks,
> Soumya
> 
>> In this case I think there is a bug in
/usr/libexec/ganesha/dbus-send.sh, since it blindly deletes the share config if
the RemoveExport fails (function check_cmd_status()), but leaves the %include
inside ganesha.conf as in the check_cmd_status() there is a runaway condition
and the other removal statements are then not executed. I believe the logic
should be fixed here, otherwise even a restart of the service will fail due to
the bad configuration.
>> 
>> What?s more worrying is the problem with the dbus. Issuing a
DisplayExport before the RemoveExport apparently fixes the problem, so something
like this always works:
>> 
>> # dbus-send --print-reply --system --dest=org.ganesha.nfsd
/org/ganesha/nfsd/ExportMgr org.ganesha.nfsd.exportmgr.DisplayExport uint16:2
>> # dbus-send --print-reply --system --dest=org.ganesha.nfsd
/org/ganesha/nfsd/ExportMgr org.ganesha.nfsd.exportmgr.RemoveExport uint16:2
>> 
>> So, it?s like the DisplayExport is forcing someway a refresh that is
needed by the RemoveExport. Any idea why?
>> I?m using the latest version of ganesha 2.2.0, i.e. 2.2.0-3.
>> Thanks,
>> 
>> 	Alessandro
>> 
>> PS: sorry for reporting so many things in a few days :-)
>> 
>> 
>> 
>>
------------------------------------------------------------------------------
>> 
>> 
>> 
>> _______________________________________________
>> Nfs-ganesha-devel mailing list
>> Nfs-ganesha-devel at lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel
>> 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 1770 bytes
Desc: not available
URL:
<http://www.gluster.org/pipermail/gluster-users/attachments/20150617/2ba68683/attachment.p7s>

On 06/17/2015 10:57 PM, Alessandro De Salvo wrote:
> Hi,
> when disabling exports from gluster 3.7.1, by using gluster vol set
<volume> ganesha.enable off, I always get the following error:
>
> Error: Dynamic export addition/deletion failed. Please see log file for
details
>
> This message is produced by the failure of
/usr/libexec/ganesha/dbus-send.sh, and in fact if I manually perform the command
to remove the share I see:
you got it wrong. '/usr/libexec/ganesha/dbus-send.sh' is used by 
Gluster-CLI to unexport the volume "gluster volume set <volname> 
ganesha.enable off" which rightly deletes the export file too while 
un-exporting the volume.
>
> # dbus-send --print-reply --system --dest=org.ganesha.nfsd
/org/ganesha/nfsd/ExportMgr org.ganesha.nfsd.exportmgr.RemoveExport uint16:2
> Error org.freedesktop.DBus.Error.NoReply: Message did not receive a reply
(timeout by message bus)
>
> So, there is a timeout and it fails completely.
Check if nfs-ganesha is still running. There was a bug in unexporting 
the volume. Its been fixed recently in V2.3-dev, yet to be back-ported 
to V2.2-stable branch.
https://review.gerrithub.io/#/c/236129/

Thanks,
Soumya
> In this case I think there is a bug in /usr/libexec/ganesha/dbus-send.sh,
since it blindly deletes the share config if the RemoveExport fails (function
check_cmd_status()), but leaves the %include inside ganesha.conf as in the
check_cmd_status() there is a runaway condition and the other removal statements
are then not executed. I believe the logic should be fixed here, otherwise even
a restart of the service will fail due to the bad configuration.
Yes. I see that the "sed -i /$VOL.conf/d $CONF" is placed after the
check_cmd_status. I shall send a fix upstream in a related bug. But dynamic
export removal
will fail in three cases, 
1. nfs-ganesha is not running.
2. The export file that is particular to that volume is somehow deleted before
you perform the removal. It does depend on that file to get the export ID.
3. The bug that Soumya pointed out. 

If it is failing consistently, there could be something that you are missing. If
you can send the exact sequence of sequence of steps that you have executed,
I can help you with it.

Ideally after exporting a particular volume, you'll see an entry in the
/etc/ganesha/ganesha.conf file and the export file in
"/etc/ganesha/exports" dir. If you
have this in place and nfs-ganesha running, then dynamic export removal should
work just fine.

Meghana

>
> What?s more worrying is the problem with the dbus. Issuing a DisplayExport
before the RemoveExport apparently fixes the problem, so something like this
always works:
>
> # dbus-send --print-reply --system --dest=org.ganesha.nfsd
/org/ganesha/nfsd/ExportMgr org.ganesha.nfsd.exportmgr.DisplayExport uint16:2
> # dbus-send --print-reply --system --dest=org.ganesha.nfsd
/org/ganesha/nfsd/ExportMgr org.ganesha.nfsd.exportmgr.RemoveExport uint16:2
>
> So, it?s like the DisplayExport is forcing someway a refresh that is needed
by the RemoveExport. Any idea why?
> I?m using the latest version of ganesha 2.2.0, i.e. 2.2.0-3.
> Thanks,
>
> 	Alessandro
>
> PS: sorry for reporting so many things in a few days :-)
>
>
>
>
------------------------------------------------------------------------------
>
>
>
> _______________________________________________
> Nfs-ganesha-devel mailing list
> Nfs-ganesha-devel at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel
>
------------------------------------------------------------------------------
_______________________________________________
Nfs-ganesha-devel mailing list
Nfs-ganesha-devel at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel