Hello, It seems the gluster hadoop plugin assumes all hadoop daemons/commands are run as root? I was having trouble getting the jobtracker to start because every time the fs is initialized a system call "mount -t glusterfs ..." is issued. Cloudera runs all daemons as the mapred user who is not allowed to run mount, so this is failing. I modified GlusterFileSystem.java (see attached diff) and set fs.glusterfs.automount to false in core-site.xml so this wouldn't happen. That fixed the initial issue of getting daemons to start. My next issue is getting hadoop jobs to run. I get an error: File /mnt/glusterfs/user/james/.staging/job_201304081221_0013/job.xml does not exist. I believe this to be a permissions issue, I can access this file fine from my account, but the .staging directory is only accessible by the user who launches the job : drwx------ 8 james james 870 Apr 8 14:10 .staging If I change the permissions, they are changed back (by Cloudera's hadoop) when I launch a job: Permissions on staging directory glusterfs://node001:9000/user/james/.staging are incorrect: rwxrwxrwx. Fixing permissions to correct value rwx------ Any ideas of a work around would be greatly appreciated. Thanks, James -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20130408/59a1f64d/attachment.html> -------------- next part -------------- A non-text attachment was scrubbed... Name: automount.diff Type: application/octet-stream Size: 1056 bytes Desc: not available URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20130408/59a1f64d/attachment.obj>
Hi james ! 1) Yes, right now, we run as root. Thanks for noticing :) ... We are working on modifying this in the very near future. The problem is that the plugin attempts to mount a filesystem, but we recently have discussed that auto mount behaviour may be a superfluous feature, since mounting can easily be automated for nodes in a cluster. 2) You're right the pervious version of the gluster hadoop filesystem implementation did not deal correctly with privileges. This is now fixed, however. You can get a "bleeding edge" jar which fixes your permissions error from the glusterfs-hadoop github repository: https://github.com/gluster/hadoop-glusterfs, where these fixes have been merged into head. Also we can get you this jar prebuilt if you want, just let me know! Thanks for trying out the GlusterFileSystem and keep the feedback coming ! ----- Original Message ----- From: "James Gurtowski" <gurtowskij at gmail.com> To: jvyas at redhat.com Cc: gluster-users at gluster.org Sent: Monday, April 8, 2013 2:17:44 PM Subject: Gluster and Cloudera's Hadoop Hello, It seems the gluster hadoop plugin assumes all hadoop daemons/commands are run as root? I was having trouble getting the jobtracker to start because every time the fs is initialized a system call "mount -t glusterfs ..." is issued. Cloudera runs all daemons as the mapred user who is not allowed to run mount, so this is failing. I modified GlusterFileSystem.java (see attached diff) and set fs.glusterfs.automount to false in core-site.xml so this wouldn't happen. That fixed the initial issue of getting daemons to start. My next issue is getting hadoop jobs to run. I get an error: File /mnt/glusterfs/user/james/.staging/job_201304081221_0013/job.xml does not exist. I believe this to be a permissions issue, I can access this file fine from my account, but the .staging directory is only accessible by the user who launches the job : drwx------ 8 james james 870 Apr 8 14:10 .staging If I change the permissions, they are changed back (by Cloudera's hadoop) when I launch a job: Permissions on staging directory glusterfs://node001:9000/user/james/.staging are incorrect: rwxrwxrwx. Fixing permissions to correct value rwx------ Any ideas of a work around would be greatly appreciated. Thanks, James
Hi james ! 1) Yes, right now, we run as root. Thanks for noticing :) ... We are working on modifying this in the very near future. The problem is that the plugin attempts to mount a filesystem, but we recently have discussed that auto mount behaviour may be a superfluous feature, since mounting can easily be automated for nodes in a cluster. 2) You're right the pervious version of the gluster hadoop filesystem implementation did not deal correctly with privileges. This is now fixed, however. You can get a "bleeding edge" jar which fixes your permissions error from the glusterfs-hadoop github repository: https://github.com/gluster/hadoop-glusterfs, where these fixes have been merged into head. Also we can get you this jar prebuilt if you want, just let me know... Thanks for trying out the GlusterFileSystem and keep the feedback coming .......! -- Jay Vyas http://jayunit100.blogspot.com -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20130408/c6eb4692/attachment.html>