mohan L
2009-Jan-20 10:40 UTC
[Gluster-users] Re : problem with running mysql on glusterfs
Hello. I would like to ask about having mysql data hosted on glusterfs. Please see my issue below. I have posted DEBUG log information on pastebin. I am using GlusterFS version: glusterfs 1.4.0rc7 FUSE Version :fuse-2.7.3glfs10 my ISSUE is: When hosting mysql data on glusterfs I have an issue: The first time I start the glusterfsd server using [root at mohan ~]# /usr/local/sbin/glusterfsd -f /usr/local/etc/glusterfs/glusterfs-server.vol.sample and mount the glusterfs client (/var/lib/mysql) mountpoint using [root at localhost ~]# /usr/local/sbin/glusterfs -f /usr/local/etc/glusterfs/glusterfs-client.vol.sample /var/lib/mysql Ok fine . then i am trying to login it through error [root at localhost ~]# service mysqld status mysqld (pid 1700) is running... [root at localhost ~]# mysql -u root ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2) Then I am unmounted using [root at localhost ~]# umount /var/lib/mysql/ Again mounted using [root at localhost ~]# /usr/local/sbin/glusterfs -f /usr/local/etc/glusterfs/glusterfs-client.vol.sample /var/lib/mysql i am getting the same problem like [root at localhost ~]# mysql -u root ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2) then I checked log ,there is no any error [root at localhost ~]# cat /var/log/mysqld.log I stoped and start mysqld in client machine using [root at localhost ~]# service mysqld stop Stopping MySQL: [ OK ] [root at localhost ~]# service mysqld start Timeout error occurred trying to start MySQL Daemon. Starting MySQL: [FAILED] Then i am trying to unmount ,it tells device id busy.but I am not using this directory [root at localhost ~]# umount /var/lib/mysql/ umount: /var/lib/mysql: device is busy umount: /var/lib/mysql: device is busy again tryed to login mysql ,still it gives the error [root at localhost ~]# mysql -u root ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2) Then Again I am started mysqld . lost time this was failed .now it is started [root at localhost ~]# service mysqld start Starting MySQL: [ OK ] Then I tryed to login . Now i am able to login .everthing is fine . [root at localhost ~]# mysql Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 2 Server version: 5.0.51a Source distribution Type 'help;' or '\h' for help. Type '\c' to clear the buffer. mysql> Now I am getting log file it has error like this . [root at localhost ~]# cat /var/log/mysqld.log 090120 20:52:20 mysqld ended 090120 20:52:29 mysqld started InnoDB: Unable to lock ./ibdata1, error: 38 090120 20:52:29 InnoDB: Retrying to lock the first data file InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 090120 20:53:42 mysqld started InnoDB: Unable to lock ./ibdata1, error: 38 090120 20:53:42 InnoDB: Retrying to lock the first data file InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 090120 20:54:09 InnoDB: Unable to open the first data file InnoDB: Error in opening ./ibdata1 090120 20:54:09 InnoDB: Operating system error number 38 in a file operation. InnoDB: Error number 38 means 'Function not implemented'. InnoDB: Some operating system error numbers are described at InnoDB: http://dev.mysql.com/doc/refman/5.0/en/operating-system-error-codes.html InnoDB: Could not open or create data files. InnoDB: If you tried to add new data files, and it failed here, InnoDB: you should now edit innodb_data_file_path in my.cnf back InnoDB: to what it was, and remove the new ibdata files InnoDB created InnoDB: in this failed attempt. InnoDB only wrote those files full of InnoDB: zeros, but did not yet use them in any way. But be careful: do not InnoDB: remove old data files which contain your precious data! 090120 20:54:09 [Note] /usr/libexec/mysqld: ready for connections. Version: '5.0.51a' socket: '/var/lib/mysql/mysql.sock' port: 3306 Source distribution InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 InnoDB: Unable to lock ./ibdata1, error: 38 NOTE : I am using load posix-locks translator in my server vol file like this volume brick type storage/posix # POSIX FS translator option directory /mnt/mymysql/mysql #bench # Export this directory end-volume volume plocks type features/posix-locks subvolumes brick # option mandatory on end-volume http://www.mail-archive.com/gluster-devel at nongnu.org/msg03125.html some one faced like this ,he is using different version .it was solved . can any one please help me ? I am using FUSE Version :fuse-2.7.3glfs10 GlusterFS version: glusterfs 1.4.0rc7 Thanks for your time L. Mohan -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20090120/c7f29e0d/attachment.html>
Keith Freedman
2009-Jan-20 19:40 UTC
[Gluster-users] Re : problem with running mysql on glusterfs
At 02:40 AM 1/20/2009, mohan L wrote:>Hello. I would like to ask about having mysql data hosted on >glusterfs. Please see my issue below. I have posted DEBUG log >information on pastebin. > > >[root at localhost ~]# cat /var/log/mysqld.log >090120 20:52:20 mysqld ended > >090120 20:52:29 mysqld started >InnoDB: Unable to lock ./ibdata1, error: 38 >090120 20:52:29 InnoDB: Retrying to lock the first data file > > >NOTE : I am using load posix-locks translator in my server vol file like this > > >volume plocks > type features/posix-locks > subvolumes brick > # option mandatory on >end-volumeun comment the option mandatory on and see if that solves your problem.>some one faced like this ,he is using different version .it was >solved . can any one please help me ? I am using > >FUSE Version :fuse-2.7.3glfs10 >GlusterFS version: glusterfs 1.4.0rc7clearly form the logs it''s a locking related problem so if that doesn''t solve the problem, then someone will have to look deeper.
Keith Freedman
2009-Jan-20 19:40 UTC
[Gluster-users] Re : problem with running mysql on glusterfs
At 02:40 AM 1/20/2009, mohan L wrote:>Hello. I would like to ask about having mysql data hosted on >glusterfs. Please see my issue below. I have posted DEBUG log >information on pastebin. > > >[root at localhost ~]# cat /var/log/mysqld.log >090120 20:52:20 mysqld ended > >090120 20:52:29 mysqld started >InnoDB: Unable to lock ./ibdata1, error: 38 >090120 20:52:29 InnoDB: Retrying to lock the first data file > > >NOTE : I am using load posix-locks translator in my server vol file like this > > >volume plocks > type features/posix-locks > subvolumes brick > # option mandatory on >end-volumeun comment the option mandatory on and see if that solves your problem.>some one faced like this ,he is using different version .it was >solved . can any one please help me ? I am using > >FUSE Version :fuse-2.7.3glfs10 >GlusterFS version: glusterfs 1.4.0rc7clearly form the logs it's a locking related problem so if that doesn't solve the problem, then someone will have to look deeper.
Anand Avati
2009-Jan-20 20:28 UTC
[Gluster-users] [Gluster-devel] Re : problem with running mysql on glusterfs
> Please see my issue below. I have posted DEBUG log information on pastebin.Can you give a link to your post?> volume brick > type storage/posix # POSIX FS translator > option directory /mnt/mymysql/mysql #bench # Export this > directory > end-volume > > > volume plocks > type features/posix-locks > subvolumes brick > # option mandatory on > end-volume >have you set the remote-subvolume on the client to 'plocks' and not 'brick' ? Please post your complete volfiles (on pastebin if they are too big) and not just snippets. avati