Ocfs2 users - Dec 2011 - OCFS2 showing "No space left on device" on a device with free space

Hi ,

I am getting the error "No space left on device" on a device with free
space which is ocfs2 filesystem.
Additional information is as below,

[root at sai93 staging]# debugfs.ocfs2 -n -R "stats" /dev/sdb1 | grep
-i
"Cluster Size"
        Block Size Bits: 12   Cluster Size Bits: 15

[root at sai93 ~]# cat /etc/redhat-release
Red Hat Enterprise Linux Server release 5.3 (Tikanga)

[root at sai93 ~]# uname -r
2.6.18-128.el5

[root at sai93 ~]# rpm -qa| grep ocfs
ocfs2console-1.4.4-1.el5
ocfs2-2.6.18-128.el5-1.4.7-1.el5
ocfs2-tools-1.4.4-1.el5

[root at sai93 ~]# cat /etc/ocfs2/cluster.conf
node:
        ip_port = 7777
        ip_address = 10.71.10.93
        number = 0
        name = sai93
        cluster = ocfs2
node:
        ip_port = 7777
        ip_address = 10.71.10.94
        number = 1
        name = sai94
        cluster = ocfs2
cluster:
        node_count =2
        name = ocfs2

[root at sai93 ~]#

The only way I overcome this error is by unmounting the ocfs2
filesystem, doing a fsck followed by server reboot.

988  fsck.ocfs2 /dev/sdb1
  989  mount -a
  990  df -h
  991  touch /mnt/staging/test
  992  man fsck.ocfs2
  993  umount /mnt
  994  service ocfs2 stop
  995  dmesg
  996  reboot

After restarting the servers  I have executed the stat_sysdir.sh script,
the output of which is attached herwith. Kindly confirm whether I need
to do any changes to ocfs2 configuration , to avoid the above error.

 <<output.zip>> 

Thanks & Regards,

Ravi  Ramappa




-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://oss.oracle.com/pipermail/ocfs2-users/attachments/20111206/e48a72ac/attachment-0001.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: output.zip
Type: application/x-zip-compressed
Size: 18351 bytes
Desc: output.zip
Url :
http://oss.oracle.com/pipermail/ocfs2-users/attachments/20111206/e48a72ac/attachment-0001.bin

Hello,

This is a known issue and was fixed in ocfs2 1.6 series by enabling the
discontig-bg feature.

If you are not using DRDB you can upgrade your kernel and ocfs2-tools
using the ULEK 5.5 series, provided by oracle at
http://public-yum.oracle.com/ .

Regards,
S?rgio

Em Tue, 6 Dec 2011 17:58:15 +0800
"Ramappa, Ravi (NSN - IN/Bangalore)" <ravi.ramappa at nsn.com>
escreveu:
> Hi ,
> 
> I am getting the error "No space left on device" on a device with
free
> space which is ocfs2 filesystem.
> Additional information is as below,
> 
> [root at sai93 staging]# debugfs.ocfs2 -n -R "stats" /dev/sdb1 |
grep -i
> "Cluster Size"
>         Block Size Bits: 12   Cluster Size Bits: 15
> 
> [root at sai93 ~]# cat /etc/redhat-release
> Red Hat Enterprise Linux Server release 5.3 (Tikanga)
> 
> [root at sai93 ~]# uname -r
> 2.6.18-128.el5
> 
> [root at sai93 ~]# rpm -qa| grep ocfs
> ocfs2console-1.4.4-1.el5
> ocfs2-2.6.18-128.el5-1.4.7-1.el5
> ocfs2-tools-1.4.4-1.el5
> 
> [root at sai93 ~]# cat /etc/ocfs2/cluster.conf
> node:
>         ip_port = 7777
>         ip_address = 10.71.10.93
>         number = 0
>         name = sai93
>         cluster = ocfs2
> node:
>         ip_port = 7777
>         ip_address = 10.71.10.94
>         number = 1
>         name = sai94
>         cluster = ocfs2
> cluster:
>         node_count =2
>         name = ocfs2
> 
> [root at sai93 ~]#
> 
> The only way I overcome this error is by unmounting the ocfs2
> filesystem, doing a fsck followed by server reboot.
> 
> 988  fsck.ocfs2 /dev/sdb1
>   989  mount -a
>   990  df -h
>   991  touch /mnt/staging/test
>   992  man fsck.ocfs2
>   993  umount /mnt
>   994  service ocfs2 stop
>   995  dmesg
>   996  reboot
> 
> After restarting the servers  I have executed the stat_sysdir.sh
> script, the output of which is attached herwith. Kindly confirm
> whether I need to do any changes to ocfs2 configuration , to avoid
> the above error.
> 
>  <<output.zip>> 
> 
> Thanks & Regards,
> 
> Ravi  Ramappa
> 
> 
> 
> 


-- 
  .:''''':.
.:'        `     S?rgio Surkamp | Administrador de Redes
::    ........   sergio at gruposinternet.com.br
`:.        .:'
  `:,   ,.:'     *Grupos Internet S.A.*
    `: :'        R. Lauro Linhares, 2123 Torre B - Sala 201
     : :         Trindade - Florian?polis - SC
     :.'
     ::          +55 48 3234-4109
     :
     '           http://www.gruposinternet.com.br

>Em Tue, 6 Dec 2011 17:58:15 +0800
>"Ramappa, Ravi (NSN - IN/Bangalore)" <ravi.ramappa at
nsn.com> escreveu:
>
>> Hi ,
>>
>> I am getting the error "No space left on device" on a device
with free
>> space which is ocfs2 filesystem.
>> Additional information is as below,
>>
>> [root at sai93 staging]# debugfs.ocfs2 -n -R "stats"
/dev/sdb1 | grep -i
>> "Cluster Size"
>>? ? ? ?  Block Size Bits: 12?  Cluster Size Bits: 15
>>
>> [root at sai93 ~]# cat /etc/redhat-release
>> Red Hat Enterprise Linux Server release 5.3 (Tikanga)
>>
>> [root at sai93 ~]# uname -r
>> 2.6.18-128.el5
>>
>> [root at sai93 ~]# rpm -qa| grep ocfs
>> ocfs2console-1.4.4-1.el5
>> ocfs2-2.6.18-128.el5-1.4.7-1.el5
>> ocfs2-tools-1.4.4-1.el5
>>
>> [root at sai93 ~]# cat /etc/ocfs2/cluster.conf
>> node:
>>? ? ? ?  ip_port = 7777
>>? ? ? ?  ip_address = 10.71.10.93
>>? ? ? ?  number = 0
>>? ? ? ?  name = sai93
>>? ? ? ?  cluster = ocfs2
>> node:
>>? ? ? ?  ip_port = 7777
>>? ? ? ?  ip_address = 10.71.10.94
>>? ? ? ?  number = 1
>>? ? ? ?  name = sai94
>>? ? ? ?  cluster = ocfs2
>> cluster:
>>? ? ? ?  node_count =2
>>? ? ? ?  name = ocfs2
>>
>> [root at sai93 ~]#
>>
>> The only way I overcome this error is by unmounting the ocfs2
>> filesystem, doing a fsck followed by server reboot.
>>
>> 988? fsck.ocfs2 /dev/sdb1
>>?  989? mount -a
>>?  990? df -h
>>?  991? touch /mnt/staging/test
>>?  992? man fsck.ocfs2
>>?  993? umount /mnt
>>?  994? service ocfs2 stop
>>?  995? dmesg
>>?  996? reboot
>>
>> After restarting the servers? I have executed the stat_sysdir.sh
>> script, the output of which is attached herwith. Kindly confirm
>> whether I need to do any changes to ocfs2 configuration , to avoid
>> the above error.
>>
>>? <<output.zip>>
>>
>> Thanks & Regards,
>>
>> Ravi? Ramappa
>
>Hello,
>
>This is a known issue and was fixed in ocfs2 1.6 series by enabling the
>discontig-bg feature.
>
>If you are not using DRDB you can upgrade your kernel and ocfs2-tools
>using the ULEK 5.5 series, provided by oracle at
>http://public-yum.oracle.com/ .
>
>Regards,
>S?rgio
We are also having a similar issue, but we are using DRBD.? We are set at the
minimum 2 node slots, so I can't reduce those any more.

Is there any other good way to fix this problem, at least for a while?? What
about ways to preemptively clear some fragmentation before we get the dreaded
error?

I think moving files off the filesystem and back works somewhat, but how can I
tell which files to move would have the most effect?

Thanks,
Herman
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://oss.oracle.com/pipermail/ocfs2-users/attachments/20111207/b9c4846e/attachment.html