CentOS - Feb 2014 - Application Hang issue in RHEL6.4

Hi All,


We see our application Hanging at shmctl (SHM_LOCK).



Kernel :  2.6.32-358.el6.x86_64

OS : Red Hat Enterprise Linux Server release 6.4 (Santiago)





1.       When it hangs, this is what strace gives:



[RHEL6.4 at 10.201.0.221 bin]# strace -p 23793 Process 23793 attached -
interrupt to quit [ Process PID=23793 runs in 32 bit mode. ]

shmget(0x110c1be, 16777216, 0666)       = 3801140

shmat(3801140, 0, 0)                    = 0x85c2c70e7755000

shmctl(3801140, IPC_64|SHM_LOCK, 0^C <unfinished ...>







2.       When we kill the application and generate a core file, Here is what 
gdb shows:



Core was generated by ` <our application> 12 12 10.33.0.230 eth1 -1 0 3
fd5d:d50a:8c17:3300::e6'.

Program terminated with signal 6, Aborted.

#0  0x007c2430 in __kernel_vsyscall ()

Missing separate debuginfos, use: debuginfo-install glibc-2.12-1.107.el6.i686
libgcc-4.4.7-3.el6.i686 libstdc++-4.4.7-3.el6.i686

(gdb) bt

#0  0x007c2430 in __kernel_vsyscall ()

#1  0x008a8d8e in shmctl@@GLIBC_2.2 () from /lib/libc.so.6

#2  0x0847c13e in MpcMpSwms_SwibIfShmSegOpen (ps=<value optimized out>,
ctl=<value optimized out>)

   at src/MpcMpSwms_SwibIfShm.c:271

#3  0x0847c551 in MpcMpSwms_SwibIfShmOpen (pvs=0xb0e8984, pvctl=0xffc35354) at
src/MpcMpSwms_SwibIfShm.c:379

#4  0x0847bb9f in MpcMpSwms_SwibIfOpen (ps=0xb0e7dc4, ctl=0xffc35354,
swib_if=SWIB_IF_SHM) at src/MpcMpSwms_SwibIf.c:126

#5  0x080b792a in MpcMpSwms_SwibOpen (ps=0xa019da8, ctl=0xffc357d8) at
src/MpcMpSwms_Swib.c:253

#6  0x080b63a0 in main (argc=9, argv=0xffc35924) at src/MpcMpSwms_Main.c:1364

(gdb)





What could be the reason for hanging?

Any solution?



-miraj

On 02/06/2014 07:39 AM, Miraj Mohamed wrote:
> Hi All,
>
>
> We see our application Hanging at shmctl (SHM_LOCK).
>
>
>
> Kernel :  2.6.32-358.el6.x86_64
>
> OS : Red Hat Enterprise Linux Server release 6.4 (Santiago)
>
>
>
>
>
> 1.       When it hangs, this is what strace gives:
>
>
>
> [RHEL6.4 at 10.201.0.221 bin]# strace -p 23793 Process 23793 attached -
interrupt to quit [ Process PID=23793 runs in 32 bit mode. ]
>
> shmget(0x110c1be, 16777216, 0666)       = 3801140
>
> shmat(3801140, 0, 0)                    = 0x85c2c70e7755000
>
> shmctl(3801140, IPC_64|SHM_LOCK, 0^C <unfinished ...>
>
>
>
>
>
>
>
> 2.       When we kill the application and generate a core file, Here is
what  gdb shows:
>
>
>
> Core was generated by ` <our application> 12 12 10.33.0.230 eth1 -1 0
3 fd5d:d50a:8c17:3300::e6'.
>
> Program terminated with signal 6, Aborted.
>
> #0  0x007c2430 in __kernel_vsyscall ()
>
> Missing separate debuginfos, use: debuginfo-install
glibc-2.12-1.107.el6.i686 libgcc-4.4.7-3.el6.i686 libstdc++-4.4.7-3.el6.i686
>
> (gdb) bt
>
> #0  0x007c2430 in __kernel_vsyscall ()
>
> #1  0x008a8d8e in shmctl@@GLIBC_2.2 () from /lib/libc.so.6
>
> #2  0x0847c13e in MpcMpSwms_SwibIfShmSegOpen (ps=<value optimized
out>, ctl=<value optimized out>)
>
>     at src/MpcMpSwms_SwibIfShm.c:271
>
> #3  0x0847c551 in MpcMpSwms_SwibIfShmOpen (pvs=0xb0e8984, pvctl=0xffc35354)
at src/MpcMpSwms_SwibIfShm.c:379
>
> #4  0x0847bb9f in MpcMpSwms_SwibIfOpen (ps=0xb0e7dc4, ctl=0xffc35354,
swib_if=SWIB_IF_SHM) at src/MpcMpSwms_SwibIf.c:126
>
> #5  0x080b792a in MpcMpSwms_SwibOpen (ps=0xa019da8, ctl=0xffc357d8) at
src/MpcMpSwms_Swib.c:253
>
> #6  0x080b63a0 in main (argc=9, argv=0xffc35924) at
src/MpcMpSwms_Main.c:1364
>
> (gdb)
>
>
>
>
>
> What could be the reason for hanging?
>
> Any solution?
>
>
This is a CENTOS mailing list, not Red Hat's. Please go to Red Hat 
bugzilla and report your problem there.


-- 
Ljubomir Ljubojevic
(Love is in the Air)
PL Computers
Serbia, Europe

StarOS, Mikrotik and CentOS/RHEL/Linux consultant