freebsd stable - Oct 2009 - libthr and daemon()

I have some code that tries to use pthread_cond_wait() and it's getting
back EPERM.  Upon further investigation, here's what I've found:

When the app starts, libthr's _libpthread_init calls init_main_thread()
to set the thread id in struct pthread's tid.

The app opens a log file then calls daemon().
daemon() calls fork()
fork() does not appear to be linked to _fork() in libthr; see below.
The app creates a thread to handle signals.
The app attempts to wait on a condition variable (pthread_cond_wait();
this gives EPERM).

Looking into libthr's cond_wait_common(), it does a THR_UMUTEX_LOCK on
the cv's c_lock using the struct pthread from _get_curthread().  Here,
curthread points to the pthread struct that got the tid from thr_self on
startup.  Because of fork() this is the same address in the daemonized
app as the original.  But curthread->tid is the tid of the original app,
not the daemonized version, hence my assumption that fork() didn't
resolve to libthr's _fork().

When cond_wait_common() calls into the kernel to actually do the
cv_wait, do_unlock_umutex/do_unlock_normal() returns EPERM since the tid
does not match.

AFAICT this has nothing to do with any code in the app itself.

The two things I don't know:

1) what utilities can I use to show me which version of fork will be
used at runtime?  ldd just shows me that the app is linked against libc
and libthr.

2) why would fork resolve to the one in libc (presumably, I'm not sure
how to prove this) instead of the one in libthr?

Thanks,
matthew

On Mon, 5 Oct 2009, Matthew Fleming wrote:
> I have some code that tries to use pthread_cond_wait() and it's getting
> back EPERM.  Upon further investigation, here's what I've found:
>
> When the app starts, libthr's _libpthread_init calls init_main_thread()
> to set the thread id in struct pthread's tid.
Is the application threaded before calling daemon()?
> The app opens a log file then calls daemon().
> daemon() calls fork()
> fork() does not appear to be linked to _fork() in libthr; see below.
> The app creates a thread to handle signals.
> The app attempts to wait on a condition variable (pthread_cond_wait();
> this gives EPERM).
Was the condition variable created before daemon() was
called?

The picture is not clear to me.

POSIX states that only async-signal-safe function calls
can be made from a child fork()'d from a threaded
application.  The intent is that the child should soon
after call a function in the exec() family.  Certainly,
any more threaded calls in the child are invalid.

-- 
DE