All,
Appreciate any leads on the following:
In a recent blind-validation study of a depression screening instrument
we used a two-stage sampling design.
In stage 1, we used a broad paper-and-pencil screen to identify likely
positives (say 30% of entire sample). In stage 2 we conducted in-depth
interviews with the 30% of likely positives plus another 20% of the
negatives as controls. We simply did not have resources to interview
all negatives.
A reviewer pointed out that measures such as sensitivity, specificity,
PPV, NPV, AUC, etc. are potentially biased because they are based on a
sample that is over-weighted by positives. That is, we let 50% of our
original sample (many negatives) "get away". The editor suggested
"weighting" to provide unbiased estimates. Perhaps we are not looking
in the right places, but we have not found good examples of how to do
this "weighting".
Has anyone in the R world encountered a similar problem, and if so can
they direct me to R packages that might help out?
Paul
[[alternative HTML version deleted]]
