R help - Apr 2020 - [FORGED] Re: pair correlation function of 3D points

I am the main author of spatstat, and the author of the code and documentation
for pcf3est.

This is the first time in 25 years that I can remember anyone complaining about
the documentation for the spatstat package.

The available documentation for spatstat includes:

                 - welcome message on startup which explains where to find help
                 - introductory vignette 'Getting Started in Spatstat'
                 - quick reference guide (in the help file
'spatstat-package' , at the front of the full manual, and on website)
                 - online help files (> 2000 help files)
                 - package manual (> 1700 pages)
                 - project website www.spatstat.org
                 - book 'Spatial Point Patterns: Methodology and
Applications with R' (>700 pages)
                 - book companion website book.spatstat.org
                 - technical descriptions in journal articles
                 - ample explanatory comments in the source code

All of these sources (even the Description file) explain that spatstat is mainly
focused on *two-dimensional* spatial point patterns but provides limited support
for other kinds of data, including three-dimensional point patterns.

Three-dimensional point patterns are a relatively small subset of the current
spatstat functionality
(covered in Section 15.3 pages 650-657 of the book, thus approximately 1% of the
book).

The full manual PDF is searchable, has an index, and its internal
cross-references are hyperlinks that can be followed.
Package manuals can also be accessed interactively in R through a web-like
interface.
Package manuals are also available online with a web interface at some websites
like rdrr.io

The help entry for 'pcf3est' says that the argument X must be a
"three-dimensional point pattern (object of class 'pp3')".
Most users would immediately look up 'pp3'. Apparently you were reading
this from the full manual PDF and you say that you couldn't look up
'pp3'. The manual is searchable; you could have searched the PDF for
keyword 'pp3', or jumped to the index, or the contents page. You could
have looked at the first entry in the manual, which is the quick reference
guide, which has a section on 'Three-dimensional point patterns' which
would have listed all the available functionality for three-dimensional point
patterns. You could have searched for 'three dimensional'.
> I was somewhat disappointed that, given how many people are/were
   > involved in this package, not one (after approx 24 hours) had tried to
   > help answer the OP's question.

That's not a complaint about the documentation. It's a complaint that
no-one responded
to your query within 24 hours. Seriously? No-one is obliged to provide this
level of service.
Twenty-four hours is a pretty short time scale for a response to a question that
was not
addressed to anyone in particular. R-help is not a commonly used forum for
asking questions about spatstat.
If you wanted a quick answer you could have emailed the package authors
directly.
> I should have noted that my comments weren't directed towards the main
> authors, but to all people listed in the description file, which is
 > many, including some R core members.

The Description file says clearly that there are three main authors: Baddeley,
Turner and Rubak with email addresses given.
The others are people who have contributed something at some time in the past.
I'm not sure what you hoped to achieve by addressing comments to a wider
audience - most of whom are not listening on this frequency.

 > It's just I strongly feel that good documentation is crucial
(especially in open source),

So do we; that's why we have put so much effort into documentation,
including writing a whole book.
> I started reading the pcf function first.
> This function has the same problem, it doesn't clearly describe the
function arguments.
> It doesn't say whether it applies to 2d, 3d or higher-dimensional data.
> After reading it, I had no idea whether the function could be applied to 3d
data or not.
> In my opinion this is not sufficient.
> Descriptions of function arguments and return values should be clear.
pcf is generic.
The argument 'X' analysed in 'pcf(X)' could be any kind of
spatial data.

The help file for a generic does not go into specific detail about 'X';
that is documented in the help files for the relevant methods.

The help file for the generic usually includes cross-references (under 'See
Also') to some of the available methods.
Alternatively, to find out what classes of objects have a 'pcf' method,
you can type 'methods(pcf)' in R with the package loaded. If you're
reading a PDF, the methods will be listed consecutively in the file.

This would have revealed that there is a 'pcf.ppp' and a couple of other
options.
The help for 'pcf.ppp' says that 'X' should be "a point
pattern (object of class 'ppp')".
The help for 'ppp' or 'ppp.object' says that this class of
object represents a point pattern in the two-dimensional plane.
> But here's a bigger problem.
> The documentation says the pcf function is a generic, but the pcf3est
function isn't a method.
That is correct. But this is not a failure of documentation.
'pcf3est' is not a method for the generic 'pcf'.
'pcf3est' has a different, older syntax, and doesn't behave exactly
the same way internally,
so it is not suitable to be a method for 'pcf'.

It would of course be logical for 'pcf' to have a method for
three-dimensional point patterns,
but that depends on someone implementing it.

The software design for future versions of spatstat specifies that the summary
functions such as
pcf.ppp and pcf3est should be modified to have a common syntax, which will be
slightly different from
either of the current ones, and at that stage, the rewritten pcf3est will become
a method for 'pcf'.
> I found the pcf function via Googling the subject.
I don't want to be rude, but this seems pretty lazy.
> But unless someone goes through a list of all the help topics, they're
unlikely to find the pcf3est function.
Try Googling
         'spatstat pair correlation function three dimensions'

or searching the full spatstat manual PDF for
         'pair correlation function' or 'three dimensions'

You could also have jumped to the 'pcf' entry in the full manual, and
scrolled up and down to see the other entries that begin with 'pcf'.

You complain that the documentation is insufficient, but at the same time, you
complain that the manual is too long (> 1700 pages) and you seem unwilling to
search the documentation or follow cross-references.

Before I answered this email, I had to answer an email from CRAN requesting me
to please cut down the size of the spatstat package, including the documentation
and examples, because the tarball is too large and the examples take too long to
run.

So please understand that package authors have to be economical with the
provision of documentation and examples. It is simply not feasible or efficient
to explain everything in every help file. The user has to make some effort to
follow up relevant information.

Finally, you cast doubt on whether the function pcf3est actually does calculate
an estimate of the pair correlation function for a three-dimensional point
pattern, as we claim in the help file. I'm at a loss to understand what you
mean. The help file gives journal paper references, and indicates that the code
in pcf3est calculates the estimator described in the paper by Baddeley et al
(1993). I am the author of the technical paper, the code, and the documentation.
What exactly are you disputing?


Regards



Prof Adrian Baddeley DSc FAA

John Curtin Distinguished Professor

Department of Mathematics and Statistics

Curtin University, Perth, Western Australia


I work Wednesdays, Thursdays and Fridays


	[[alternative HTML version deleted]]

NOTE CITATIONS USE BRIEF EXCERPTS
> It's a complaint that no-one responded to your query within 24 hours.
Correction, it wasn't my query.
I was replying to someone else's query.
> Finally, you cast doubt on whether the function pcf3est actually does
calculate...
You've taken what I said out of context.
As I said in my response to Rolf, I didn't run the code.
> email from CRAN requesting me to please cut down the size of the spatstat
package
I can't speak on behalf on CRAN.
But reiterating what I said earlier, you'd be better to create smaller
packages, each with a more specific focus.
Each package could have an unofficial co-maintainer to help you out.
> You complain that the documentation is insufficient, but at the same time,
you complain that the manual is too long (> 1700 pages) and you seem
unwilling to search the documentation or follow cross-references.
Did you read my posts...

The functions DO NOT cross-reference.
The pcf generic/methods DO NOT reference the pcf3est function, at all.
The pcf3est function DOES NOT reference the pp3 function, but rather
references the class of the object.

Also, the seealso sections are minimalist.

If you are not going to create smaller packages, then the least you
could do it improve the standard of the seealso sections and
cross-referencing.

Furthermore, if functions are not cross-referenced, the reader (I this
case me) doesn't necessarily know what package those functions are
defined in.
You assume that readers will assume the functions are defined in your
package, and then do a search for them.

Instead of making assumptions about assumptions, just use hyperlinks...
> > I was somewhat disappointed ... not one (after approx 24 hours) had
tried to
> > help answer the OP's question.
> Seriously?
Yes, seriously.
>From my experiences, if people don't reply to an R-help question in
the first 24 hours, chances that they will reply later are low.
Also, I've found a bias on this forum, with few replies to
physics-related questions.

Personally, I like physicists, one of the most under-valued
professions on the planet.
> I found the pcf function via Googling the subject.
> I don't want to be rude, but this seems pretty lazy.
> Try Googling  'spatstat pair correlation function three dimensions'
That sounds like a contradiction.

Also, it's off the mark, given that I was the only one who attempted
to answer the OPs question.

Hi Abby,

Once again I must say your form of communication puzzles me. Do you
believe it benefits the open source community to communicate like this?
What about contacting the package maintainer in this way:

"Hi, I was helping someone else using your package to estimate a pair
correlation function. I was really puzzled that the generel help page
for `pcf` said it was a generic, but it doesn't seem to be generic for
the three dimensional case we needed, and no reference was given from
`pcf` to `pcf3est` which I had to search around to find. Maybe you
should consider throwing in a few cross references. Have a nice day."

I think this all would have played out much nicer then. None of us were
ignoring the original poster. We simply weren't aware of the OPs
problem until Rolf discovered it. I think Adrian and I focus on more
spatstat devoted channels to maximize the benefit of our time.

Regarding splitting the package into smaller pieces the thought has
occurred to us without your helpful suggestion. We started the process
of doing so years ago with the introduction of `spatstat.utils` and
`spatstat.data`. This is well documented on the spatstat GitHub page
referenced in the DESCRIPTION file. However, we have to prioritize our
time and it is quite a complex task to split out the relevant parts in
standalone packages due to all the interdependencies of the existing
code.

On a final note, to give you another example of a way to bring
attention to possible improvements of documentation: Let's say I was
looking at the function `marginal.set` in the package `probhat`, then I
would contact the author and say:

"Hi Abby, I was looking at the help for `marginal.set` and I couldn't
quite figure out what the argument constructor was supposed to be. The
help doesn't mention any class and contains no cross references
whatsoever and searching for 'constructor' in the R help system
didn't
lead anywhere. Maybe a few cross references and actual usage of the
section 'See Also' would be beneficial for new users of your package.
It would be much better if instead of making assumptions about
assumptions you just use hyperlinks. Have a nice day. Kind regards,
Ege"

Where of course I would never use the sentence about hyperlinks in a
real example. I guess at this point I dug myself into the hole of bad
communication which is so killing for good software projects so
apologies for that. I do wish you a good day and hope you will consider
helping us all improve the state of open source software by
constructive suggestions rather than us claiming that each others
packages are unfit for CRAN.

Kind regards,
Ege

On Thu, 2020-04-30 at 06:53 +1200, Abby Spurdle wrote:
> NOTE CITATIONS USE BRIEF EXCERPTS
> 
> > It's a complaint that no-one responded to your query within 24
> > hours.
> 
> Correction, it wasn't my query.
> I was replying to someone else's query.
> 
> > Finally, you cast doubt on whether the function pcf3est actually
> > does calculate...
> 
> You've taken what I said out of context.
> As I said in my response to Rolf, I didn't run the code.
> 
> > email from CRAN requesting me to please cut down the size of the
> > spatstat package
> 
> I can't speak on behalf on CRAN.
> But reiterating what I said earlier, you'd be better to create
> smaller
> packages, each with a more specific focus.
> Each package could have an unofficial co-maintainer to help you out.
> 
> > You complain that the documentation is insufficient, but at the
> > same time, you complain that the manual is too long (> 1700 pages)
> > and you seem unwilling to search the documentation or follow cross-
> > references.
> 
> Did you read my posts...
> 
> The functions DO NOT cross-reference.
> The pcf generic/methods DO NOT reference the pcf3est function, at
> all.
> The pcf3est function DOES NOT reference the pp3 function, but rather
> references the class of the object.
> 
> Also, the seealso sections are minimalist.
> 
> If you are not going to create smaller packages, then the least you
> could do it improve the standard of the seealso sections and
> cross-referencing.
> 
> Furthermore, if functions are not cross-referenced, the reader (I
> this
> case me) doesn't necessarily know what package those functions are
> defined in.
> You assume that readers will assume the functions are defined in your
> package, and then do a search for them.
> 
> Instead of making assumptions about assumptions, just use
> hyperlinks...
> 
> > > I was somewhat disappointed ... not one (after approx 24 hours)
> > > had tried to
> > > help answer the OP's question.
> > 
> > Seriously?
> 
> Yes, seriously.
> From my experiences, if people don't reply to an R-help question in
> the first 24 hours, chances that they will reply later are low.
> Also, I've found a bias on this forum, with few replies to
> physics-related questions.
> 
> Personally, I like physicists, one of the most under-valued
> professions on the planet.
> 
> > I found the pcf function via Googling the subject.
> > I don't want to be rude, but this seems pretty lazy.
> > Try Googling  'spatstat pair correlation function three
dimensions'
> 
> That sounds like a contradiction.
> 
> Also, it's off the mark, given that I was the only one who attempted
> to answer the OPs question.
-- 
Ege Rubak, Associate Professor,
Department of Mathematical Sciences, Aalborg University
Skjernvej 4A, 9220 Aalborg East, Denmark
Phone: (+45)99408861
Mobile: (+45)30230252
Email: rubak at math.aau.dk