A null model analysis debunks widely-cited evidence of dispersal limitation among freshwater diatoms
caught my attention for two reasons. Firstly the language is unusual – I don’t think I have ever attended a presentation that claimed to debunk widely-cited evidence. Secondly, I have a paper on dispersal limitations of freshwater diatoms, that has been reasonably well cited…
As I read the abstract, I soon realise that it is indeed my paper that has been “debunked”.
Are diatoms ubiquitously dispersed? Or, like larger organisms, do diatoms only disperse over limited geographic extents? Unequivocal answers to these questions remain elusive, primarily because direct measurements of diatom dispersal are scarce. In 2006, Telford et al. published a study in Science purporting to provide indirect evidence against the controversial “ubiquitous dispersal hypothesis”. Their article has garnered widespread attention: it has been cited almost 100 times (including articles published within Diatom Research), and citations continue to accumulate rapidly. Here, I use a traditional null model analysis – involving a randomization procedure with the original datasets – to first show that the patterns documented by Telford et al. were expected entirely by chance, and therefore do not constitute evidence of dispersal limitation. Then, drawing upon large calibration datasets from North America and Europe, in conjunction with null model analyses of pH tolerance, I provide preliminary evidence in support of the hypothesis that taxa adapted to the most common pH environments are the least dispersal‐limited.
The author is Jason Pither. It is unfortunate that Pither thought it more important to mention that the “debunked” paper had been cited in Diatom Research than to describe what his “traditional null model” did.
The paper Pither is attacking is a brevia I published in Science with Vigdis Vandvik and John Birks. Using several diatom-pH calibration sets developed for palaeoecology, we noticed that in regions where lakes were predominantly acidic, the diatom species richness optimum was more acidic than in regions where lakes were predominantly alkaline. We realised that we could use this as a test of the ubiquitous dispersal hypothesis, as such a pattern could only readily be explained by metacommunity dynamics which implies that there are dispersal limitations.
Previous (and many subsequent) tests of the ubiquitous dispersal hypothesis have been bedevilled by taxonomic issues (is a taxon endemic, or is it just a synonym), environmental gradients (is a taxon not at a site because of dispersal limitations or because it is outside its niche), and undersampling (is the taxon really here, but too rare to find). Our analysis neatly sidestepped these problems.
Our analysis lacked just one thing, a formal statistical test to demonstrate that the pattern we observed was not just a chance occurrence or an artefact of how the datasets were arranged on a pH gradient. Pither noticed this and, hours before the six month period in which Science will consider comments expired, submitted a comment on this paper. His first version introduced a null model that was so bad the reviewer gave him an example in Excel to show how it could be done better. Better, but still not good. A more appropriate null model showed that the pattern we found was statistically significant.
It is not clear if the latest “debunking” is a new analysis or a reincarnation of Pither’s earlier comment.
This comment of our Science brevia was not the first time I had come across Pither’s work. In 2005 he had published a paper that claimed to find, using a null model analysis, that most diatoms are environmental generalists rather than specialists. This came as a great surprise to those who had been successfully using diatoms as an environmental and palaeoenvironmental indicator for decades. Intrigued, I tested Pither’s methods with some simulated data – could the method recognise a specialist when given one. Hardly. Pither’s method had a very high Type-II error: the null hypothesis that species are generalists was accepted even when it was false. With standard tools like GLM, most of the species in the dataset analysed by Pither were identified as environmental specialists (those that were not included soil-dwelling diatoms washed into the lake). This analysis was written up and became the first comment published in Ecological Letters.
|Null hypothesis (H0) is true||Null hypothesis (H0) is false|
|Reject null hypothesis||Type I error
Generalist classified as specialist
Specialist classified as specialist
|Fail to reject null hypothesis||Correct outcome
Generalist classified as generalist
|Type II error
Specialist classified as generalist
Pither and his coauthor’s reply relied on a claim that whereas for their analysis a Type-I error was to misclassify a generalist as a specialist and a Type-II error was to incorrectly classify a specialist as a generalist, our analysis did the converse. This complete nonsense was apparently suggested by a reviewer! For our GLM analysis, the null hypothesis was, of course, that the species had no relationship with the environment, so a Type-I error was to believe a species was specialised when it was not. The rest of the reply was no better, and did not get to grips with the fundamental problem that when given a simulated specialist species Pither’s methods were typically unable to identify it as a specialist.
So given Pither’s previous luck with null model analysis, I shall not fret about my “debunked” paper tonight.