In the summer of 2003, our first summer in Bergen, Cathy and I took a grand tour of Scandinavia on the way to the Paleolimnology Symposium in Espoo. On the way, we stopped in Abisko, northern Sweden, for a few days to hike through the forests and alpine grassland. It was beautiful, but that is not what I have to write about today.
Larocque and Hall (2003) correlate high-resolution (1–7 years) reconstructions of July air temperature inferred from chironomid assemblages in four lakes near Abisko with instrumental temperature data. These sub-decadal resolution palaeoenvironmental reconstructions are within the scope of a review paper I am writing, I thought I ought to finally have a closer look at them.
The paper reports that
Strong, statistically significant correlations (p <= 0.05) were observed between chironomid-inferred mean July air temperatures and mean meteorological data at all four sites.
The correlations and their associated significance values are given on one of the figures, I’ve put them into a table below.
|Lake||Number of reconstructions||r||p|
Although the text reports that the all the p-values are below 0.05, the figure reports that Lake 850 has a p-value greater than this threshold. However, since Lake 850’s correlation is comparable to those of the other lakes and the number of reconstructed values is high, this must be an error.
It is easy to recalculate the p-value from the correlations and the number of reconstructed values. Using a one-sided test (which is reasonable as we expect a positive correlation), Lake 850 has a p-value below 0.05, but Vuoskkujavri has a p-value of 0.11. The other two lakes have p-values below 0.05.
Intrigued by these discrepancies, and, noticing some other oddities such as Lake 850 having a reconstruction but no assemblage for 1999, I want to recalculate the correlations. I also really want to see a plot of reconstructed against instrumental temperature, not just plots against time. It is time to use xyscan again.
With the digitised data, I can only replicate the reported correlation for one of the lakes. One of the lakes has far worse correlation than reported, and one far better.
Correlations are not the most difficult of statistics to calculate (most people can estimate them fairly well by eye), so if the correlations have been blatantly miscalculated, I am disinclined to trust the reconstructions as these involve more complicated calculations (which the first author has been known to get wrong – see for example the corrigendum to Larocque-Tobler et al (2015). Since the chironomid assemblage data have not yet been archived (hopes spring eternal), I cannot tell whether the reconstructions are correct.
A few weeks ago, I wrote to the first author, advising them of the problems with this and other several papers, suggesting that they should correct or retract non-reproducible papers and archive data for the remainder so that the community can be assured that the results are reproducible. I have had no reply.
Three of the lakes studied by Larocque and Hall are also analysed by Bigler and Hall (2003) who reconstruct July air temperature from diatom assemblages. They do not quantify the agreement between the reconstruction and the instrumental data smoothed with a 13-year running mean, but write in the abstract that the reconstructions “correspond in general closely with the meteorological records”. The results section is still optimistic, but lists several differences. It is not clear from the method section whether the lakes for which reconstructions were made were included in the calibration set. If so, there is a lack of independence between the reconstruction and the observed temperature. Because of the 13-year smooth, this paper falls outside the scope of my review.