Autocorrelation and transfer functions

I forget when I first read Legendre’s (1993) paper “Spatial Autocorrelation: Trouble or New Paradigm?”, but it has greatly influenced my work over the last decade. I realised its relevance to transfer functions when Björn Malmgren visted Bergen from Göteborg to talk about his work Malmgren et al. (2001) using artificial neural networks to reconstruct sea surface temperature from planktonic foraminifera. The neural networks performed well, but not as well as a method called SIMMAX created by Pflaumann et al. (1996). SIMMAX is a variant of the modern analogue technique (MAT), using the dot product as the distance metric (computationally very efficient but not very powerful for identifying good analogues, probably because it works in Euclidean space), and weighting the analogues by their geographic distance from the sample being reconstructed. It was this geographic distance weighting that makes SIMMAX appear to perform so well. Malmgren et al. (2001) attempted to emulate this behaviour by including latitude as a predictor in for the neural networks. I thought it obvious that the SIMMAX algorithm was inappropriate (the test observation is not independent of the training set during cross-validation), so I asked wouldn’t it be better to demonstrate that SIMMAX was inappropriate rather than creating another inappropriate method? I don’t recall being impressed by the answer given, so I quickly set out to write the paper that I thought Malmgren should have written.

This work became Telford et al (2004). I showed that SIMMAX cheated by geographically weighting analogues, and that other methods also cheated: the revised analogue method (RAM) because it failed to cross-validate properly; and MAT if performance and model choice (number of analogues) are determined from the same data. An early version of this paper included a section on spatial autocorrelation, pointing out that the improvement in performance from Imbrie and Kipp’s (1971) 61 observation training set to Pflaumann’s latest training set with over 900 observations could, at least in part, be explained by spatial autocorrelation becoming increasingly important as the density of observations in the Atlantic increased. I argued that this apparent improvement in performance would not necessarily be reflected in improved reconstructions. At this stage, I could offer only these arguments, no demonstration of the problem or solution. My co-author Steve Juggins suggested that I drop that section of the manuscript as it wasn’t really ready, and there was enough material in the rest of the manuscript.

With the rest of the manuscript duly published, I set about trying to demonstrate that spatial autocorrelation was a problem for transfer functions. After several false starts, I managed to demonstrate that all transfer function methods had some skill at reconstructing simulated autocorrelated environmental data using planktonic foraminifera from the North Atlantic, with some methods being much better at reconstructing nonsense than others. The second demonstration was that the transfer functions which performed best when trained and tested on one region of ocean, performed worst when tested on another region. In both cases, local reconstruction methods (MAT, ANN) were more sensitive to autocorrelation, global methods (weighted averaging, maximum likelihood) least. While there may be some problem with regionally endemic taxa in the second test, I found these results sufficiently persuasive, and submitted the manuscript as “The secret assumption of transfer functions”. This paper, Telford and Birks (2005) has now been cited about a hundred times.

It was about this time that the MARGO papers were published. One that particularly interested me was de Vernal et al (2005), which used dinocysts to reconstruct a variety of environmental variables and presented some surprising results such as reconstructed LGM SST being warmer than modern in parts of the Nordic Seas. Since de Vernal et al used MAT , it was inevitable their results were affected by spatial autocorrelation. From reading Barrie Dale’s (2001) work on the ecology of dinocysts, I found the reconstructions dubious. For example, dinocysts are the overwintering stage of dinoflagellates, so seemed unlikely to be directly sensitive to winter conditions. Analysis of the dinocyst training set showed it to largely consist of a set of geographically distinct clusters. The Celtic Sea, for example, had seven dinocyst observations. All found all their five analogues within the Celtic Sea. As the environmental gradients within the Celtic Sea are small, at least compared with the Mediterranean Sea-Arctic Ocean gradient in the training set, the transfer function predicted almost exactly the correct values for all environmental variables for these sites. This does not mean that these variables are ecologically important, the transfer function would have reconstructed geomagnetic field strength just as easily. These and other problems with the dinocyst-based reconstructions were expounded in Telford (2006) (my only single author paper). As a courtesy, I sent a copy of the manuscript to Anne de Vernal when I submitted it under the provocative title “Should dinocyst-salinity transfer functions be taken with a pinch of salt”. It was published as “Limitations of dinoflagellate cyst transfer functions” in QSR.

Having demonstrated the problem of autocorrelation with transfer functions, particularly with the popular MAT, I wanted to find some solutions. Telford & Birks (2009) presents some tests of the importance of spatial autocorrelation for transfer functions that I will discuss in another post. It was written and submitted fairly soon after the previous papers. The reviews were OK, but I wasn’t happy with what I had written so the manuscript resided in the dark recesses of my hard drive together with many other partially written manuscripts. Then I read Fréchette et al (2008). Not only were their pollen-sunshine reconstructions ecologically implausible, simply the result of the autocorrelation, but they included a test that tried to show that autocorrelation was not a problem. Unfortunately the test was badly devised and had precisely zero power for detecting autocorrelation. This stimulated me to finish the revisions of Telford & Birks (2009). Again, as a courtesy, I sent Bianca Fréchette a copy of the manuscript. She passed my manuscript to Joel Guiot who complained that I hadn’t dealt with the disproof of the importance of autocorrelation he had presented in Guiot & de Vernal (2007). I had thought it kinder not to draw attention to the nonsense test he had produced (it tested how well transfer function methods can extrapolate rather than how autocorrelation affected them).

That I thought was the end of my work on autocorrelation and transfer functions. I had nothing more to add. Until one morning, still in bed, up popped Guiot & de Vernal (2011) on Google Reader. It is a dreadful paper, both in content and language, that should never have been published anywhere. That it was published in QSR reflects rather badly on the journal and one has to ask whether an editorial conflict of interests played some part in ensuring its publication. Writing a comment on it was the easiest and quickest manuscript I have ever written. Unfortunately neither our comment nor Guiot and de Vernal’s reply was peer reviewed, permitting them to make further personal attacks and write more nonsense.

Tonight I found a new paper by Fréchette & de Vernal reconstructing sunshine, apparently blissfully unaware of the problems with their methods. They don’t even mention autocorrelation. I find it tragic that these authors, publishing so much research that is probably simply an artefact of autocorrelation work in the same city as Pierre Legendre who has done so much with his group to highlight the problems and potential of autocorrelation and appropriate methods to deal with it.

Advertisements

About richard telford

Ecologist with interests in quantitative methods and palaeoenvironments
This entry was posted in Peer reviewed literature, transfer function. Bookmark the permalink.

One Response to Autocorrelation and transfer functions

  1. Pingback: New paper: forams and sea-level Kemp et al (2013) | Musings on Quantitative Palaeoecology

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s