Mathias Trachsel and I have a paper in open discussion at Climate of the Past Discussions developing methods for estimating the performace of transfer functions in spatially autocorrelated environments. The paper began life as a blog post a couple of years ago which proposed the methods. The paper uses lots of simulated species data, with known spatial properties, to test how well the methods work.
Abstract. Conventional cross-validation schemes for assessing transfer-function performance assume that observations are independent. In spatially-structured environments this assumption is violated, resulting in over-optimistic estimates of transfer-function performance. H block cross-validation, where all samples within h km of the test samples are omitted is a method for obtaining unbiased transfer function performance estimates. In this study, we assess three methods for determining the optimal h. Using simulated data, we find that all three methods result in comparable values of h. Applying the three methods to published transfer functions, we find they yield similar values for h. Some transfer functions perform notably worse when h block cross-validation is used.
We think the methods perform fairly well – certainly we would recommend using them rather than ignoring spatial autocorrelation in strongly spatially structured calibration sets. Examples of how to run the tests will be in a vignette in the next version of the palaeoSig R package.
Trachsel, M. and Telford, R. J.: Technical Note: Estimating unbiased transfer-function performances in spatially structured environments, Clim. Past Discuss., 11, 4729-4749, doi:10.5194/cpd-11-4729-2015, 2015.