Keenan’s test with added drift

Doug Keenan visited to defend his methods. His main arguments were argument from vicarious authority and argument by condescension, but he also left this:

“The ARIMA model is specified to be driftless: this means that the drift (i.e. average of the first differences of the series) in the model is 0. Your claim that differencing the series removes a trend makes no sense: if there was a trend, the drift would obviously not be 0.”

The mean of the first differences is 0.005 °C/yr. Obviously not zero, but Keenan argues that “inferences are not drawn directly from the data. Rather, a statistical model is fit to the data and then inferences are drawn from the model.” Fair enough, so I’m going to add a linear trend to the HadCrut4 data to determine how much extra linear trend is required before the linear trend model has a lower AIC than the ARIMA(3,1,0) model.

res<-sapply(0:50, function(n){
  y<-had$temp+seq(0,n,length=nrow(had))#add linear trend to data
  mod1<-lm(y~had$year)
  mod2<-arima(y, order=c(3,1,0))
  c(AIC(mod1), AIC(mod2))
})
x11(4,4);par(mar=c(3,3,1,1), mgp=c(1.5,.5,0))
matplot(0:50, t(res), type="l", lty=1, col=c(4,2), ylab="AIC", xlab="Added temperature change 1850-2013")
legend("bottomright", legend=c("Linear trend", "ARIMA"), lty=1, col=c(4,2), bty="n")

AIC of linear trend and ARIMA(3,1,0) models with added temperature gradient.

However much linear trend is added to the data the AIC of the ARIMA(3,1,0) model is always below that of the linear model. Even when we know that drift is non-zero because we added huge amount of it, Keenan’s test would still show that the ARIMA(3,1,0) model was better than the linear model. This is absurd. Keenan’s test has no power. Either it is hopelessly sensitive to non-linearities in the data, or as Gavin suggested, the AIC of the linear model and the ARIMA model are not directly compatible. Whatever the explanation is, the test is not fit for purpose.

HadCrut4 data with an extra 50°C increase. Still not significant according to Keenan’s methods.

Testing Doug Keenan’s methods

Doug Keenan’s allegation that he has a confession from the Met Office that “statistically significant temperature rise can’t be supported” has been devoured with relish by the pigeons.

But elsewhere, Lucia has argued that

Doug is going on about the fact that a statistical model treating the of trendless data with ARIMA noise with d=1 appears to fit the data better linear trend+ ARIMA with d=0. It probably does so but that means very little because:
1) Physically no one expects the AGW forcings would have caused the trend to look like “straight line + noise” since 18whatever.
2) ARIMA with d=1 alone would violate the first law of thermo. (i.e. violates the 1st law of thermo. We don’t even need to get fancy and go to the 2nd.)

I want to briefly look at her first argument. I’m going to generate some simulated data that is linear trend + noise and test whether the linear trend model or Keenan’s ARIMA(3,1,0) has the lower Akaike information criterion (AIC). Lower AIC’s indicate a better model.

x<-1:100
y<-rnorm(length(x), x, 10)#linear
plot(x,y)

AIC(lm(y~x))
AIC(arima(y, order=c(3,1,0)))

With the particular set of random data I used, the linear trend model had an AIC of 750, far below the ARIMA model’s AIC of 774. The linear trend model is the better model.

Now let’s try making the relationship slightly curved by adding a quadratic to the trend.

x<-1:100
y<-rnorm(length(x), x+0.02*x^2, 10)#slight curve
plot(x,y)

AIC(lm(y~x))
AIC(arima(y, order=c(3,1,0)))

Now the linear model has an AIC of 877, far above the ARIMA model’s AIC of 809.

In this second test, the ARIMA(3,1,0) appears to be a far better model according to the AIC. Is the simulated data drawn from an ARIMA(3,1,0) process? No. Are the coefficients of the ARIMA model interpretable? No.

This demonstrates that Keenan’s test is very sensitive to deviations from the linear trend in the temperature record. That the global temperature has not had a linear trend over the instrumental period is not in the least unexpected as the climate forcing has not had a linear trend. It would seem a mistake to reject a linear trend model that nobody thinks is perfect in favour of an ARIMA(3,1,0) process that violates the first law of thermodynamics. Keenan should remember that AIC can only indicate the best model, from a purely statistical point of view, of those tested — there may be a much better but untested model.

Red Herrings, Cats and Pigeons at WUWT

Hold the front page, Doug Keenan has a confession from the Met Office that “statistically significant temperature rise can’t be supported”.

In a long post, more concerned with the details of which minister would not answer which parliamentary question than the statistics, Keenan gloats over extracting from the Met Office the admission that an ARIMA(3,1,0) model explains global temperature change in the instrumental record better than a linear trend with autocorrelated (AR1) residuals, and hence he declares that the rise in temperatures is not significant.

Keenan assures the reader that “unfamiliarity with the model does not matter here”. I would demur, model choice on purely statistical criteria is a empty pursuit of meaningless models. If we do not understand what the models are doing, we cannot evaluate if the models are sensible.

The linear trend model fits a regression to the data, but rather than assuming that the residuals from this regression are independent, they are expected to be autocorrelated. That is neighbouring residuals are expected to be more similar than residuals selected at random.

Keenan would have us replace this model with ARIMA(3,1,0), an autoregressive integrated moving average model. The 3 designates the number of terms in an autoregressive model. This is not dissimilar to the first model that expected the residuals to be from a first order autoregressive model. More interesting is the second number, 1, which indicated the number of times that the data must be differenced (subtracting the temperature of the previous year from the temperature of the current year) to make the data stationary, ie trendless.

Yes, Keenan argues there is no trend in the data by using a method that removes trends from data. By demonstrating that a differencing is needed, Keenan demonstrates there is a trend in the data. Whether this trend is accounted for by a linear trend or a differencing operation is a choice. Neither carry much physical meaning.

So lets have a look at applying his ARIMA(3,1,0) model to the Met Office data. First we import and plot the data and the differenced data.

had<-read.table("http://www.metoffice.gov.uk/hadobs/hadcrut4/data/current/time_series/HadCRUT.4.2.0.0.annual_ns_avg.txt")[,1:2]
names(had)<-c("year", "temperature")

x11(6,4);par(mgp=c(1.5,.5,0), mar=c(3,3,1,1), mfrow=c(1,2))
plot(had, type="l")
plot(diff(had$temp), type="l")

Global temperatures (left) and differenced global temperatures (right)

The strong trend in the left-hand plot is obvious. The right-hand plot shows no trend — the differencing operation has removed it — and the data lack strong autocorrelation. An ACF plot of the differenced data confirms this, there is weak negative autocorrelation for two or three lags.

ACF of differenced global temperature data.

We can then fit the ARIMA(3,1,0) model to the raw data, or equivalently, an ARIMA(3,0,0) model to the differenced data.

acf(diff(had$temp))

mod1<-arima(had$temp,order=c(3,1,0))
mod2<-arima(diff(had$temp),order=c(3,0,0), include.mean=F)

The three AR coefficients are all small and negative (-0.38,-0.37, and -0.27), and their physical meaning is not obvious.

We can test if Keenan’s model is a plausible representation of climate by simulating a Holocene-length time series. The Holocene is known to have a rather stable climate, can this model simulate that?

sims<-replicate(5,{
  arima.sim(list(order=c(3,1,0), ar=coef(mod1)),10000, sd=sqrt(mod1$sigma2))
})
matplot(sims, xlab="year", ylab="temperature anomaly °C", type="l", col=1:5, lty=1)

Five Holocene-length realisations of an ARIMA(3,1,0) process

That I think is a NO! In these simulations, there are up to 15°C of global temperature change, rather larger than the actual ~1°C change in the Holocene. All the realisations have more temperature change than the glacial-interglacial temperature difference. But perhaps the climate is usually stationary, maybe from an ARIMA(3,0,0) process, and the non-stationarity that needs differencing only occurs during the last 150 years. What could possibly have caused this non-stationarity, this trend during the last 150 years? It couldn’t be greenhouse gases could it?

Solar-salt marsh signal: Review of Di Rita 2013

This is part of my critical review of the palaeoenvironmental evidence for the influence of solar activity on climate.

Di Rita (2013) A possible solar pacemaker for Holocene fluctuations of a salt-marsh in southern Italy. Quaternary International, 288, 239–248

Di Rita (2013) investigates the pollen assemblages in a sediment core from a coastal wetland next to the Adriatic Sea. The paper reports large fluctuations in pollen percentages of the salt marsh indicators Salicornia and Ruppia maritima and uses wavelet analysis to find periodicities of 130 and 260 years, which are consistent with solar forcing. Further, the minima in the Salicornia record coincide with minima in the GISP2 ¹⁰Be record.

This paper has been cited by fake climate sceptics as evidence that the IPCC is embroiled in a vast conspiracy to ignore solar influence on climate.

Salicornia europaea

It is not immediately obvious why one would a priori select the percentages of salt-marsh herb pollen as a proxy likely to be sensitive to solar activity (Helianthus annuus, I would understand). Reading the introduction, it turns out that the proxy was not selected a priori, but that the cycles were first observed in the proxy, and the cycles were then tested for possible solar influence. A major problem with a post hoc analysis like this is that the Type I error rate — the probability of incorrectly rejecting the null hypothesis — is greatly inflated: random but interesting looking patterns are much more likely to appear statistically significant than other random data. This alone should make us cautious about accepting the conclusions of Di Rita (2013). Post hoc analyses are far from the only doubt-inducing issue with Di Rita (2013).

Di Rita (2013) uses wavelet analysis on the Salicornia pollen curve. Wavelet analysis is a useful tool to visualise the time and frequency distribution of cycles in the data. The data are interpolated to ten year intervals, greatly in excess of the actual data resolution. This will greatly increase the apparent number of degrees of freedom available, making cycles appear more significant than they are, and induce strong temporal autocorrelation in the data. It is not discussed how these issues will affect the wavelet analysis, the reader is not even told if the 95% significance level in the wavelet plot is based on a red noise or a white noise null model. Nor is the reader told which software was used. The wavelet analysis is not applied to the entire Salicornia record, nor even the whole period when the wetland was a lagoon, instead it is applied to a shorter section where “fluctuations are especially pronounced” between 4800 BP and 6350 BP (the base of the core). Regular might have been a more appropriate word than pronounced, as the fluctuations after 4800 BP are as large as those before.

So the wavelet results are from a post hoc analysis of a carefully selected section of a selected pollen profile using methods that might be prone to exaggerate the evidence for strong cyclicity. All this could be ignored and I would still be sceptical about the significance of solar-frequencies in the pollen data. When discussing Chen et al (2011; 2013) I argued that an 11 year cycle in noisy proxy data cannot be considered evidence of solar forcing of climate as such cycles are not evident in analyses of long instrumental records. The same argument cannot be used against the longer cycles found by Di Rita (2013), the instrumental record is just not long enough to detect periods this long. Instead the problem is that there are too many possible solar frequencies. From Di Rita (2013):

Ogurtsov et al. (2002) demonstrated that the DeVries–Suess cycle shows a variation with a period of 170–260 years, while the Gleissberg cycle is even more complex presenting a wide frequency band with a double structure that consists in 50–80 year and 90–140 year periodicities.

The majority (80%) of periods between 50 and 260 years fall within either the DeVries–Suess or Gleissberg frequency bands, so the majority of proxy cycles, whether real or otherwise, whether solar-forced or otherwise, will appear consistent with solar frequencies. As cycles of any cause are likely to have a frequency within the solar frequency bands, finding a cycle within these bands in scant evidence for solar forcing.

To make a spectral analysis more credible, we need to demonstrate that the proxy not only has the right frequencies, but that these are in phase with solar forcing. Di Rita (2013) attempts this, finding that the minima in the pollen profiles match minima in the ¹⁰Be GISP2 record within 50 years. But the ¹⁰Be record is spiky: there are many minima between 6350 and 4000 BP, a minority of these coincide with minima in the pollen record. Maxima in the pollen data match the main maxima in the ¹⁰Be data rather poorly. No null model is attempted to demonstrate that the observed minima matching is better than expected by chance, and no consideration is given to the chronological uncertainty in both records (indeed the presence of the 260 year cycle is used in Di Rita et al ( 2011) as evidence that the age-depth model is appropriate).

Di Rita (2013) Fig. 5. The ¹⁰Be GISP2 dataset is compared with the CN3 salt-marsh indicators (Salicornia type and Ruppia maritima) on the basis of their respective independent chronologies. Black dots represent the minima that can be visually correlated.

A salt marsh is a dynamic environment. Over the 2300 years investigated by Di Rita (2013), sediment accumulated (four metres in Lago Salso), sea level rose by about 2 m and tectonic and isostatic subsidence lowered the land by about a metre, the bar separating the lagoon from the Adriatic would have been more open at some times that others, affecting the salinity and tidal regime of the lagoon. We are asked to believe that Salicornia and Ruppia ignored all this and other noise, to focus on the Suess and Gleissberg cycles. If such a noisy environment could reliably record solar cycles, then we should expect solar cycles in proxies with less non-climatic noise to be as clear as a bell. They are not.

Conclusions

Di Rita (2013) contains no credible evidence of a solar climate link.

But even if Salicornia cannot reconstruct solar activity, there is at least one thing that it can be used for, even if it doesn’t taste very nice.

More solar-dinocyst correlations in the Eastern Mediterranean: Review of Chen et al 2013.

This is part of my critical review of the palaeoenvironmental evidence for the influence of solar activity on climate.

Chen et al (2013) Paleoclimate of the Southern Adriatic Sea region during the ‘Medieval Climate Anomaly’ reflected by organic walled dinoflagellate cysts. The Holocene 23: 645-655.

I found Chen et al (2013) via Maarten Blaauw’s Club du Soleil, which lists papers relevant to solar forcing of climate. Having concluded that the previous paper by Chen et al (2011) was not credible evidence of a solar-climate relationship, I thought I would look at their new paper.

Chen et al (2011) covers the period 60 BC–AD 200 from a core off the south coast of Italy. Chen et al (2013) covers the Medieval Climate Anomaly, AD 990–AD 1200. An interesting publication strategy: take one core and publish two-three hundred-year segments of the proxy data, rinse and repeat the site description and methods and add the new results. I look forward to the other ten papers they will be able to write from their 3600 year long core. I particularly look forward to the one covering the last 300 years, so the comparison with the instrumental record can validate, or otherwise, their dinocyst indices.

Like Chen et al (2011), the more recent paper calculates indices based on the dinocyst assemblages in their core and subjects these data to spectral analysis and plots the proxies with other proxy data for comparison. The spectral analysis is simplified in Chen et al (2013), using only REDFIT, and finds cycles at 11.4 and 26 years in one proxy and at about 9 and 13 years in the other two. The 11.4 year cycle in the dinocyst temperature index is described as “strong support” for a solar activity-climate relationship. I demur. In the absence of a strong, clear, and widespread 11-year cycle in the instrumental record that can be related to solar activity climate, it is folly to believe that an 11-year cycle in noisy palaeoenvironmental data is solar driven. The alternative hypotheses, that it is chance or some internal cycle, are much more plausible.

The other line of evidence in Chen et al (2013) in an alleged correlation between the Δ¹⁴C anomalies and the dinocyst temperature index. The close link is determined by eyeballing the graph. The correlation coefficient is not calculated; autocorrelation is not accounted for; no significance is estimated.

Chen et al 2013 Figure 5. Dinocyst temperature index (grey) and Δ¹⁴C anomaly (black).

The correlation in the raw data looks weak. The correlation in the smoothed data looks stronger, but of course has fewer degrees of freedom. I doubt that the relationship is statistically significant, and that is before chronological uncertainties are considered.

Chen et al (2013) are aware of the possible impact of chronological uncertainties: “… the sediments of the core section could have been sedimented about 180 years earlier or later compared with our assumption.”. So 180 years of potential error in a section of core that is 210 years long. This calls into doubt even the sign of the correlation Chen et al (2013) eyeball.

Conclusion

The sediment core is the same in Chen et al (2013) and Chen et al (2011), the methods are the same, and my conclusions are the same: this paper contains no credible evidence of a solar-climate relationship.

Solar-Rhine ice link melts under scrutiny

This is part of my critical review of the palaeoenvironmental evidence for the influence of solar activity on climate.

Sirocko et al. (2012) Solar influence on winter severity in central Europe. Geophysical Research Letters, 39, L16704.

Sirocko et al. find that severe winters in Central Europe, identified by documentary evidence of the Rhine freezing, tend to occur during sunspot minima.

Via Stoat, I find news at Klimazwiebel and Simple Climate, that van Oldenborgh et al. (2013) [open access] have published a comment on Sirocko et al. The comment is published in Environmental Research Letters because, shamefully, Geophysical Research Letters does not publish comments.

Briefly, van Oldenborgh et al. (2013) express surprise that there is a strong relationship between sun-spot numbers and the Rhine freezing as the relationship is not detectable above weather noise in long instrumental series of temperature. They then show that there is evidence that the Rhine froze on more years than reported by Sirocko et al., whose data were not a homogeneous times series, but compiled from diverse sources including poems, postcards and paintings. If these extra years are included in the analysis, the relationship breaks down. There is more: read the paper or Klimazwiebel/Simple Climate for details.

I was interested by a quote from Pittock that literature reporting solar-climate links is full of common errors, such as ‘poor data quality, data selection, data smoothing and autocorrelations, and post hoc elaboration of hypotheses to explain discrepancies’. Pittock first expressed this view in 1978 in a review of 140 papers purporting to find a solar-weather relationship. It is a view he reiterates in 1983 and 2009. I suspect that these problems will be common in palaeoenvironomental solar-climate studies.

Solar-dinocyst correlations in the Eastern Mediterranean: Review of Chen et al. 2011.

This is part one of my critical review of the palaeoenvironmental evidence for the influence of solar activity on climate.

Chen et al. 2011 Short term climate variability during “Roman Classical Period” in the eastern Mediterranean. Quaternary Science Reviews, 30, 3880–3891.

This paper was cited by Engels and van Geel (2012) as evidence of a solar-climate relationship. Chen et al. (2011) use the dinocyst record from a core from just outside the Adriatic Sea, off the Italian coast, to reconstruct climatic conditions during the “Roman Classical Period” (60 BC–AD 200). Spectral analysis of their results show 7-8 and 11 year cyclicities, which they suggest relate to variability of the North Atlantic Oscillation and make a link to the 11 year sunspot cycle. Chen et al. (2011) also report strong correlations between their reconstruction and Δ¹⁴C anomalies, a proxy for solar activity, and also with global volcanic activity.

Rather than use (dubious) transfer functions, Chen et al. (2011) make qualitative climate reconstructions from their dinocyst assemblages using several indices. For example, temperature patterns are estimated from the ratio of warm to warm plus cold indicating taxa. No information on the predictive skill of this index is given. I have no doubt that this index would have utility in determining that polar oceans are colder than the Mediterranean Sea, but I am doubtful about its utility for the relatively small changes expected during the Roman Classical period.

The spectral analysis in Chen et al. (2011) is done using redfit (which uses Lomb-Scargle Fourier transform – ideal for data that are not evenly spaced) and the multi-taper method (MTM). At least they write that they do, yet the figure is labelled MEM, which would suggest maximum entropy method. I have not used MTM or MEM, so cannot comment on them. Chen et al. (2011) also use wavelets.

Spectral (Redfit and MEM) and wavelet power of Accumulation rates of indicators of A) Adriatic Surface water (ASW) indicators, B) nutrient indicators, C) and the temperature index W/C.

The redfit spectral only have a significant (at the 95% level) peak at 11 years for one of the three indices. Significance levels are not shown the for MTM (or MEM) results (unless the redfit significance levels also apply to the MTM results), but two of the three indices show an 11 year peak. The wavelet analysis shows that this 11 year cycle does not occur throughout the record. This spectral analysis is weak evidence of an 11 year cycle that could be attributed to solar activity.

Chen et al. (2011) present a figure showing their dinocyst-temperature index and Δ¹⁴C anomalies. The correlation looks very impressive except for the period after AD 160, when the two curves diverge, however the correlation is not calculated (and would have to consider the autocorrelation in both records).

Chen et al. Figure 8 A. W/C ratio in the “Roman Classical Period”, thick line represents 5 point running average, 20th century mean value is in horizontal dash dot line, and comparison with global Δ14C anomalies (dotted line). B. Worldwide volcano eruptions with explosive intensity (VEI). Thick curve indicates Vesuvius eruption at 79 and 172 AD.

When viewing this figure, we need to remember that the calibrated radiocarbon dates (raw dates not given) which form the basis of the chronology have a 2-sigma uncertainty of up to 195 years. The chronological uncertainty is almost as large as the period being examined is long – the sample dated to 60 BC could actually date to AD 100 and the correlation would look very different! This means that the strong positive correlation between dinocyst-temperature index and Δ¹⁴C anomalies may be entirely fortuitous, a negative correlation is perfectly plausible given these data.

What is needed is an analysis that takes account of the chronological uncertainty when calculating the correlation between the dinocyst-temperature index and Δ¹⁴C anomalies. One procedure is to generate a family of age-depth models and calculate the correlation for each. If most of the age-depth models yield a positive correlation, the correlation is robust to chronological uncertainty. I did an analysis relating grebes and chironomids using this method.

Because the chronological uncertainty is so large, this correlation between dinocyst-temperature index and Δ¹⁴C anomalies is not useful evidence of a solar-climate relationship. Accepting this correlation as evidence risks a huge publication bias – if the dates had been slightly different and the correlation was lost, who would have published this as evidence against a solar-climate relationship?

Conclusion
This paper is not credible evidence of a solar-climate link.