I’ve been searching for robust evidence of the influence of solar variability on Earth’s climate. Zhao & Wang (2014) promise it in a paper I found at the Club du Soleil. They are looking for a correlation between annual sunspot number and the mean latitude of the East Asian monsoon rainband during the summer half of the year.
More than that, they are looking for the time period with the highest correlation between the latitude of the monsoon rainband and sunspot number over the 55 -year period 1958-2012. They do this by finding the correlation of sunspot number with the mean latitude of the rainband calculated over different period of time between 1st April and 30th September, beginning with all single days, then all pairs of adjacent days, then …, and finally the entire 183 day period.
They find that the 53-day period between the 22nd May and 13th July has the highest correlation with sunspot number with a coefficient of 0.47 (>99.9% confidence level).
To find this maximum correlation, the authors had to calculate 16 836 correlations (=183 + 182 + 181 + … +1). They write this four times. How many times do they write how they dealt with the multiple testing? You’ve guessed it: zero. They do not discuss how calculating 16 836 correlations might just bias the chance of getting a high correlation.
Of course these 16 836 correlations are not independent tests – the correlation for 22nd May and 13th July will be very similar to that for 23rd May and 13th July and so on – so a Bonferroni correction would be extremely conservative.
We can get a hint of how excited we should be about a correlation of 0.47 with a simple simulation. I’ve simulated 183 days of white noise for 55 years and then calculated the correlation of different periods within these 183 days with 55 years of sunspots. I’ve done this 100 times (calculating 16 836 x 100 correlations upsets my computer!). The maximum correlation exceeds 0.47 in 15% of trials.
This test is imperfect: in reality the latitude of the rainband is probably autocorrelated from day-to-day. This will reduce the effective number of tests and make correlations that exceed 0.47 rarer. Conversely, there is probably some year-to-year autocorrelation caused by, for example, El Nino and the Pacific Decadal Oscillation, which will reduce the effective number of degrees of freedom in the tests and enhance the chance of having a correlation that exceeds 0.47.
I don’t know which effect will win out, but without considering the effects of multiple testing, Zhao & Wang (2014) cannot claim to have robust evidence the effect of solar variability on climate.
Liang Zhao and Jing-Song Wang, 2014: Robust Response of the East Asian Monsoon Rainband to Solar Variability. J. Climate, 27, 3043–3051. doi: http://dx.doi.org/10.1175/JCLI-D-13-00482.1