Anthony Watts has reported a poster (archived here). He doesn't comment on it. I doubt he understands it. Most of the people commenting at WUWT don't understand it either. And I don't blame them.
Now I'm no Tamino, however in my view, what Pat Michaels and Chip Knappenberger have done, ostensibly, is flawed from the outset and demonstrates that they don't understand the CMIP5 climate models.
Rather than show a chart with observations vs models (combined), what they've done is show a chart of trend changes. That's fine, but further down I'll explain why I believe the conclusions they draw are flawed.
Now they don't provide their data, only the poster. So you haven't got a lot to go on. But here's a copy of the chart. You can see a summary on the AGU14 website here. They didn't upload their poster to the AGU website for some reason, but they did make it available on the CATO website here. Most people who've an interest in climate can probably figure out what they've done and where they went wrong. (Click the chart to enlarge it.)
|The annual average global surface temperatures from 108 individual CMIP5 climate model runs forced with historical (+ RCP45 since 2006) forcings were obtained from the KNMI Climate Explorer website. Linear trends were computed through the global temperatures from each run, ending in 2014 and beginning each year from 1951 through 2005. The trends for each period (ranging in length from 10 to 64 years) were averaged across all model runs (black dots). The range containing 90 percent (thin black lines), and 95 percent (dotted black lines) of trends from the 108 model runs is indicated. The observed linear trends for the same periods were calculated from the annual average global surface temperature record compiled by the U.K. Hadley Center (HadCRUT4) (colored dots) (the value for 2014 was the 10-mon, January through October, average). Observed trend values which were less than or equal to the 2.5th percentile of the model trend distribution were colored red; observed trend values which were between the 2.5th and the 5th percentile of the model trend distribution were colored yellow; and observed trend values greater than the 5th percentile of the model trend distribution were colored green.|
Source: Michaels and Knappenberger, CATO Institute
What they've shown is interesting in a way, but their conclusions are invalid in my opinion. What they've plotted is:
- The trends of multi-model mean of CMIP5 RCP45 runs with trends from 64 years to 10 years, up to 2014 (that is, a 64 year trend from 1951 to 2014 through to a ten year trend from 2005 to 2014)
- 5 and 95 percentiles, which Pat Michaels describes as "the error bars are based upon the spread of the model results".
- A plot with green, yellow and red circles, purportedly representing where the trend of HadCRUT4 observations are consistent with the trend of the multi-model mean.
In other words, they've done a Tisdale. (Bob Tisdale likes plotting derivatives.) When you read their conclusions you'll see that they've assumed that the trend of the models should always match the trend of observations, even over a period as short as ten years. What they don't seem to allow for is that the CMIP5 models are climate models, not weather models. Therefore the trend for shorter periods will be expected to differ from observations.
I say that because the internal variability - multi-year patterns like the PDO and the IPO (and ENSO), will show up at different times in different models and won't be timed to coincide with what happens. Random chance dictates that the internal variability in some models will coincide with observations some of the time, but that's just up to chance. Much of the time when a single model shows a short term hike in temperature observations may show a short term dip. Taking a mean of 108 models will cancel out much of the internal variability in the models. The observations on the other hand, will include internal variability. So for example, variability between models, such as from the different phases of the PDO, will generally be cancelled out by averaging the models. The observations will be affected by the phase of the PDO. It would be expected that the observed trend in the cool phase would be less than the trend in a warm phase.
Now extend the time of the trend beyond, say, 40 years and that will mean you can start to compare trends, if you want to. That's because most multidecadal variability would be smoothed out in the longer term trendline of observations. It's already mostly been smoothed out in the models by the very fact of averaging the 108 models.
In short, when you plot trends for periods less than around 30 or 40 years, you can expect the model mean to be different from observations. That means that what Pat'n Chip found was what is expected.
Remember Pat'n Chip plotted trends in temperature change for periods from 10 years long to 64 years long. What they ended up with is in the above chart. In the curve with the colours, the green circles show that the trend of observations is consistent with the trend of the models. That applies for anything longer than 40 years. Exactly what you'd expect.
When it comes to shorter periods for the trend. That is, the trend over 10 years, 20 years, 30 years etc, you'd expect that the observations would be different from the multi-model mean. And that's what Pat'n Chip found.
So far so good. Where I believe Pat'n Chip went wrong is they jumped to the wrong conclusion. They put up these two charts:
What they concluded was that the models are "wrong". In my view, all they've shown is that twenty years is probably not long enough to show the climate trend. You can see that in the thirty year trend, the observations are closer to the model mean. They don't show a similar chart for longer periods, but what they found you can see up top. As the trend line extends over longer periods, the model mean trends are closer to observations.
To illustrate what I mean, I've plotted HadCRUT4 with a CMIP5 multi-model mean below. I've used RCP8.5, but it doesn't make much difference to their RCP4.5, because they track each other fairly well to the middle of this century. Click the chart to enlarge it.
|Sources: Hadley Centre and KNMI Climate Explorer (CMIP5)|
You can see that CMIP5 mean jumps up above HadCRUT4 from around 2005, which affects the slope. What you can also figure is that the 20 year trend line is not as good a fit as the 40 year trend line is. The R^2 for the 40 year trendline is 0.829. For the 20 year trendline it's only 0.37. (I know the R^2 isn't the only or best way to determine how well the trendline fits. The point is that if you want to determine the long term trend, you're better off with more than twenty years of data.)
If you enlarge the above chart, you can also see what I mean by the multi-model mean (lighter blue) smoothing out internal variability. The dips are volcanoes AFAIK (eg Krakatoa 1883, Agung 1963, Pinatubo 1991 etc), so they'll all be synchronised because the forcing is built into all the models. But other than that, the CMIP5 curve is much smoother than the observations. [Added by Sou a bit later.]
Now look at the CMIP5 trend. I've only put in the 40 year trendline. But it's not all that different from the forty year trendline for HadCRUT4. Yes, it's different - mainly because of the fact that recent temperatures have been lower than those modeled. And that's because the CMIP5 data only has observed forcings up until 2006. The forcings beyond that have turned out to be lower than those observed in regard to solar radiation in particular (a positive forcing), and probably higher volcanic aerosol forcings (which are negative) as well. Not only that, but the PDO has been in a cool phase, which has dampened global surface temperatures.
There's just one more oddity. You'll notice the shortest trends - around 10 years, are marked in "green" as good. Yet when you look at the observations, the most recent observations are quite a bit below the models. Now actuals are different to trends, I know. Still, surely that points again to the analysis being a bit wonky.
That's just my take for what it's worth. Feel free to disagree in the comments.
For interest, here are the forty year trends to 2006, the year the models were plugged with actual forcings, after which they have only estimated forcings. [Added a bit later - Sou]
|Sources: Hadley Centre and KNMI Climate Explorer (CMIP5)|
From the WUWT comments
I don't think too many people at WUWT could figure out what was done, and I didn't see anyone who came to the same conclusion as me.
RoHa parroted what he was told to parrot, like an obedient trained parrot (with apologies to parrots, who are very friendly birds and much smarter than your average WUWT-er, particularly when it comes to climate and weather):
December 19, 2014 at 2:43 pm
So the models don’t actually work? They just stand there looking pretty? Shame we can’t have any photos to make our own observations of them.
Lots of people got all huffy about not being able to take photos of posters. I didn't see anyone point out that the AGU and many of the scientists (but not Pat'n Chip) made their posters available for downloading. David Schofield whined, in ignorance:
December 19, 2014 at 8:54 am
If people put up a poster they are openly publicising it. Don’t understand why you can’t photo them.
Others jumped in and said it was standard at scientific and technical conferences to not allow photographs.
Was n.n was talking about Pat'n Chip's prediction of an ECS of 2 degrees or something else? You can decide.
December 19, 2014 at 9:14 am
The system is incompletely or insufficiently characterized and unwieldy, which ensures our perception will remain limited to correlation between cause and effect. It’s like the scientific consensus has lost touch with the scientific domain. They really should resist urges to dabble in predictions.
JeffC reckons science is pointless and it's all too hard. Thinking about it probably makes his head hurt.
December 19, 2014 at 11:33 am
why ? its silly and arrogant to think we can accurately model a chaotic system … its a waste of time and money …
Robert of Ottawa wrote:
December 19, 2014 at 10:29 am
That graph is really stretching it – 2.5th and 97.5th between .55 and MINUS 0.1 C/decade.
To which average joe replied:
December 19, 2014 at 12:29 pm
Robert – that’s because at the far right the trend is taken over a period of only 10 years, the shorter the time period of the trendline, the noisier the signal. The wide error bars at the right threw me too until I gave it some thought, then it made sense.
Resourceguy reckons certain words are not permitted in science:
December 19, 2014 at 10:37 am
From the abstract, the word “unfortunately” is inappropriate for professional statistical presentations. It is either acceptance or rejection of the null and there is no unfortunate this or that to it. Period
michaelspj (Pat Michaels) explains the multi-model mean and the error bars:
December 19, 2014 at 5:40 pm
Its the average of each of the IPCC’s 108 2014 ensemble models. And the error bars are based upon the spread of the model results. If you download our entire poster you will see that they are normally distributed and therefore we can do very straightforward tests on the mean output versus “reality
Jake J can't follow it, and I'm not surprised. It's not an easy chart for a layperson to decipher. I'm not certain I got it right either.
December 19, 2014 at 11:32 am
Terrible chart. Too many lines, impossible for a non-specialist to interpret or use.
rgbatduke wrote a very long comment as usual, of which I'll just post the first two paragraphs:
December 19, 2014 at 11:58 am
Interesting, flawed, and curious. Interesting because it quantifies to some extent the observation that the climate models “collectively” fail a hypothesis test. Flawed because it once again in some sense assumes that the mean and standard deviation of an “ensemble” of non-independent climate models have some statistical meaning, and they do not. Even as a meta-analysis, it is insufficient to reject “the models of CMIP5″, only the use of the mean and variance of the models of CMIP5 as a possibly useful predictor of the climate. But we didn’t need a test for that, not really. The use of this mean as a predictor is literally indefensible in the theory of statistics without making assumptions too egregious for anyone sane to swallow.
What we (sadly) do not see here is the 105 CMIP5 model results individually compared to the data. This would reveal that the “envelope” being constructed above is a collective joke. It’s not as if 5 models of the 105 are very close to the actual data at the lower 5% boundary — it is that all of the models spend 5 percent of their time that low, but in different places. Almost none of the models would pass even the most elementary of hypothesis tests compared to the data as they have the wrong mean, the wrong variance, the wrong autocorrelation compared to the actual climate. Presenting them collectively provides one with the illusion that the real climate is inside some sort of performance envelope, but that is just nonsense.
Tom quibbles with some of the conclusions of Pat'n Chip, which really and truly are very farfetched, not just for the reasons that Tom gives.
December 19, 2014 at 3:09 pm
Quoting from the poster:
“From the recent literature, the central estimate of the equilibrium climate sensitivity is ~2°C, while the climate model average is ~3.2°C, or an equilibrium climate sensitivity that is some 40% lower than the model average.
….., it means that the projections of future climate change given by both the IPCC and NCA are, by default, some 40% too large (too rapid) and the associated (and described) impacts are gross overestimates.”
I buy that 2 is about 40% lower than 3.2 degrees, but it does not work in reverse. 3.2 is 60% higher than the appropriate estimate of ECS, not 40%.
Eli Rabett from Rabett Run sneaks in under the mods' radar and makes a comment, but not about the silly poster from Pat'n Chip. It's about the reason for disallowing photographs.
December 19, 2014 at 4:08 pm
AGU has no copyright on any poster. What they are getting at is that posters and presentations occupy a peculiar netherworld in scientific publishing, being often preliminary in nature and unpublished in the dead electron or tree world. Sometimes, the good times, people use posters to provoke discussion, scientific that is. Thus, when someone photographs a poster and puts it up for everyone to see problems can arise.
An amusing, well not at the time, to the authors example of this was an encounter Eli had with Bruce Malamud and Don Turcotte at AGU. Turns out that they had given a seminar at York University, and somebunny put up the powerpoints, which was picked up by Tim Curtin, then Eli. Malamud and Turcotte had, in their own words, no idea that it was out there, and indeed it took then another couple of years to complete the study. Take a look at the link, and the links at the link to Marohasy and Curtin and the comments at both places.
That's all I will bother with for now. You can check the archive for other comments by Pat Michaels, commenting as michaelspj.