A physics lecturer from Duke University, rgbatduke (see DuKE** below) aka Robert G Brown, has been writing for Anthony Watts on WUWT about how he, a humble physics teacher, knows more about climate science and climate modelling than anyone else in the world. It doesn't seem to bother him that he has never published anything more than a blog article on the topic. I commented earlier that he was not familiar with the IPCC reports and this little lecture he's giving to climate specialists plainly illustrates he's not, and that he knows probably less than nothing about climate or climate models.
Spaghetti graphsHere is some of what he wrote initially:
This is reflected in the graphs Monckton publishes above, where the AR5 trend line is the average over all of these models and in spite of the number of contributors the variance of the models is huge. It is also clearly evident if one publishes a “spaghetti graph” of the individual model projections (as Roy Spencer recently did in another thread) — it looks like the frayed end of a rope, not like a coherent spread around some physics supported result.My comment - those frayed ends of rope represent the noise in the climate. It's caused by weather as well as differences between the models. Weather has the properties of chaos. Climate is all about boundaries that mark expected weather ranges and extremes. Climate change is all about trends. I don't think there are any spaghetti charts in the IPCC report, but I could be wrong.
Mean, standard deviation and varianceNow back to rgbatduke. Take note of the bold section, we'll come back to that later:
Note the implicit swindle in this graph — by forming a mean and standard deviation over model projections and then using the mean as a “most likely” projection and the variance as representative of the range of the error, one is treating the differences between the models as if they are uncorrelated random variates causing >deviation around a true mean!.
...What I’m trying to say is that the variance and mean of the “ensemble” of models is completely meaningless, statistically because the inputs do not possess the most basic properties required for a meaningful interpretation. They are not independent, their differences are not based on a random distribution of errors, there is no reason whatsoever to believe that the errors or differences are unbiased (given that the only way humans can generate unbiased anything is through the use of e.g. dice or other objectively random instruments).
The DuKE doesn't like chaos - too messy
You'll probably groan reading this next bit. Rgbatduke seems to want a nice, neat straight line chart with no chaotic properties. Look at how he proposed to get it:
...First of all, we could stop pretending that “ensemble” mean and variance have any meaning whatsoever bynot computing them. Why compute a number that has no meaning? Second, we could take the actual climate record from some “epoch starting point” — one that does not matter in the long run, and we’ll have to continue the comparison for the long run because in any short run from any starting point noise of a variety of sorts will obscure systematic errors — and we can just compare reality to the models. We can then sort out the models by putting (say) all but the top five or so into a “failed” bin and stop including them in any sort of analysis or policy decisioning whatsoever unless or until they start to actually agree with reality.
Modellers - pick the winners then sit around and wait for 30 years or so ...
Then real scientists might contemplate sitting down with those five winners and meditate upon what makes them winners — what makes them come out the closest to reality — and see if they could figure out ways of making them work even better. For example, if they are egregiously high and diverging from the empirical data, one might consider adding previously omitted physics, semi-empirical or heuristic corrections, or adjusting input parameters to improve the fit.
Then comes the hard part. Waiting. ...So one has to wait and see if one’s model, adjusted and improved to better fit the past up to the present, actually has any predictive value....My comment: I can't really see all the scientists sitting around for thirty years fiddling their thumbs while they wait to see how well their top five winners worked out. Whether they actually had any predictive value.
...It would take me, in my comparative ignorance, around five minutes to throw out all but the best 10% of the GCMs (which are still diverging from the empirical data, but arguably are well within the expected fluctuation range on the DATA side), sort the remainder into top-half models that should probably be kept around and possibly improved, and bottom half models whose continued use I would defund as a waste of time. That wouldn’t make them actually disappear, of course, only mothball them. If the future climate ever magically popped back up to agree with them, it is a matter of a few seconds to retrieve them from the archives and put them back into use.
It's warmed by magicThe above is more evidence that he's talking through his hat. But there's more. He's now attributing the warming "since the LIA" to magic from the look of things - no forcing required. Is climate just a bouncing ball?
Of course if one does this, the GCM predicted climate sensitivity plunges from the totally statistically fraudulent 2.5 C/century to a far more plausible and still possibly wrong ~1 C/century, which — surprise — more or less continues the post-LIA warming trend with a small possible anthropogenic contribution. This large a change would bring out pitchforks and torches as people realize just how badly they’ve been used by a small group of scientists and politicians, how much they are the victims of indefensible abuse of statistics to average in the terrible with the merely poor as if they are all equally likely to be true with randomly distributed differences.
Compare what the DuKE** wrote with what really happensSorry, got a bit carried away with his nonsense. Let's get back to basics. What rgbatduke is saying up front is that the IPCC reports "a mean and standard deviation over model projections and then using the mean as a “most likely” projection and the variance as representative of the range of the error."
We saw in my previous article that the mean is not necessarily presented as the "most likely" projection.
In the comments, here is what he changed it to, sans links:
Second, to address Nick Stokes in particular (again) and put it on the record in this discussion as well, the AR4 Summary for Policy Makers doesexactly what I discuss above. Figure 1.4 in the unpublished AR5 appears poised to do exactly the same thing once again, turn an average of ensemble results, and standard deviations of the ensemble average into explicit predictions for policy makers regarding probable ranges of warming under various emission scenarios.We've already seen that he's wrong about the AR4 Summary for Policy Makers. I posted the chart in my previous article, but if you want to check for yourself, go here. The model ensemble means are shown but the "best estimate" for each scenario at 2100 is not the model mean. And the ranges aren't simple standard deviations or variance. As for "poised to do" - well AR5 isn't out yet. However, the caption to figure 11.33 in the draft was provided at WUWT and once again it shows rgbatduke is wrong, it demonstrates that standard deviations of the ensemble average are NOT used for predictions at all, let alone explicit predictions.
This is not a matter of discussion about whether it is Monckton who is at fault for computing an R-value or p-value from the mish-mosh of climate results and comparing the result to the actual climate — this is, actually, wrong and yes, it is wrong for the same reasons I discuss above, because there is no reason to think that the central limit theorem and by inheritance the error function or other normal-derived estimates of probability will have the slightest relevance to any of the climate models, let alone all of them together. One can at best take any given GCM run and compare it to the actual data, or take an ensemble of Monte Carlo inputs and develop many runs and look at the spread of results and compare THAT to the actual data.Does he seriously think that climate modellers don't hindcast? Don't refine the models?
Here is the most relevant chart from IPCC AR4 Working Group I:
|Figure 10.4. Multi-model means of surface warming (relative to 1980–1999) for the scenarios A2, A1B and B1, shown as continuations of the 20th-century simulation. Values beyond 2100 are for the stabilisation scenarios (see Section 10.7). Linear trends from the corresponding control runs have been removed from these time series. Lines show the multi-model means, shading denotes the ±1 standard deviation range of individual model annual means. Discontinuities between different periods have no physical meaning and are caused by the fact that the number of models that have run a given scenario is different for each period and scenario, as indicated by the coloured numbers given for each period and scenario at the bottom of the panel. For the same reason, uncertainty across scenarios should not be interpreted from this figure (see Section 10.5.4.6 for uncertainty estimates).|
the variance and mean of the “ensemble” of models is completely meaningless, statistically
an average of ensemble results, and standard deviations of the ensemble average into explicit predictions
What the IPCC actually did previously (link)
Lines show the multi-model means, shading denotes the ±1 standard deviation range of individual model annual means.
Means are more accurate, biases cancel out, means reduce noise
Following the TAR, means across the multi-model ensemble are used to illustrate representative changes. Means are able to simulate the contemporary climate more accurately than individual models, due to biases tending to compensate each other (Phillips and Gleckler, 2006). It is anticipated that this holds for changes in climate also (Chapter 9). ...The use of means has the additional advantage of reducing the ‘noise’ associated with internal or unforced variability in the simulations. Models are equally weighted here, but other options are noted in Section 10.5.
An average of models compares better to observations than a single model, but the correlation between biases among CMIP3 GCMs makes the averaging less effective at canceling errors than one would assume. For present-day surface temperature, for example, a large fraction of the biases would remain even for an inﬁnite number of models of the same quality. Extreme biases tend to disappear less quickly than smaller biases. Thus, models are dependent and share biases, and the assumption of independence made in some studies is likely to lead to overconﬁdence, if the uncertainty is measured by the standard error of the ensemble means (inversely proportional to the square root of the ensemble size). Quantitative methods to combine models and to estimate uncertainty are still in their infancy....
...The overconﬁdence achieved by improper weighting may well be more damaging than the loss of information by equal weighting or no aggregation at all. As long as there is no consensus on how to properly produce probabilistic projections, the published methods should be used to explore the consequences arising from different speciﬁcations of uncertainty....
...However, there is some danger of not sampling the extreme ends of the plausible range with a few cases...
There will be hell to pay
I make this point to put the writers of the Summary for Policy Makers for AR5 that if they repeat the egregious error made in AR4 and make any claims whatsoever for the predictive power of the spaghetti snarl of GCM computations, if they use the terms “mean and standard deviation” of an ensemble of GCM predictions, if they attempt to transform those terms into some sort of statement of probability of various future outcomes for the climate based on the collective behavior of the GCMs, there will be hell to pay, because GCM results are not iid samples drawn from a fixed distribution, thereby fail to satisfy the elementary axioms of statistics and render both mean behavior and standard deviation of mean behavior over the “space” of perturbations of model types and input data utterly meaningless as far as having any sort of theory-supported predictive force in the real world. Literally meaningless. Without meaning.So 'literally meaningless' = 'without meaning'. Luckily he translates that for us or we'd never have been able to figure out what he meant. Wait, there's more:
If any of the individuals who helped to actually write this summary would like to come forward and explain in detail how they derived the probability ranges that make it so easy for the policy makers to understand how likely to certain it is that we are en route to catastrophe, they should feel free to do so.Why call for "individuals" on a crappy blog like WUWT? What climate modeller is going to visit there or read his pontificating. Far better to go and look for himself. Maybe he could start with the IPCC report, and read the notation under the charts that show uncertainty. Or he could visit HotWhopper because I've added the links to the papers themselves:
The 5 to 95% ranges (vertical lines) and medians (circles) are shown from probabilistic methods (Wigley and Raper, 2001; Stott and Kettleborough, 2002; Knutti et al., 2003; Furrer et al., 2007; Harris et al., 2006; Stott et al., 2006b).
And a couple more since then to get him started, and it looks as if there are more papers focusing on regional projections, not just global projections now.
Knutti et al (2010) Challenges in Combining Projections from Multiple Climate Models (that I quoted from above).
Suggestions for rbgatduke
For now, I'll just make some final suggestions for the DuKE from Duke.