A DuKE** goes to town at WUWT

A physics lecturer from Duke University, rgbatduke (see DuKE** below) aka Robert G Brown, has been writing for Anthony Watts on WUWT about how he, a humble physics teacher, knows more about climate science and climate modelling than anyone else in the world.  It doesn't seem to bother him that he has never published anything more than a blog article on the topic.  I commented earlier that he was not familiar with the IPCC reports and this little lecture he's giving to climate specialists plainly illustrates he's not, and that he knows probably less than nothing about climate or climate models.

Spaghetti graphs

Here is some of what he wrote initially:

This is reflected in the graphs Monckton publishes above, where the AR5 trend line is the average over all of these models and in spite of the number of contributors the variance of the models is huge. It is also clearly evident if one publishes a “spaghetti graph” of the individual model projections (as Roy Spencer recently did in another thread) — it looks like the frayed end of a rope, not like a coherent spread around some physics supported result.

My comment - those frayed ends of rope represent the noise in the climate.  It's caused by weather as well as differences between the models.  Weather has the properties of chaos.  Climate is all about boundaries that mark expected weather ranges and extremes.  Climate change is all about trends.  I don't think there are any spaghetti charts in the IPCC report, but I could be wrong.

Mean, standard deviation and variance

Now back to rgbatduke.  Take note of the bold section, we'll come back to that later:

Note the implicit swindle in this graph — by forming a mean and standard deviation over model projections and then using the mean as a “most likely” projection and the variance as representative of the range of the error, one is treating the differences between the models as if they are uncorrelated random variates causing >deviation around a true mean!.

Say what?...

...What I’m trying to say is that the variance and mean of the “ensemble” of models is completely meaningless, statistically because the inputs do not possess the most basic properties required for a meaningful interpretation. They are not independent, their differences are not based on a random distribution of errors, there is no reason whatsoever to believe that the errors or differences are unbiased (given that the only way humans can generate unbiased anything is through the use of e.g. dice or other objectively random instruments).

The DuKE doesn't like chaos - too messy

You'll probably groan reading this next bit.  Rgbatduke seems to want a nice, neat straight line chart with no chaotic properties.  Look at how he proposed to get it:

...First of all, we could stop pretending that “ensemble” mean and variance have any meaning whatsoever bynot computing them. Why compute a number that has no meaning? Second, we could take the actual climate record from some “epoch starting point” — one that does not matter in the long run, and we’ll have to continue the comparison for the long run because in any short run from any starting point noise of a variety of sorts will obscure systematic errors — and we can just compare reality to the models. We can then sort out the models by putting (say) all but the top five or so into a “failed” bin and stop including them in any sort of analysis or policy decisioning whatsoever unless or until they start to actually agree with reality.

Modellers - pick the winners then sit around and wait for 30 years or so ...  

Then real scientists might contemplate sitting down with those five winners and meditate upon what makes them winners — what makes them come out the closest to reality — and see if they could figure out ways of making them work even better. For example, if they are egregiously high and diverging from the empirical data, one might consider adding previously omitted physics, semi-empirical or heuristic corrections, or adjusting input parameters to improve the fit.

Then comes the hard part. Waiting. ...So one has to wait and see if one’s model, adjusted and improved to better fit the past up to the present, actually has any predictive value....

My comment: I can't really see all the scientists sitting around for thirty years fiddling their thumbs while they wait to see how well their top five winners worked out.  Whether they actually had any predictive value.

...It would take me, in my comparative ignorance, around five minutes to throw out all but the best 10% of the GCMs (which are still diverging from the empirical data, but arguably are well within the expected fluctuation range on the DATA side), sort the remainder into top-half models that should probably be kept around and possibly improved, and bottom half models whose continued use I would defund as a waste of time. That wouldn’t make them actually disappear, of course, only mothball them. If the future climate ever magically popped back up to agree with them, it is a matter of a few seconds to retrieve them from the archives and put them back into use.

It's warmed by magic

The above is more evidence that he's talking through his hat.  But there's more.  He's now attributing the warming "since the LIA" to magic from the look of things - no forcing required.  Is climate just a bouncing ball?

Of course if one does this, the GCM predicted climate sensitivity plunges from the totally statistically fraudulent 2.5 C/century to a far more plausible and still possibly wrong ~1 C/century, which — surprise — more or less continues the post-LIA warming trend with a small possible anthropogenic contribution. This large a change would bring out pitchforks and torches as people realize just how badly they’ve been used by a small group of scientists and politicians, how much they are the victims of indefensible abuse of statistics to average in the terrible with the merely poor as if they are all equally likely to be true with randomly distributed differences.

Compare what the DuKE** wrote with what really happens

Sorry, got a bit carried away with his nonsense.  Let's get back to basics.  What rgbatduke is saying up front is that the IPCC reports "a mean and standard deviation over model projections and then using the mean as a “most likely” projection and the variance as representative of the range of the error."

We saw in my previous article that the mean is not necessarily presented as the "most likely" projection.


In the comments, here is what he changed it to, sans links:

Second, to address Nick Stokes in particular (again) and put it on the record in this discussion as well, the AR4 Summary for Policy Makers doesexactly what I discuss above. Figure 1.4 in the unpublished AR5 appears poised to do exactly the same thing once again, turn an average of ensemble results, and standard deviations of the ensemble average into explicit predictions for policy makers regarding probable ranges of warming under various emission scenarios.

We've already seen that he's wrong about the AR4 Summary for Policy Makers.  I posted the chart in my previous article, but if you want to check for yourself, go here.  The model ensemble means are shown but the "best estimate" for each scenario at 2100 is not the model mean.  And the ranges aren't simple standard deviations or variance. As for "poised to do" - well AR5 isn't out yet.  However, the caption to figure 11.33 in the draft was provided at WUWT and once again it shows rgbatduke is wrong, it demonstrates that standard deviations of the ensemble average are NOT used for predictions at all, let alone explicit predictions.

This is not a matter of discussion about whether it is Monckton who is at fault for computing an R-value or p-value from the mish-mosh of climate results and comparing the result to the actual climate — this is, actually, wrong and yes, it is wrong for the same reasons I discuss above, because there is no reason to think that the central limit theorem and by inheritance the error function or other normal-derived estimates of probability will have the slightest relevance to any of the climate models, let alone all of them together. One can at best take any given GCM run and compare it to the actual data, or take an ensemble of Monte Carlo inputs and develop many runs and look at the spread of results and compare THAT to the actual data.

Does he seriously think that climate modellers don't hindcast?  Don't refine the models?

Here is the most relevant chart from IPCC AR4 Working Group I:

Figure 10.4. Multi-model means of surface warming (relative to 1980–1999) for the scenarios A2, A1B and B1, shown as continuations of the 20th-century simulation. Values beyond 2100 are for the stabilisation scenarios (see Section 10.7). Linear trends from the corresponding control runs have been removed from these time series. Lines show the multi-model means, shading denotes the ±1 standard deviation range of individual model annual means. Discontinuities between different periods have no physical meaning and are caused by the fact that the number of models that have run a given scenario is different for each period and scenario, as indicated by the coloured numbers given for each period and scenario at the bottom of the panel. For the same reason, uncertainty across scenarios should not be interpreted from this figure (see Section 10.5.4.6 for uncertainty estimates).

Here the comparison again:

rgbatduke version 1 he talks of the variance and the mean of the ensemble:

the variance and mean of the “ensemble” of models is completely meaningless, statistically

rgbatduke version 2, so now he shifts to looser terminology, talking average not mean, but talks of standard deviation of the ensemble average

an average of ensemble results, and standard deviations of the ensemble average into explicit predictions

What the IPCC actually did previously (link)

Lines show the multi-model means, shading denotes the ±1 standard deviation range of individual model annual means.

So - the IPCC figures don't show what rgbatduke said they do.  The IPCC projections show multi-model means with the +/-1 standard deviation range of individual model annual means, NOT as rgbatduke wrote, standard deviation of the ensemble average.  

Moreover the report cautions against using the above chart to interpret uncertainty.  WGI has a separate section discussing and quantifying uncertainty / likely ranges and a box discussing equilibrium climate sensitivity (where they use the mode as the best estimate).  In addition there is a section describing the projected global temperature with probability ranges, which are not a simple standard deviation or variance from the mean. There is a very complicated box diagram showing the mean, the likely ranges and the ranges using different models and different approaches to uncertainty.  If rgbatduke had bothered to glance at the IPCC report he might have seen that.  

I don't usually nitpick like this, but rgbatduke is so vocal and rude in his pronouncements that I figure he deserves it.

Means are more accurate, biases cancel out, means reduce noise

More from the IPCC:

Following the TAR, means across the multi-model ensemble are used to illustrate representative changes. Means are able to simulate the contemporary climate more accurately than individual models, due to biases tending to compensate each other (Phillips and Gleckler, 2006). It is anticipated that this holds for changes in climate also (Chapter 9). ...The use of means has the additional advantage of reducing the ‘noise’ associated with internal or unforced variability in the simulations. Models are equally weighted here, but other options are noted in Section 10.5.

Now I don't know if they will be taking the same approach in AR5.  There has been more work on models and projections since 2007.  For example in this 2010 paper, Knutti et al write:

An average of models compares better to observations than a single model, but the correlation between biases among CMIP3 GCMs makes the averaging less effective at canceling errors than one would assume. For present-day surface temperature, for example, a large fraction of the biases would remain even for an infinite number of models of the same quality. Extreme biases tend to disappear less quickly than smaller biases. Thus, models are dependent and share biases, and the assumption of independence made in some studies is likely to lead to overconfidence, if the uncertainty is measured by the standard error of the ensemble means (inversely proportional to the square root of the ensemble size). Quantitative methods to combine models and to estimate uncertainty are still in their infancy....

...The overconfidence achieved by improper weighting may well be more damaging than the loss of information by equal weighting or no aggregation at all. As long as there is no consensus on how to properly produce probabilistic projections, the published methods should be used to explore the consequences arising from different specifications of uncertainty....

...However, there is some danger of not sampling the extreme ends of the plausible range with a few cases...

There will be hell to pay

I don't think any climate scientist will be quivering in her high-heeled shoes after reading this little rant from rgbatduke.  Or maybe she will, from laughter.

I make this point to put the writers of the Summary for Policy Makers for AR5 that if they repeat the egregious error made in AR4 and make any claims whatsoever for the predictive power of the spaghetti snarl of GCM computations, if they use the terms “mean and standard deviation” of an ensemble of GCM predictions, if they attempt to transform those terms into some sort of statement of probability of various future outcomes for the climate based on the collective behavior of the GCMs, there will be hell to pay, because GCM results are not iid samples drawn from a fixed distribution, thereby fail to satisfy the elementary axioms of statistics and render both mean behavior and standard deviation of mean behavior over the “space” of perturbations of model types and input data utterly meaningless as far as having any sort of theory-supported predictive force in the real world. Literally meaningless. Without meaning.

So 'literally meaningless' = 'without meaning'.  Luckily he translates that for us or we'd never have been able to figure out what he meant. Wait, there's more:

If any of the individuals who helped to actually write this summary would like to come forward and explain in detail how they derived the probability ranges that make it so easy for the policy makers to understand how likely to certain it is that we are en route to catastrophe, they should feel free to do so.

Why call for "individuals" on a crappy blog like WUWT?  What climate modeller is going to visit there or read his pontificating.  Far better to go and look for himself.  Maybe he could start with the IPCC report, and read the notation under the charts that show uncertainty.  Or he could visit HotWhopper because I've added the links to the papers themselves:

The 5 to 95% ranges (vertical lines) and medians (circles) are shown from probabilistic methods (Wigley and Raper, 2001; Stott and Kettleborough, 2002; Knutti et al., 2003; Furrer et al., 2007; Harris et al., 2006; Stott et al., 2006b).

And a couple more since then to get him started, and it looks as if there are more papers focusing on regional projections, not just global projections now.

Knutti et al (2010) Challenges in Combining Projections from Multiple Climate Models (that I quoted from above).

Mikhail A. Semenov, Pierre Stratonovitch (2010) Use of multi-model ensembles from global climate models for assessment of climate change impacts

CMIP5 portal

Suggestions for rbgatduke

My article probably has some errors too.  Please excuse me, I'm not a climate scientist or a climate modeller.  I'm going to quit here though I feel I should double check what I've written, I'll let people do that in the comments if they want to.  For a little snark blog I've spent way too much time on this :D   But if you think this article is long, you should see all the bits of rbgatdukes two rants that I didn't include - here and here.

For now, I'll just make some final suggestions for the DuKE from Duke.

1. Learn how to do a literature search

Someone ought to show rgbatduke where the library is and maybe a kind librarian will show him how to use one of the various scholarly search engines.  If he still has trouble he could always ask someone to show him how to use Google Scholar like non-academic bloggers do.

2. Read the latest IPCC report

There are good sections on climate, weather, climate models and all sorts of related information that might help him avoid looking like a goose next time he lectures specialists outside his own field.

3. Read, read, read and observe

Read up on climate science wherever he can.  Tell him to visit realclimate.org and read the archives.  Suggest he not pipe up with a comment until he learns something about climate or with his style and attitude he'll be given short shrift and his comments confined to the bore hole.  If he finds realclimate too sciency, suggest skepticalscience.com.  There is a heap of information that even a physics teacher might understand.  And loads of references if they aren't beyond his capability.

4. Stick to teaching physics

If he finds the above too much to cope with, politely suggest that he stick to teaching physics and leave the research to the experts.

Footnote:

WM Briggs has written some stuff about rgbatduke's rant.  I don't think Dr Brown will be too pleased.  h/t Nick Stokes.

---

**DuKE = collective noun for a group of deniers