A physics lecturer from Duke University, rgbatduke (see DuKE** below) aka Robert G Brown, has been writing for Anthony Watts on WUWT about how he, a humble physics teacher, knows more about climate science and climate modelling than anyone else in the world. It doesn't seem to bother him that he has never published anything more than a blog article on the topic. I commented earlier that he was not familiar with the IPCC reports and this little lecture he's giving to climate specialists plainly illustrates he's not, and that he knows probably less than nothing about climate or climate models.

#### Spaghetti graphs

Here is some of what he wrote initially:This is reflected in the graphs Monckton publishes above, where the AR5 trend line is the average over all of these models and in spite of the number of contributors the variance of the models is huge. It is also clearly evident if one publishes a “spaghetti graph” of the individual model projections (as Roy Spencer recently did in another thread) —it looks like the frayed end of a rope, not like a coherent spread around some physics supported result.

**My comment**- those frayed ends of rope represent the noise in the climate. It's caused by weather as well as differences between the models. Weather has the properties of chaos.

**is all about boundaries that mark expected weather ranges and extremes. Climate**

*Climate***is all about trends. I don't think there are any spaghetti charts in the IPCC report, but I could be wrong.**

*change*#### Mean, standard deviation and variance

Now back to rgbatduke. Take note of the bold section, we'll come back to that later:Note the implicit swindle in this graph —by forming a mean and standard deviation over model projections and then using the mean as a “most likely” projectionand thevariance as representative of the range of the error, one is treating the differences between the models as if they are uncorrelated random variates causing >deviation around a true mean!.

Say what?...

...What I’m trying to say is that the variance and mean of the “ensemble” of models is completely meaningless, statistically because the inputs do not possess the most basic properties required for a meaningful interpretation. They are not independent, their differences are not based on a random distribution of errors, there is no reason whatsoever to believe that the errors or differences are unbiased (given that the only way humans can generate unbiased anything is through the use of e.g. dice or other objectively random instruments).

#### The DuKE doesn't like chaos - too messy

You'll probably groan reading this next bit. Rgbatduke seems to want a nice, neat straight line chart with no chaotic properties. Look at how he proposed to get it:

...First of all, we could stop pretending that “ensemble” mean and variance have any meaning whatsoever bynot computing them. Why compute a number that has no meaning? Second, we could take the actual climate record from some “epoch starting point” — one that does not matter in the long run, and we’ll have to continue the comparison for the long run because in any short run from any starting point noise of a variety of sorts will obscure systematic errors — andwe can just compare reality to the models.We can then sort out the models by putting (say) all but the top five or so into a “failed” binand stop including them in any sort of analysis or policy decisioning whatsoever unless or until they start to actually agree with reality.

#### Modellers - pick the winners then sit around and wait for 30 years or so ...

Thenscientists might contemplate sitting down with those five winners andrealwhat makes them winners — what makes them come out the closest to reality — and see if they could figure out ways of making them work even better. For example, if they are egregiously high and diverging from the empirical data,meditate upon, semi-empirical or heuristic corrections, or adjusting input parameters to improve the fit.one might consider adding previously omitted physics

Then comes the hard part. Waiting. ...So one has to wait and see if one’s model, adjusted and improved to better fit the past up to the present, actually has any predictive value....

**My comment:**I can't really see all the scientists sitting around for thirty years fiddling their thumbs while they wait to see how well their top five winners worked out. Whether they actually had any predictive value.

...It would take me, in my comparative ignorance, around five minutes to throw out all but the best 10% of the GCMs(which are still diverging from the empirical data, but arguably are well within the expected fluctuation range on the DATA side), sort the remainder into top-half models that should probably be kept around and possibly improved, and bottom half modelswhose continued use I would defund as a waste of time. That wouldn’t make them actually disappear, of course,only mothball them. If the future climate ever magically popped back up to agree with them, it is a matter of a few seconds to retrieve them from the archives and put them back into use.

#### It's warmed by magic

The above is more evidence that he's talking through his hat. But there's more. He's now attributing the warming "since the LIA" to magic from the look of things - no forcing required. Is climate just a bouncing ball?Of course if one does this, the GCM predicted climate sensitivity plunges from the totallystatistically fraudulent2.5 C/century to a farmore plausible and still possibly wrong~1 C/century, which — surprise — more or lesscontinues the post-LIA warming trendwith a small possible anthropogenic contribution. This large a change would bring out pitchforks and torches as people realize just how badly they’ve been used by a small group of scientists and politicians, how much they are thevictims of indefensible abuse of statistics to average in the terrible with the merely pooras if they are all equally likely to be true with randomly distributed differences.

#### Compare what the DuKE** wrote with what really happens

Sorry, got a bit carried away with his nonsense. Let's get back to basics. What rgbatduke is saying up front is that the IPCC reports "**a mean and standard deviation over model projections and then using the mean as a “most likely” projection**and the

**variance as representative of the range of the error**."

We saw in my previous article that the

**mean**is not necessarily presented as the "most likely" projection.

In the comments, here is what he changed it to, sans links:

Second, to address Nick Stokes in particular (again) and put it on the record in this discussion as well, the AR4 Summary for Policy Makers doesexactly what I discuss above. Figure 1.4 in the unpublished AR5 appears poised to do exactly the same thing once again,We've already seen that he's wrong about the AR4 Summary for Policy Makers. I posted the chart in my previous article, but if you want to check for yourself, go here. The model ensemble means are shown but the "best estimate" for each scenario at 2100 is not the model mean. And the ranges aren't simple standard deviations or variance. As for "poised to do" - well AR5 isn't out yet. However, the caption to figure 11.33 in the draft was provided at WUWT and once again it shows rgbatduke isturn an average of ensemble results, and standard deviations of the ensemble average into explicit predictionsfor policy makers regarding probable ranges of warming under various emission scenarios.

**, it demonstrates that**

*wrong***standard deviations of the ensemble average**are

**used for predictions at all, let alone explicit predictions.**

*NOT*This is not a matter of discussion about whether it is Monckton who is at fault for computing an R-value or p-value from the mish-mosh of climate results and comparing the result to the actual climate — this is, actually, wrong and yes, it is wrong for the same reasons I discuss above, because there is no reason to think that the central limit theorem and by inheritance the error function or other normal-derived estimates of probability will have the slightest relevance to any of the climate models, let alone all of them together. One can at best take any given GCM run and compare it to the actual data, or take an ensemble of Monte Carlo inputs and develop many runs and look at the spread of results and compare THAT to the actual data.Does he seriously think that climate modellers don't hindcast? Don't refine the models?

Here is the most relevant chart from IPCC AR4 Working Group I:

Figure 10.4. Multi-model means of surface warming (relative to 1980–1999) for the scenarios A2, A1B and B1, shown as continuations of the 20th-century simulation. Values beyond 2100 are for the stabilisation scenarios (see Section 10.7). Linear trends from the corresponding control runs have been removed from these time series. Lines show the multi-model means, shading denotes the ±1 standard deviation range of individual model annual means. Discontinuities between different periods have no physical meaning and are caused by the fact that the number of models that have run a given scenario is different for each period and scenario, as indicated by the coloured numbers given for each period and scenario at the bottom of the panel. For the same reason, uncertainty across scenarios should not be interpreted from this figure (see Section 10.5.4.6 for uncertainty estimates). |

**rgbatduke version 1**he talks of the

**and the**

*variance**mean*of the

*ensemble*:

the variance and mean of the “ensemble” of models is completely meaningless, statistically

**rgbatduke version 2**, so now he shifts to looser terminology, talking

**not**

*average***mean,**but talks of

**standard deviation of the ensemble average**

an average of ensemble results, and standard deviations of the ensemble average into explicit predictions

**What the IPCC actually did previously (link)**

Lines show themulti-model means, shading denotes the ±1 standard deviation range ofindividual modelannual means.

**of individual model annual means,**NOT as rgbatduke wrote,

*standard deviation of the**ensemble average.*

**the report cautions against using the above chart to interpret uncertainty.**

*Moreover**WGI has a separate section discussing and quantifying uncertainty / likely ranges and a box discussing equilibrium climate sensitivity (where they use the mode as the best estimate). In addition there is a section describing the projected global temperature with probability ranges, which are not a simple standard deviation or variance from the mean. There is a very complicated box diagram showing the mean, the likely ranges and the ranges using different models and different approaches to uncertainty. If rgbatduke had bothered to glance at the IPCC report he might have seen that.*

#### Means are more accurate, biases cancel out, means reduce noise

Following the TAR, means across the multi-model ensemble are used to illustrate representative changes.Means are able to simulate the contemporary climate more accurately than individual models, due to biases tending to compensate each other(Phillips and Gleckler, 2006). It is anticipated that this holds for changes in climate also (Chapter 9). ...The use of meanshas the additional advantage of reducing the ‘noise’associated with internal or unforced variability in the simulations. Models are equally weighted here, but other options are noted in Section 10.5.

An average of models compares better to observations than a single model, but the correlation between biases among CMIP3 GCMs makes theaveraging less effective at canceling errors than one would assume. For present-day surface temperature, for example, a large fraction of the biases would remain even for an inﬁnite number of models of the same quality. Extreme biases tend to disappear less quickly than smaller biases. Thus, models are dependent and share biases, and theassumption of independence made in some studies is likely to lead to overconﬁdence, if the uncertainty is measured by the standard error of the ensemble means(inversely proportional to the square root of the ensemble size). Quantitative methods to combine models and to estimate uncertainty are still in their infancy....

...The overconﬁdence achieved by improper weighting may well be more damaging than the loss of information by equal weighting or no aggregation at all. As long as there is no consensus on how to properly produce probabilistic projections, the published methods should be used to explore the consequences arising from different speciﬁcations of uncertainty....

...However, there is some danger of not sampling the extreme ends of the plausible range with a few cases...

### There will be hell to pay

I make this point to put the writers of the Summary for Policy Makers for AR5 that if they repeat the egregious error made in AR4 and make any claims whatsoever for the predictive power of the spaghetti snarl of GCM computations, if they use the terms “mean and standard deviation” of an ensemble of GCM predictions, if they attempt to transform those terms into some sort of statement of probability of various future outcomes for the climate based on the collective behavior of the GCMs,So 'literally meaningless' = 'without meaning'. Luckily he translates that for us or we'd never have been able to figure out what he meant. Wait, there's more:, because GCM results are not iid samples drawn from a fixed distribution, thereby fail to satisfy the elementary axioms of statistics and render both mean behavior and standard deviation of mean behavior over the “space” of perturbations of model types and input data utterly meaningless as far as having any sort of theory-supported predictive force in the real world. Literally meaningless. Without meaning.there will be hell to pay

If any of the individuals who helped to actually write this summary would like to come forward and explain in detailWhy call for "individuals" on a crappy blog like WUWT? What climate modeller is going to visit there or read his pontificating. Far better to go and look for himself. Maybe he could start with the IPCC report, and read the notation under the charts that show uncertainty. Or he could visit HotWhopper because I've added the links to the papers themselves:that make it so easy for the policy makers to understandhow they derived the probability ranges, they should feel free to do so.how likely to certain it is that we are en route to catastrophe

The 5 to 95% ranges (vertical lines) and medians (circles) are shown from probabilistic methods (Wigley and Raper, 2001; Stott and Kettleborough, 2002; Knutti et al., 2003; Furrer et al., 2007; Harris et al., 2006; Stott et al., 2006b).

And a couple more since then to get him started, and it looks as if there are more papers focusing on regional projections, not just global projections now.

Knutti et al (2010) Challenges in Combining Projections from Multiple Climate Models (that I quoted from above).

### Suggestions for rbgatduke

**didn't**include - here and here.

For now, I'll just make some final suggestions for the DuKE from Duke.

I think the real giveaway is that he can't tell the difference between a model and a model run.

ReplyDeleteRyan, what do you make of this? I kept coming back to it trying to figure out what rgbatduke meant and what, if anything, it said about his understanding of models:

DeleteOne can at best take any given GCM run and compare it to the actual data, or take an ensemble of Monte Carlo inputs and develop many runs and look at the spread of results and compare THAT to the actual data.In the end I decided to leave it alone. But to talk of "at best take any given GCM run and compare it to actual data" suggested he doesn't understand how the models are used. Or why his "single model run" wouldn't tell you anything much compared to multiple model runs.

Also, I know Monte Carlo analysis has its place in some areas, but I don't know that it's used in the way he suggests (he writes

"take an ensemble of Monte Carlo inputs") with what he calls "GCMs" - the physics-based models.I think he just wants a nice single narrow line, preferably straight and preferably trendless. That would be nice but it's not the way climate models, weather or climate works.

It only takes one butterfly... :D

Actually, you can do Monte Carlo runs with a single GCM by starting the model run with slightly different initial conditions, or slightly different forcings.

DeleteIf you have a set of inputs that are equally likely, you get a set of outputs that are equally likely, and so give you a distribution that reflects the internal variability of the model. By internal variability I mean the model's approximation of natural variability - things like ENSO and so on.

For a really good model, that internal variability would be close to the natural variability as observed. We do know that the current suite of GCMs do okay, but not great, at reproducing the real world's natural variability. For example, most GCMs manage to reproduce a quasi-oscillation in the Pacific Ocean that looks a bit like ENSO, but in most of the models, the size of the oscillation is less than in the real world.

In practice, the runtime necessary for most GCMs rules out doing lots of Monte Carlo studies. The multi-model ensembles are a kind of substitute for MC runs, with the models differing rather than the inputs. It doesn't tell you anything about the internal variability of a single model, but it does tell you something about the range of internal variability across models. It's not the same thing, but it's still useful info.

Thanks. I did wonder about how initial conditions might be selected and figured that's where Monte Carlo might come into the picture. But I got the impression that wasn't what rgbatduke had in mind. Could be wrong.

DeleteThere is some issue with bias one from what I read. It came up in one of the sessions at the recent AGU meet on communication. And the paper by Knutti et al that I mentioned discusses it and I expect it's discussed often. That's the fact that there is a lot of overlap in the individual models - in their architecture, code etc - it gets a bit incestuous, I gather, especially as the models grow and encompass more of the earth system - major collaborations between different teams etc.

I think it's fascinating that he thinks the central limit theorem doesn't apply to random variates from a model. If you start the model with random samples from the distribution of input parameters, then your output will be a random variable. But he seems to think it's inappropriate to apply the CLT to that. Why? Are there classes of "good" random variables that are random enough for the CLT and classes that are not random enough? I've never heard that theory before!

ReplyDeleteAlso as far as I know the IPCC doesn't calculate Standard deviations of model ensembles, they give uncertainty ranges (which are different) and they don't give R-squareds - only Monckton does that and only if the value is small...

Yes, I've shown above that the IPCC doesn't do standard deviations of model ensembles. In one chart it shows the +/-1 standard deviation, but that's of the annual means of individual models not the standard deviation of the mean of the entire ensemble, and is used to indicate spread.

DeleteAs far as uncertainty goes, that's where the IPCC report goes into quite some detail. At the gross level it just used a -40% to +60% range around the AOGCM means. But it goes into a lot more detail than that for example here:

http://www.ipcc.ch/publications_and_data/ar4/wg1/en/figure-10-29.html

...showing the different methods and sources for estimating likelihood.

It's funny that the potty peer comes out almost ahead of the batty duke except for one major error made by Monckton. Rgbatduke finally admitted he was ranting about Monckton's charts after all. Makes Monckton look the idiot for calling Nick Stokes a liar and a troll. If Monckton had kept his mouth shut he might have come out ahead of the batty duke.

WM Briggs would not be an "authority" that I would bother to quote if I valued my own credibility. He has form himself in the credibility department

ReplyDeleteYes, Martin. He does that even in that comment where his denialism shows through. But that makes this whole episode more entertaining if you're into black comedy.

Deletergbatduke slams Monckton and Spencer and Christy

Nick Stokes sticks up for Monckton

Monckton attacks Nick for doing so

rgbatduke reaffirms his attack on Monckton and Spencer and Christy

WM Briggs slams rgbatduke

Monckton is strangely silent

It's not often you see deniers all turn on each other. And it's not often you see anyone but the nutty defend Monckton. Usually fake skeptics are quite comfortable with holding contradictory positions like "it's cooling" plus "it's warming but it's natural" all at the same time.