The relevance of (climate) models - increasing understanding

Climate science deniers in the main, do not understand why models are used in science. Nor do they typically understand how they are used, or how they are constructed.

Today Wondering Willis Eschenbach demonstrated this quite well (archived here). He wrote about a recent article in Science, by Professor Alex Hall. The article was discussing the merits and limitations of using General Circulation Models (GCMs) to model regional climate change, through a process known as down-scaling.

In his article, Dr Hall describes downscaling as follows:

The concept behind downscaling is to take a coarsely resolved climate field and determine what the finer-scale structures in that field ought to be. In dynamical downscaling, GCM data are fed directly to regional models. Apart from their finer grids and regional domain, these models are similar to GCMs in that they solve Earth system equations directly with numerical techniques. Downscaling techniques also include statistical downscaling, in which empirical relationships are established between the GCM grid scale and finer scales of interest using some training data set. The relationships are then used to derive finerscale fields from the GCM data.

He then goes on to discuss some of the limitations. In particular, GCM's typically have atmospheric biases, which are amplified when expanding to the regional scale. He also wrote about the strengths:

Fortunately, there are regionalscale anthropogenic signals in the GCMs that are not contaminated by regional biases. The best example might be the models' direct thermodynamic responses to anthropogenic forcing, most notably warming. Warming signals arise from hemispheric- to global-scale processes (5). Water vapor, cloud, and surface albedo feedbacks, as well as the ocean's relatively slow heat uptake, are the main factors that shape warming and its spatial distribution. 

Alex Hall gave two specific examples of where downscaling enhances understanding of climate at the local level. One was the Great Lakes region in the USA and the other was the headwater of the Ganges River in India. About the latter, he wrote that "the high elevation headwaters of the Ganges River warmed by a further 1.0°C by 2100 beyond the warming projected by the GCM. The reason is that well-understood snow albedo feedback effects are not resolved by the GCMs. "

Alex concluded that downscaling can be of particular value to investigate climate change in regions having complex coastlines and topography. He wrote that only those GCMs with reasonably realistic atmospheric local circulation changes should be used (since it's atmospheric circulation that generally has the largest biases).  Even then the results need to be examined to check they are realistic.

A model is relevant if it improves understanding

Willis Eschenbach thinks all this is hogwash. He decided to take a shot at Alex Hall for writing this:

The appropriate test of downscaling's relevance is not whether it alters paradigms of global climate science, but whether it improves understanding of climate change in the region where it is applied. 

Willis Eschenbach thinks scientists have a time machine

Willis disagreed that a test of relevance is whether a model is, umm, sufficiently relevant to improve understanding of local climate change. He decided that the test of relevance is whether or not it matches observations.

Now I don't know if Willis has a time machine - or whether he thinks that scientists do. The next sentence that Alex Hall wrote was about the Great Lakes example he gave. Willis chose not to include it, probably because it would spoil his little jibe. Alex Hall wrote:

The snowfall example above meets that test. In many places, such fine spatial structures have important implications for climate change adaptation. In the urban areas of the United States and Canada most affected by lake effect snow, infrastructure and water resource planning must proceed very differently if lake effect snow is not projected to decrease significantly.

What Dr Hall was talking about was that regional downscaled model suggested that although in the future, the Great Lakes will not be frozen for longer, there won't be a decrease in local snow precipitation around the Great Lakes. This is because lake effect snow is possible for more of the winter. That effect cancels out the overall snow decrease (more precipitation falling as rain). By contrast, the full scale (not downscaled) GCM had a much larger decrease in snowfall, with rain replacing it. It was only in the more finely scaled regional model that the details showed up the increase in lake effect snow.

Here is the image that Alex Hall provided, demonstrating this. It compares a GCM (left) and a downscaled model (right) for 2050-2060 (compared to 1979-2001). Click to enlarge it:

Simulated snowfall changes (%) over the North American Great Lakes region by 2050 to 2060 are shown for two cases. (A) Results from CCSM (Community Climate System Model), a global climate model (GCM). (B) Results when the GCM output is downscaled with the Weather Research and Forecasting (WRF) dynamical model (6). In the GCM, snowfall decreases because anthropogenic warming causes a smaller fraction of precipitation to fall as snow. When the GCM is downscaled, snowfall also generally decreases, but with substantial spatial variations. In particular, in areas affected by lake effect snow, the snowfall decrease is almost nonexistent in the regional simulation. Because the lake effect snow can only occur when the lake is unfrozen, a warming climate enhances the likelihood of lake effect snow, largely canceling out the general snow decrease due to a warmer atmosphere. Changes are calculated by differencing the 2050 to 2060 future period and the 1979 to 2001 historical period (6).

Willis either couldn't understand the article he read, or he just felt like taking a pot shot at science. The fact that he thinks relevance is measured by future observations paints him as dumb ignorant. And in case you think that he didn't mean to wait until 2050 or 2060 to see if the model was relevant, here is a comment from him:

Willis Eschenbach wrote, in response to the quoted question (my emphasis):

January 5, 2015 at 6:15 pm
...So, how would you propose to compare model projections to future observations?

I’m gonna assume that this is a serious question, although it seems obvious. Two choices. Either:

1) Initialize the models on the first half of the historical dataset, and then compare the output of the models to the second half of the dataset, or,

2) Make the actual predictions, and then wait and see if they come to pass.

Not all that tough …
Best to you, 

So Willis wants the scientists to wait for fifty or sixty years or more to see if this year's model output "comes to pass", before determining whether or not a particular regional model is relevant.

I wonder if Willis is confusing relevance with accuracy.  As I've discussed, the paper does talk about the importance of accuracy. That's necessary for usefulness - but the actual usefulness is gauged by the extent to which a model increases understanding. Being accurate without adding to understanding has very limited value. Probably Willis doesn't care one way or another. He just wanted to prove how clever he is. And he did, didn't he.

Decision-makers are pushing scientists for regional climate projections

One of the main points of Alex Hall's article was that planners and policy makers are pushing scientists hard to tell them what to expect from climate change at the regional level. The planners and policy makers aren't going to put all their plans and policies on hold for fifty or sixty years while waiting to see if a regional model run from 2015 turns out to be relevant. They would much rather the scientists let them know if the regional model increased understanding of climate at the local level. Then they can decide the specs for their bridges, and water supply infrastructure, and storm drainage systems, and transportation infrastructure etc etc.

What is the test of relevance for other (non-climate) models?

I'll just make one more point, which will be obvious to scientists and engineers and economists and financial planners and anyone who uses models. Many models can't be tested on observations to determine their relevance. Models are often used to help plan for the future, not always to understand the present or the past. Their relevance is determined by the extent to which they increase understanding of whatever it is that is being modeled.

The simplest examples, like models for aircraft design or bridge construction - are used to test for design flaws, to determine the materials needed, to work out the steps in construction, to see how the design will stand up to various stresses etc. In other words, to improve understanding of whatever it is that is being modeled. For many things, it's a bit late to wait to see if observations match the models. Do you build and fly an aircraft thousands of km over years and take observations before deciding if the model is relevant?

If you're wanting to read about climate models, Scott K. Johnson's article at ArsTechnica is the article I generally recomment.

From the WUWT comments

JKrob can't imagine that people who build roads and water reservoirs and drainage systems and telecommunications infrastructure would need projections of precipitation, or temperature or flood likelihood or drought. I'm guessing he/she is a conspiracy theorist, too:

January 5, 2015 at 6:36 pm
“Pressure to use (downscale) techniques to produce policy-relevant information is enormous…”
Interesting, but not surprising. ‘Pressure’ from whom – management, specific governments, UN…others??

RomanM writes a one-liner. He's a denier statistician I believe, who I guess doesn't use models (or has no faith in his models, or doesn't think his models are relevant or extend understanding of anything)

January 5, 2015 at 6:40 pm
Lipstick on a pig… and a not-so-good looking one at that…. 

Andres Valencia wouldn't have a clue about climate or models or relevance or understanding but can't resist adding a meaningless comment. He also shifts from laughter to weeping readily. Hysteria?:

January 5, 2015 at 7:54 pm
“whether it improves understanding of climate change in the region where it is applied.”?
This must have come out of the “Humor” section of the paper, it’s just a joke.
Oh, wait, there’s no “Humor” section in this paper.
Thanks, Willis. I had to laugh, then cry. 

Tom Trevor has a grand idea. Imagine building an economics model, starting with a single individual's random purchase of a packet of chewing gum:

January 5, 2015 at 9:09 pm
It seems to me that to Upscale would make more sense. First try to make an extremely accurate model of local weather over a very short period of time. Say something like this: It is now 65 degrees and 74% humidity on my porch I predict, based on my model that one minute from now it will be 65 degrees and 74% humidity on my porch. If over time your model show skill, then expand it in space and time, if still show skill expand it further. Eventually you might work it up to a global model of the climate in 100 years, but before it gets there it would have to show the ability to reasonable predict regional weather over at least a month. Working from future global climate to future local weather seems working backwards to me.

michaelwiseguy's comment is fairly typical of most of them at WUWT:

January 6, 2015 at 12:35 am
Are we talking about natural climate change or that mythical man-made climate change everyone is talking about?

That's enough. You get the picture. You can read more here if you want to waste a few more minutes of your life.

---

Hall, Alex. "Projecting regional change." Science 346, no. 6216 (2014): 1461-1462. DOI: 10.1126/science.aaa0629 (subs req'd)