Bob Tisdale says "it's obvious" (my bold italics):
January 26, 2014 at 4:59 am
markstoval says: “One question. Why is there not some agreed upon method or rule for choosing the base period? I always get the feeling that there is bias in the choice of the baseline period no matter who picks it.”
You’d have ask the three suppliers.
But the other question is, why aren’t they using 1981 to 2010 for base years as requested by the WMO? The answer to that is obvious: the anomalies wouldn’t look so high using the most recent 30-year window (ending on a multiple of 10). It’s all a matter of perspective.
 |
| Credit: Plognark |
If any climate newbies are reading this and want to know what it's all about, read this.
Addendum:
This is in response to comments below. It's a chart I did last month, so it doesn't have the very latest data. Still, it allows comparison of all the different data sets. They are all computed using the 1981-2010 baseline. Click to enlarge it.
The differences are minute. You can see the trend lines are almost identical. Over a longer period differences would most likely disappear altogether. The smallest slope is
Update
This article annoyed Bob Tisdale so much that he wrote a blog article in response, in which he showed he doesn't know anything of my sparse history at WUWT, getting it completely wrong. (He didn't notice me at WUWT coz I was there so little and so rarely. He notices HotWhopper though :)
Anyway, in his latest article (which I won't link to) he thought he was having a shot at me. Instead he proved again the very point I made above, writing:
As shown, when the data are presented using the standard NCDC base years instead of the WMO-preferred base years, the anomalies (the data) are about 0.4 deg higher (warmer). Thus my comment:
But the other question is, why aren’t they using 1981 to 2010 for base years as requested by the WMO? The answer to that is obvious: the anomalies wouldn’t look so high using the most recent 30-year window (ending on a multiple of 10). It’s all a matter of perspective.
If you were the NOAA/NCDC and you were trying to assure that you continued to receive your share of tax dollars for global warming and climate change research, which statement would you want to present to the public and those who distribute the funds?
The annual global combined land and ocean surface temperature was 0.62°C (1.12°F) above the 20th century average…
Or?
The annual global combined land and ocean surface temperature was 0.2°C (0.36°F) above the 1981-2010 average…
Ummm... Bob's probably hoping his "public" are no more cluey than he is.
Sou
2 February 2014
The Effect on slope using base period from 1931 to 1995
ReplyDeletehttp://climateandstuff.blogspot.co.uk/2012/08/the-effect-on-slope-using-base-period.html
Is a bit of investigation into baseline periods and the effect on simple straight line slopes from data generated from these baselines
.
Starting a baseline in 1976 gives minimum slope and
a 1981 baseline is certainly not giving the worst slope
cheers1
If the linear slope changes as you change the baseline (i.e., offset), you're doing something wrong.
DeletePL
Exactly PL. If y=mx+b, b has nothing to do with m.
DeleteThefordprefect, I've had a quick look at your spreadsheet which is very detailed, by the way.
DeleteIt's hard to tell what you're doing but it looks as if you've made it overly complex. You seem to be recalculating anomalies each time from a *shifting* baseline rather than from a static (fixed) baseline each time. In other words, you aren't really calculating anomalies from a baseline. That is why the slope shifts. At least that's as far as I can make it out from your macros.
Really, you only need to work out the average temperature once. Then if you want to shift the baseline all you have to do is get a new average for the period you've selected as the new baseline period and see what the temperature difference is from that new baseline temperature.
If the first baseline has an actual temperature of 3, then a temperature of 4 would be one above the baseline. The baseline of 3 becomes the new zero and the anomaly is plus one.
If you shift the baseline to down to 2 (eg by picking an earlier period), the a temperature of 4 would be 2 above the baseline. Provided you've got fixed data, it's not possible to have different slopes for different baselines.
If it were me, I'd be first working out the average temperature for the period you want to study over the area you want to study. I expect you're using some sort of gridding. The temperatures look to be in degrees Celsius but it doesn't matter what the unit is.
1. Work out the average temperature first.
2. Then select a baseline period. Work out the average temperature for that baseline period.
3. After you've got the baseline temperature in whatever units (eg degrees Celsius) that effectively becomes your zero line (or baseline).
4. All you have to do is subtract the baseline temperature from actual for each month/year (whatever period you're working with). A temperature higher than the baseline temperature will be a positive anomaly and a temperature less than the baseline temperature will be a negative anomaly.
The only thing that will shift is the zero and therefore the anomaly. The slope won't change.
I hope that makes sense.
Ford, if you describe exactly what you did, or maybe post your spreadsheet (is that possible on blogs?) we can help figure out where things went wrong.
ReplyDelete54 mbyte file is here.
DeleteThere was one known error with earliest point.
Have modified now to not average data before deyermining the slope.
There seems to be problems with lack of data (insufficient reporting sites early on) There is still a change in slope noticed with change in baseline.
The sheet is big it krew from other use for other purposes
Documentation is poor
It is complex.
Will help if desired (use my blog rather than clutter this one!
Macros are used to convert formats of downloaded data to used format in sheet.
download data in data tab
invalid data removed by error removal sheet
baseline data per mont over 30 year period is generated by avg 1961 to 1990 tab this also generates the plot just change the start date for the 30 year period to one in the left column of the ploted points
Anomaly calculates the anomaly from the 30 year baseline
Av Anom calc tab selects required sites the average period a data quality allowed.
Averaged sites chart plots the average of the selected sites
Spaghetti Chart is a plot of individual selected stations
the rest is not relevant
Comments welcome!
file is here
Deletehttps://drive.google.com/file/d/0B37eUysRTl9oNFlaWkVFREctRFE/edit?usp=sharing
See my comment above - you can tell me if what I've guessed is what you've done. As far as I can make out, you haven't selected a static baseline in your calculations - you keep resetting it all the time so there is effectively no "baseline".
Delete@thefordprefect
DeleteThere is a little image in Sou's post with the caption 'The stupid, it burns'. For some inexplicable reason, you've decided to come to this blog and prove it beyond any doubt. You are failing 5 grader math. Any school aged child will tell you that the baseline set's the 0 point on the Y axis, it does NOT change the data, or any trends in the data.
Check this out, and see where you are going wrong.
http://www.mathworks.com.au/help/matlab/creating_plots/modify-baseline-of-bar-graph-1.html
A quick Google search reveals that you have obsessively been on this crusade for a number of years, posting on anti-science sites like tall bloke, scienceofdoom, drroyspencer, principia-scientific, reasonandreality, noconsensus, and then trolling and thread bombing on science sites like this one. Why are deniers so obsessed with showing off their stupidity?
The fact that you can't even do basic math is really typical of your average denier, and of course when they are told 'you are doing it wrong', you will even deny that. Please, your excruciating stupidity is just painful. Just stick with impressing and fooling other stupid people.
The baselines are a series of monthly averages calculated only once for each baseline start date
Deletei.e. the average of 30 years of january monthly figures for each station. starting at say 1936
similarly for feb to december
these station specific monthly baseline averages are then subtracted from each data point in the monthly data for that station. i.e.
January 1951 - baseline january:
january 1952 - baseline january etc.
This is repeated for all the data points, all the months, for all stations analysed.
This removes a large portion of the yearly temperature changes.
and produces an anomaly for each station (shown in the spaghetti plots)
These anomalies are then everaged to produce the average temperature plot and a straight line fit made for this average. The slope of this line is then plotted against various baseline start dates.
I believe this is the correct method.
I too was surprised that there was a difference - BUT NOTE that it is only a couple of % different!!
Dave
Please check a few comments I made to the likes of tallbloke/slayer/spencer and why do you include science of doom in your anti-science blogs? Perhaps you should also have a look at my postings on my blog http://climateandstuff.blogspot.co.uk/ before acusing me of anti-science!
The basic maths in this analysis is done by excel. I put in a method which of course may be wrong - but I do not think so but will be pleased to correct if you show me my errors!.
@thefordprefect
DeleteI think I might owe you an apology. I've actually looked at your blog, and although at first it appeared to be a denier site, after some reading it seems that you do have some good material, although it does look like you have a lot of links to denier sites. I actually avoid the anti-science sites as I don't like my clicks paying for their propaganda, so if you were there as a supporter of science I do apologise.
"why do you include science of doom in your anti-science blogs?"
I've never heard of it before, and it sounded like a denier site. It is very confusing sometimes as the denier sites will use sciency sounding names. I've actually had a better look, and it looks like a good site. We all make mistakes now and then.
In the first link you provided you mentioned "As requested for WUWT here is a plot of linear curve fit to plots of the same data referenced to 30 year periods from 1931 to 1995"
To me, that looked like you were part of the WUWT crowd.
Now with regards to the baseline, changing baselines to get different 'appearances' is a common denier tactic, and you were following it to a tee. Please note, that changing baselines does not change the data. You are overcomplicating it, and are doing something terribly wrong.
Dave, Science of Doom is an outstanding resource. Highly recommended, but it can be very hard going as he does not shy from the detail.
DeleteI would say that a few percent differences are not a sign that the computation is wrong.
DeleteStations need to have data in the climate normal period, to be able to compute the mean temperature over that period. Thus by changing the normal period, you will in practice also change your selection of stations used to compute the global mean.
Related to this post. The WMO has written a proposal to update the climatological normals every decade, but it retains the 1961-90 for long term climate change assessments and keeps that unchanged as long as there are no scientific reasons to do so. The new 10-year periods are intended more for user products.
UAH are now the ones with the latest baseline (Jan 1981 - Dec 2010); RSS is
ReplyDelete(Jan 1979 - Dec 1998). They recently moved it up to this, which is difficult to understand, except maybe for making the numbers smaller. Having the same baseline as RSS (or vice versa) is the least anyone can ask.
David, I don't have any problem with different baselines. They are presumably picked for different reasons than simply comparing one dataset to another. Once you've picked one stick to it, I say. Otherwise there are oodles of charts and calculations everywhere that would need to be reset.
DeleteIt's easy enough to convert the different ones if you want to compare them. See addendum in the article above.
Change a baseline and you have to edit all previous published plots to maintain consistency. Lots of work to no useful purpose.
DeleteSou you show a plot for 1981 baseline. Have you tried a 1941 baseline? You of course cannot simply rebaseline the baseline, I think thart the small differences is casued by noise in the monthly data
DeleteSorry about the deleted post. It was accidental. I've managed to retrieve it above.
DeleteAs for a 1941 baseline do you mean 1941-1970 or do you just want to select the average for 1941 and use that as a baseline?
You can certainly do that for GISTemp and HadCRUT. Obviously it's not possible for UAH and RSS because there were no satellites in 1941, hence no lower tropospheric temperature to use as a baseline.
I've no idea what you mean by "of course you cannot simply rebaseline the baseline". That's what you just asked me if I could do. Rebase the baseline to the 1941 average temperature. And yes, I can.
I'll put a link here when its done.
Here's a chart of GISTemp global surface temperatures, with a baseline of 1941 and a baseline of 1941-1970 average.
DeleteI plotted the same period as above - 1979 to 2012 with linear trendlines. As you can see the slopes are identical.
sou
Deleteif your baseline however it is obtained only contains one number then this will of course only shift the plot up and down (no slope change)
Mine is a monthly baseline which will produce different results - a baseline started in 1981 will closely match monthly data arround this date but will be less of a match outside the base line period and it is this that creates the change in slope. data from close to the baseline period will be less noisy than data outside the baseline period.
I think I see what you are getting at, thefordprefect.
DeleteThe reason you are getting different slopes is because the data is different. You are talking about something different to what my article was about.
I can see that you are attempting to get anomalies for different stations presumably to compare them or to work out a regional temperature anomaly.
But when you refer to "baselines", you are talking about monthly baselines for a single station, which isn't the same thing (as you've pointed out) as a baseline for the aggregate - such as the average annual global surface temperatures.
I expect you are attempting to follow the method used in this paper - or partly:
http://pubs.giss.nasa.gov/docs/1987/1987_Hansen_Lebedeff_1.pdf
Have you gridded the stations? Or weighted them? And how are you dealing with the gaps? All of these will have an impact.
In the paper I quoted, it looks to me that Hansen et al didn't use a base period to work out anomalies at the station level. From what I gather, they did a calculation using the longest period common to stations being compared (at the station level). It looks as if it's after gridding they shoot for 1951-1980 as the baseline period, which is what they now use for global surface temperatures. I could be wrong about their method. I've not worked through the paper in detail.
Anyway, I can see what you are saying now. You are saying that if a different period was used as a baseline for each station then there may be slightly different wiggles as they were aggregated and a regional temperature anomaly was worked out, which may impact the global anomaly. And this is understandable at the small scale you are working on and the methods you are using.
I can't see that it would make much difference on the global scale. The fact that HadCRUT, GISTemp and Berkeley Earth all have similar trends and shapes despite using different algorithms, supports that.
BTW (for other readers) if you were just shifting the baseline for January for a single station, that would be comparable to what the above article is about. You'd get exactly the same slope no matter what the baseline was.
Here's another more recent paper on GISTemp, which shows the complexity in working out the global surface temperature changes over time. I don't think it discusses the calculations at the station level though.
http://pubs.giss.nasa.gov/abs/ha00510u.html
Feel free to weigh in - anyone else who's played around with individual station data at the regional or global level. (Nick Stokes does this sort of thing.) It's not something I've done.
thefordprefect,
DeleteYou might want to take a look at Clear Climate Code to see thier implementation of the GISTemp algorithm. It sounds like you have a coding error.
SkepticalScience has a series of articles on how global surface temperature anomalies are calculated by different groups, if anyone is interested. All parts are accessible from this first article in the series:
Deletehttp://www.skepticalscience.com/OfAveragesAndAnomalies_pt_1A.html
Tamino has frequently done this, calculating anomalies relative to monthly values over a baseline period. However, if done properly, I would think the only differences between monthly baselines over different periods would be a constant offset to each of those monthly (anomaly) values, and _no_ change whatsoever in trend lines over the course of a year (that is, over periods that are multiples of 12 months).
DeleteThe only way to get differences in trendlines should be if you are _not_ computing trends over multiples of 12 months, and I cannot see those as having significant differences.
thefordprefect: you say "data from close to the baseline period will be less noisy than data outside the baseline period." If this is what you are seeing, then your calcs are incorrect. All a baseline does is add a *single fixed* offset to *all* the data. The amount of noise in the data does not change (must not!) with a change in baseline.
DeleteFrom your description above, you take away the "baseline january" and the "baseline february" and so on. That is fine if you are *only* looking at the Januaries and Februaries etc in *isolation* from each other. It makes no sense if you are then analysing the Jans and Febs and the rest of the months together to find the trend, as you have shifted each month by a different value, and therefore changed the whole shape of the data!
A simple test to see if you are applying a baseline correctly is this: the difference between the raw data and the baselined data *must* be the same constant value for *all* data points. Your description above would *not* satisfy this requirement.
Hope that helps.
The plot would look exactly the same with a different 0 point on the x-axis.
ReplyDeleteIs "rebaseline the baseline" a typo or is it supposed to mean something?
Aargh! "0 point on the y-axis". Murphy's Typo ...
DeleteIt sounds like the slight change in trend is caused by baselining different months separately, then doing the linear trend on the recombined monthly anomalies. So, any change is relative values for, say, Jan vs July will lead to slightly different trends.
ReplyDeleteThat's actually interesting: what are the changes in seasonality from one epoch to another? Worth looking at.
PL
The baseline should always be the latest record warm year. That way no AGW. This method is called ostrichising.
ReplyDeleteOh I love this:
ReplyDelete"I always get the feeling that there is bias in the choice of the baseline period no matter who picks it."
Occam's razor tells us the bias lies with the commenter.
Sou, I think you mean 0.173 degrees per decade, not 1.73 as in the text under the graph.
ReplyDeleteOops, thanks. We'd be goners by 2050 at that rate, wouldn't we. I'll fix right away.
Delete