.

Thursday, June 5, 2014

BUSTED: How Ridiculous Richard Tol makes myriad bloopers and a big fool of himself and proves the 97% consensus

Sou | 7:48 PM Go to the first of 24 comments. Add a comment

Update - see below.


Anthony Watts is all excited (archived here, latest here) because economist Richard Tol finally found a journal to publish a paper he's been trying to get someone to publish for nearly a year.  Anthony wrote, mischievously and wrongly a headline and opening comment:
BUSTED: Tol takes on Cook’s ’97% consensus’ claim with a re-analysis, showing the claim is ‘unfounded’
A new paper by Dr. Richard Tol published today in ScienceDirect, journal of Energy Policy, shows that the Cook et al. paper claiming that there is a 97% consensus among scientists is not just impossible to reproduce (since Cook is withholding data) but a veritable statistical train wreck rife with bias, classification errors, poor data quality, and inconsistency in the ratings process. The full paper is available below.

Anthony Watts is wrong, and wrong and wrong again - he's busted!


Firstly, Richard Tol didn't do any reanalysis. He didn't categorise all the abstracts himself. He just did some wonky sums and got the wrong answer, based on flawed assumptions and more. And he threw in a large number of unfounded speculative statements. Not only did Richard not show the claim was "unfounded", he wrote that he accepts the main finding of Cook13.

Secondly, thirdly and fourthly etc, John Cook isn't withholding any data. He provided more than just all the data anyone would need to repeat the analysis (here and here), he even provided a web tool to help people who wanted to to categorise abstracts for themselves. John Cook provided all the data needed to reproduce the analysis. Richard Tol didn't even try to do so. As for the quality of the data - it's a complete set of around 12,000 abstracts returned using a Web of Science search of key terms. So I don't know if Anthony is trying to say that Web of Science isn't any good.  Anthony probably doesn't know either. He just likes blowing hot air.

Anthony's wrong on another point, too. The full paper isn't available at the site Anthony linked to unless you're a paid up subscriber or are willing to pay for the privilege.  (You can read one of Richard's earlier rejected versions here, which I got from Richard's own blog. It's not much different to the final paper.)

This is a very long article, although it barely scratches the surface in the myriad flaws in Ridiculous Richard's paper. If you're on the home page, click here for more.



Richard Tol is about on par with Christopher Monckton


I've got to say that this paper of Richard's is one of the silliest papers I've seen in - oh, a few months I'd say. Probably not since the dismal effort at disinformation that Legates, Briggs, Soon, and Monckton managed to get into a journal of dubious reputation.

After being rejected FOUR times by THREE other journals, Ridiculous Richard finally found one that was prepared to put a dent in its reputation by publishing it. He's given his paper the innocuous title:
Quantifying the consensus on anthropogenic global warming in the literature: A re-analysis
The journal is Energy Policy, which describes itself as:
international peer-reviewed journal addressing the policy implications of energy supply and use from their economic, social, planning and environmental aspects. Papers may cover global, regional, national, or even local topics that are of wider policy significance, and of interest to international agencies, governments, public and private sector entities, local communities and non-governmental organisations. Within this broad spectrum, topics of particular interest include energy and environmental regulation, energy supply security, the quality and efficiency of energy services, the effectiveness of market-based approaches and/or governmental interventions, technological innovation and diffusion, and voluntary initiatives where the broader policy implications can be recognised. Policy prescriptions are required to be supported by rigorous analysis and balanced appraisal.

I guess they were having a slow month to stretch their "policy implications of energy supply and use" to a "re-analysis" of the Cook et al study of the scientific consensus on global warming. I see from their website that their impact factor has dropped of late from 3.382 to 2.743, so maybe they were hoping for a few citations on denier blogs :)

The journal editors and reviewers didn't pick up some really odd things about his paper. Richard not only made big bloopers in his quantitative workings, he relied on material that no self-respecting academic (or journal) would tolerate. For example he:
  • cited musings and speculations from denier blogs and denier publications as "evidence" of something or other
  • misrepresented snippet from emails stolen from a private SkepticalScience website, which were published by denier bloggers
  • speculated that the people might have got tired - yes, really (using a flawed analogy - see below)
  • cited no less than three market research studies as if this was a poll of people rather than a classification of scientific abstracts and papers.  (The closest thing to a "poll" was when the researchers asked the papers' authors themselves - by email - to categorise their own papers. An exercise by which the findings of Cook13 were validated. I doubt the authors of the abstracts would have been too tired to categorise their own papers!)

Richard even acknowledged he had "useful discussions" with a bunch of denier bloggers and a confessed script kiddie. I don't know all the people, but I do know Poptart (Andrew K.), Brandon Shollenberger, Carrick, Lucia Liljegren, Shub Niggurath - all of whom are denier bloggers or commenters of varying degrees of "cuckoo-ness". He also claimed to have had "useful discussions" with people who tried to help him by pointing out the flaws and fallacies in his paper, but failed - Dana Nuccitelli, Willard and wottsupwiththatblog.


Nonsensical nonsense from the Ridiculous Richard Tol


As an example of silly and flawed speculation, Richard implied that an analysis of published science is not the right way of determining what published science shows. Yep, you read that right. This is what Ridiculous Richard wrote in his final published paper:
Cook's estimate of a 97% consensus was thus probably inflated by self-selection. Researchers who think that climate change is real and anthropogenic are more likely to study climate impacts and climate policy than those who are unconvinced.

Really? I suppose that explains the dearth of published literature by loud-mouthed denier bloggers and paid disinformers, who find it takes much less effort to go the route of the tobacco lobby manufacturing fear, uncertainty and doubt by insinuation that do any research themselves. Deniers and disinformers daren't do research because they know the basic fact of human-caused global warming is irrefutable. Never mind that if someone were to prove successfully that all atmospheric physics and chemistry was wrong and CO2 isn't a greenhouse gas after all, they'd be on their way to winning a Nobel Prize. Not bad motivation I'd have thought.

The abstracts got tired!


At one point Richard speculated, that:
Fatigue may have been a problem, with low data quality as a result

If you look at the literature Richard cites, the article to which he refers is Chapter 22 of the International Handbook of Survey Methodology - about market research and opinion polls etc. In other words, it's not about a literature review or a categorisation of scientific abstracts, it's how people being polled (the subjects) might give lower quality responses over time. People can get a bit jaded if they are polled multiple times. The interviewers on the other hand (the equivalent of the Cook13 researchers) become more proficient as time goes by, not less so.

Therefore on the subject of tired subjects, in this context the "fatigue" would refer to the equivalent of the abstracts that Cook13 reviewers analysed.  Do abstracts suffer fatigue? Well I guess that sometimes they get out-dated but the words don't change. The researchers would be the equivalent of the people doing the polling - and would be expected to get more skilled over time.

This was the section of his paper where Richard misrepresented a quote from an comment stolen (hacked) from a private forum to support his contention. That's on par with the misrepresentation of deniers about the emails stolen from CRU.  Richard linked to an denier blog where the unprincipled Lucia Liljegren unethically published some of the snippets of stolen material. The person making the comment wasn't complaining that he was tired. What he was saying was that occasionally he'd come across an abstract that he thought he might have seen before. He wasn't sure if it was because after looking at lots of abstracts it might just be that one is similar to another he'd seen or if it had inadvertently been duplicated - and that someone could double check for duplications.  Nothing at all to do with "fatigue".

(I'm surprised that the editors of Energy Policy allowed that one through. I'd have thought that even if they closed their eyes to the ethics of using stolen private material that Richard didn't have permission to use, that at least one of the editors or reviewers would have checked that Richard interpreted it correctly. It's patently obvious to anyone who checks that Richard didn't.)


Richard agrees with the 97% consensus but has it in for John Cook and colleagues


You may remember that Richard claimed last year in the version he tried to get ERL to publish, that there is a scientific consensus that humans are causing global warming. Well his latest blooper isn't changed much and he still maintains that:
There is no doubt in my mind that the literature on climate change overwhelmingly supports the hypothesis that climate change is caused by humans. I have very little reason to doubt that the consensus is indeed correct.
 Richard's latest version (subs req'd) has the exact same sentence in the conclusion.  So he's not disputing the consensus, he was instead obsessed with trying to find something wrong with the analysis. Richard is a weird chap and often behaves in a manner unbecoming to a senior academic.

SkepticalScience has torn Richard's paper apart already. It took Richard months to get his paper published and SkS have been able to rip it to shreds already. Apparently they've had their rebuttal to Richard's paper accepted by the Journal but the journal hasn't yet published it. (Aren't they usually published at the same time?).


Richard's big blooper


I'm finding it hard to decide which is the biggest blooper. At the moment I've got his the abstracts got tired running about level pegging with a really dumb assumption he made in one of his few attempts at quantitative analysis.

What Richard did was make a mistake that anyone in their right mind would have immediately seen was wrong. His big mistake resulted in him deciding that Cook13 should have found "only" a 91% consensus!

Richard's mistake is a bit complicated. What he did was try to estimate the error rate in the classification of abstracts and decided that 6.7% of abstracts may have been categorised wrongly. So far so good. That's entirely possible. It's what he did next that was hilariously wrong. He assumed that any errors were split 55% to 45% in favour of rejection.

This is a bit complicated, so I'll let Dana Nuccitelli explain it (my bold italics):
To minimize our uncertainties, we had at least two people categorize each scientific abstract. Where those two raters disagreed, we had a reconciliation process. The disagreeing raters first checked their ratings again; if the disagreement persisted, a third person acted as the tiebreaker to establish the final rating.
Using the difference between our initial and final ratings, it's possible to estimate the number of papers that still remain in the improper categories after our reconciliation process. Tol put the estimate at about 6.7% of the total, and noted that 55% of our reconciliations from initial to final ratings were 'towards stronger rejection', while 45% were 'towards stronger endorsement' of human-caused global warming.
Tol then made a basic and critical error. His methodology resulted in assumptions that, for example, 55% of the remaining incorrectly rated 'no position' category papers should actually be rejections, while 45% should be endorsements. He didn't check to see how the reconciliations changed the initial and final ratings for each category, and this assumption led him to incorrectly conclude the consensus is actually 91%. Still a high percentage, but nonetheless in error.
In reality, as our response to Tol's critique (accepted by Energy Policy but not yet published) shows, there simply aren't very many peer-reviewed papers that minimize or reject human-caused global warming. Most of the papers that were reconciled 'towards stronger rejection' went from explicit to implicit endorsement, or from implicit endorsement to no position. For abstracts initially rated as 'no position,' 98% of the changes were to endorsement categories; only 2% were changed to rejections.
That makes sense when you think about it, because less than 3% of all climate papers reject or minimize human-caused global warming. There's no reason to expect 55% of incorrectly rated 'no position' papers to reject the consensus – in reality there just aren't that many rejection papers.
Now the fact that fewer than 2% of papers that attribute a cause to global warming say it's not primarily from human activities like tossing gigatonnes of CO2 into the air each year - should have been a reality check for Richard. He should have figured that it was most unlikely that more than half the number of papers that were reclassified would have been counted as rejection. Going by the numbers it should have been fewer than 3%, a similar distribution to that of all the papers that were categorised. Any other distribution would be out of line with the entire analysis.

The key point is that where there were differences between the raters and a third person had the final say, most papers that were changed from one category to another went from:

  • explicit endorsement of AGW to implicit endorsement of AGW or
  • implicit endorsement of AGW to no position on AGW or vice versa.

Contrary to what Richard postulated, there was not a huge shift from AGW endorsement or "no position" to rejection of AGW.

SkepticalScience has a couple of charts illustrating this in detail with numbers. Here's a simplified illustration as an animation to show what Richard's blunder resulted in.




In other words, where there was a shift in the category of an abstract, it just shifted slightly in the categorisation (mainly from explicit to implicit endorsement, or implicit to no position or vice versa) - and if anything the researchers tended to err on the conservative side if there was a marginal decision. There were hardly any abstracts that were reclassified from "no position" to rejection let alone from endorsement to rejection. Dana's diagram shows that out of 595 reclassified abstracts, only five papers would have gone from "no position" to "rejection of AGW" and only one paper from endorse to reject. Which makes sense, because it's well known that denier papers are very few and far between. Not only that, if a paper in its abstract clearly rejected AGW it would stand out from the crowd and it would be a no-brainer to identify it as such. For most of them it would hardly be a line-ball call. Where it was line-ball, in most cases it would be deciphering whether or not a paper actually expressed a position or not, or if it did was it explicit or implicit.

What Richard did was make a foolish and baseless assumption that papers went from "no position" on the following ratios:
  • 45% to endorsing AGW
  • 55% to rejecting AGW.
He reportedly said that he "used the marginal distribution of error corrections rather than the conditional ones" - whatever he meant by that.  Anyway, if you multiplied that out using his numbers, you'd end up with 293 papers shifting from "no position" to "rejecting AGW" and only 239 being shifted from "no position" to endorsing AGW - which anyone can see is completely implausible.


Most of the papers that were reconciled 'towards stronger rejection' went from explicit to implicit endorsement, or from implicit endorsement to no position. For abstracts initially rated as 'no position,' 98% of the changes were to endorsement categories; only 2% were changed to rejections.


Richard's flawed calculations still point to an overwhelming scientific consensus


Now remember, Richard decided on the basis of his flawed assumption that many more abstracts would have rejected AGW, that the consensus is 91%. That would still be regarded as an overwhelming consensus. So all his effort went into drawing an erroneous conclusion, but despite this he ended up with an overwhelming proportion of scientific papers endorse the science - that humans are causing global warming.


Richard's flawed calculation would get the same result no matter what


The really funny thing about all this is that an anonymous scientist worked out that using Richard's analysis, if many more abstracts were shifted to "endorse AGW" rather than "reject AGW", then using Richard's flawed analysis that would also have still led him to conclude only a 91% consensus. That's because there was more than one big flaw in his calculations.


Richard presents evidence that Cook13 conclusions are robust


Sadly Richard was so intent on finding fault with the method that he failed to see the big picture and ask the sort of questions of himself that any decent academic would do.  That's despite lots of people trying to save him from himself over the past few months, not least of which were the ERL referees who rejected his paper.  Richard was a bit naughty and published their comments (archived here) but took no notice of them. (The referees and editors at Energy Policy were not as astute as the ERL editors and referees.) For example, in recommending rejection one ERL referee commented:
Many of the claims in the abstract and conclusion are not supported by the author’s analyses. Much of the analyses explore methods choices made by the Cook et al paper, and often find differences when a different database or search term or subset of papers is analyzed, but the larger point is completely missed – that both the Cook authors and this paper’s author make assumptions about representativeness, appropriateness of search terms, appropriateness of different fields in calculations made. These are, in fact, assumptions. Thus, it is impossible to claim that the Cook dataset is “unrepresentative” of any larger population, as the other scenarios investigated by the author are just a different (and not necessarily better or “more true”, even in some cases less likely to be a good sample) set of assumptions. Regarding later calculations of consensus, the author finds largely similar percentages to that of the Cook paper and also seems to ignore the self-rated abstract consensus rate, presenting evidence in fact that the Cook paper’s main conclusions do seem to be quite robust, which is the opposite of what is claimed by the author.
In other words, Richard's own analysis supports Cook13 rather than the opposite!

There is a whole heap more wrong with Richard's paper and there will be other people writing about this as well, I'm sure. Meanwhile, go read Dana Nuccitelli evisceration of it or Graham Readfern's scathing article at DeSmogBlog, and Collin Maessen at realsceptic.com and weep for Richard Tol and all the deniers who tried and failed to dispute the overwhelming scientific consensus.

PS More bloggers are eating Richard for breakfast - here and here and here and here.


Update - 24 Tol Errors


There's a paper been put out by John Cook and colleagues. Do you remember when I said there were a lot of other errors Richard Tol made and I'd barely skimmed the surface? Well, if you want more nitty gritty in a more academic but readable style, read this paper.


From the WUWT comments


Many of the commenters at WUWT don't have a clue about Richard's paper or Cook13 either. In among the paranoid conspiracy ideation and arguably libelous accusations, there are a lot of comments from people who didn't much like Ridiculous Richard's paper or his conclusions. It's little wonder that Anthony didn't make it a "sticky".

This one's good. Anthony Watts is very reluctant to do any actual research himself because he knows there's nothing to support the silly unfounded claims made by Richard. rustneversleeps says:
June 4, 2014 at 3:15 pm
Anthony,
The only “veritable statistical train wreck rife with bias… poor data quality” is Tol’s bizarre paper.
Can we get you on record, Anthony, endorsing the math behind it? I know that posting it here is an implicit endorsement of Tol’s mangled effort, but could you step up and say outright how impressed you are? Please?
You are going to be embarrassed for having highlighted this. But that’s ok. Nothing we haven’t seen you do many times before.
By the way, Anthony, those extra ~ 300 papers Tol “found” rejecting the consensus. Care to point us to a few? 
REPLY: Here’s the thing, I don’t deal with punks that demand that I do specific things who are too timid to use their own names. Bug off – Anthony

Anthony loves people who are too timid to use their own names, as long as they salute Anthony himself. For example, I've never seen Anthony chastise "not his real name" Latitude, who is a WUWT regular and the first to pick up on the fact that Richard Tol accepts the scientific consensus and asks "really?":
June 4, 2014 at 3:12 pm
There is no doubt in my mind that the literature on climate change overwhelmingly supports the hypothesis that climate change is caused by humans…
really?
Yes, really, Latitude. Really and truly!

John explains Richard's blunder more clearly than I did and says (excerpts):
June 5, 2014 at 2:04 am
...Firstly, Tol corrects it to a 91% consensus which really just reinforces the idea of a strong scientific consensus.
Secondly the maths is bad. 2 people rated each abstract. In the case where the rating differed they went through a 1st reconciliation stage where the raters discussed and tried to agree on a single rating. Tol examined the reconciliation stage and looked at how the papers resolved in this stage were correct. Obviously one of the 2 raters would see their rating changed. Tol created this histogram of how each rating changes.
https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjsZ1f1xAqQqe20kFqTQqKwLiv4gatG8a19ijUgBDkvJvCdm_2bPU-P25E_gDXk3xKXcAVUlANMRQ4X6B4RGW1tHY07ebnRDz8hB-MdlSfld45-0TaQU0h9Mermicpt1mOPJMCHEy-A3CbJ/s1600/bias.png
Unfortunately he created this histogram across all ratings. So he has lost the important information that in general the reconciliation process moved papers towards a more neutral position. i.e. if a paper endorsed AGW then it was more likely it would be resolved to having no position and visa versa. Papers that rejected AGW were more likely to be resolved to no position.
Richard Tol then took this histogram and applied it to the disputed rating in each category. i.e. a paper rated 2 he assumed had a 50% chance of becoming a 1 or 50% chance of becoming a 3. This is not what is seen in the real data at all. The real data showed reconciliation moved towards the neutral position. If you were to test Tol’s method against the real data for the 1st reconsiliation process it gets it completely wrong.
Tol major error was assuming this histogram applies equally to every rating. This was always going to result in moving papers from high population categories to the extremes.
So not only is this paper reinforcing a scientific consensus the flawed maths allows “rustneversleeps” to argue that skeptics will accept any crap.

Steve Oregon isn't interested in reading either paper either and settles for the normal denier wail for any science they don't like - "fraud":
June 4, 2014 at 3:26 pm
Why would Tol (or anyone else_ have “no doubt in their mind” when they acknowledge some have resorted to such “Well-publicized but faulty analyses”?
It’s not like Cook’s work is the only cooked up AGW stunt.
Is Cook’s stunt fraud?

John Eggert notices that Richard accepts the scientific consensus and goes for the tired denier conspiracy theory:
June 4, 2014 at 3:35 pm
Latitude says: June 4, 2014 at 3:12 pm
There is no doubt in my mind that the literature on climate change overwhelmingly supports the hypothesis that climate change is caused by humans…
really?
Of course. The funding for research overwhelmingly supports research that supports the hypothesis, so the literature that flows from said funding will also support the hypothesis. It is just like tobacco in the 1950′s. The funding was all in support of tobacco being harmless, so that is what the research showed. If you show otherwise, you don’t get any more funding. Climate research today is at about the same place that tobacco research was about 1960. The dollars being thrown at it are huge compared to the tobacco scam, so it will take more than a surgeon general’s report to begin a shift.

ilma630 doesn't like Richard Tol's position much either and quite legitimately asks what's the point of his paper (my italics):
June 4, 2014 at 3:58 pm
On the one hand Tol debunks Cook et al, then immediately expresses the overt warmist case “There is no doubt in my mind that the literature on climate change overwhelmingly supports the hypothesis that climate change is caused by humans.” and “It will take decades or longer to reduce carbon dioxide emissions to zero—the only way to stabilize its atmospheric concentration.”. He might as well just agree with Cook’s consensus and have done with it.

David Ball isn't very happy with Richard either and says:
June 4, 2014 at 4:01 pm
Sorry, also wondering about the “polarization of the climate debate” comment? There has been only one side doing the polarizing.
This is also pointless as the POTUS quoted the 97% crap already and the damage has been done. This is just another distraction from the important questions that remain unanswered.
You earn a living doing this Dr. Tol? 

Nor is davidmhoffer happy with Richard's paper and seems to agree with the ERL reviewer in that the paper contradicts itelf, he says he's "rather disappointed":
June 4, 2014 at 4:55 pm
The conclusions of Cook et al. are thus unfounded. There is no doubt in my mind that the literature on climate change overwhelmingly supports the hypothesis that climate change is caused by humans.
What a tangled up mess.
1. Tol says the conclusions of Cook et al are unfounded, and then in the very next sentence pretty much says he thinks they are right.
2. The debate is not and never was about IF human activity causes climate change. The debate is about how much and quantifying any harm that may result.
I haven’t seen the full paper as it is pay walled, but from the little I see here, this paper contradicts itself (see 1. above) and adds nothing to our understanding of endangerment, just some hand waving on the issue along with some references to the ability to reduce emissions to zero on some vague assumption that despite the lack of consensus discovered in Tol’s own paper, it still needs to be done.
Rather disappointed actually.

Richard's paper went down like a lead balloon with Pamela Gray, who says "Tol is that mad scientist...":
June 4, 2014 at 5:36 pm
Having read the preprint now, I don’t see much in terms of standard scientific refutation. It seems filled with first person opinion as to methods but supports the conclusion. It’s almost like he is trying to imbue what was dead with Frankenstein-ish life. It’s ugly but it’s alive, says he.
The problem is that we are coming up to a majority of the common people who are now seeing catastrophic AGW as just that. Another attempt by some fool mad scientist to make something look real, and scary, that isn’t real or scary. My new name for catastrophic AGW: Al Grankenstein, in the flesh…er…rotting flesh. Tol is that mad scientist trying to make ugly walk again.  

strike is similarly unimpressed by Tol's paper and says, quoting Richard:
June 5, 2014 at 3:13 am
“There is no doubt in my mind that the literature on climate change overwhelmingly supports the hypothesis that climate change is caused by humans.”
That’s a guess, the first step in creating a scientific theory. If the paper doesn’t show any further evidence for his guess, his paper is even less significant than Cook et. al., who at least tried to proof their claim 


JimS sees Richard Tol (despite his affiliation with the denier lobby group the GWPF) as an "AGWer" and says:
June 4, 2014 at 4:19 pm
Are the AGWers now eating their own? What a shame, and what a shock too. 


andrewbranca is another conspiracy theorist who says it's all about power (not the electric kind):
June 4, 2014 at 3:13 pm
It’s never been about reducing CO2. It’s always been about power. Always.
–Andrew, @LawSelfDefense

Neil is a "just a little bit" denier and says:
June 4, 2014 at 3:44 pm
From the Conclusion: “climate change is caused by humans”.
Shouldn’t that be, “SOME climate change is caused by humans?”

Jake J doesn't want to stump up any funds and says:
June 4, 2014 at 4:57 pm
I’m not inclined to pay $20 to read Tol’s paper. If someone knows how to get it for free, please say so.

Anthony Watts, quite wrongly AFAIK, for those who don't have a subscription, says:
June 4, 2014 at 5:26 pm
Jake J, Hoffer, and Pamela, and others.
try following this link and using the scroll bar, the full paper is visible. No paywall.
http://www.sciencedirect.com/science/article/pii/S0301421514002821

There were quite a few people falling for Anthony's disinformation that Cook13 didn't provide their data. John Cook provided more than just all the data anyone would need to repeat the analysis (here and here), he even provided a web tool to help people who wanted to to categorise abstracts for themselves.


Let's finish with a sensible suggestion from Felix, who says:
June 4, 2014 at 6:24 pm
So, now that the consensus has been bounded between 91 and 97% isn’t it time drop the pseudo-skepticism and talk about how to best reduce ghg emissions?



Cook, John, Dana Nuccitelli, Sarah A. Green, Mark Richardson, Bärbel Winkler, Rob Painting, Robert Way, Peter Jacobs, and Andrew Skuce. "Quantifying the consensus on anthropogenic global warming in the scientific literature." Environmental Research Letters 8, no. 2 (2013): 024024. doi:10.1088/1748-9326/8/2/024024

Tol, Richard SJ. "Quantifying the consensus on anthropogenic global warming in the literature: A re-analysis."  Energy Policy (2014). http://www.sciencedirect.com/science/article/pii/S0301421514002821

24 comments:

  1. Sou, the current paper is "in press". The response will most likely be in the same official issue, right after Tol's paper.

    ReplyDelete
    Replies
    1. Thanks, Marco. I'll keep my eyes peeled and add an update when it appears :)

      Delete
  2. I originally made this observation at Skeptical Science but it's worth repeating here.

    "[Tol's] implicit assumption [is] that the causes for categorisation discrepancies in those papers where the scorers initially disagreed, are present at the same statistical distribution within the population of papers where there was initial concordance.

    By the very nature of the scoring process the more ambiguous categorisations would 'self-select' and manifest as discordances, leaving those initial concordances more likely to be so than if the whole population was retested blind.

    Tol appears to be making assumptions of homogeneity where there is quite probably no basis for such. And inhomogeneity is a significant modifer of many analytical processes.
    "

    In other words, in extrapolating the error rates from the originally discordantly-scored abstracts to the rest of the abstracts, Tol appear tp have assumed that they would all have the same likelihood of being mis-scored. A moment spent considering the diversity of commentary in abstracts should blow this assumption out of the water...

    ReplyDelete
    Replies
    1. Exactly, Bernard. I think he might have thought of the categories as some sort of continuous scale with an almost equal likelihood of being in one spot or an adjacent spot on either "side", which is pretty dumb.

      One cannot assume that a change of rating from a "no position stated" would be equally likely to be changed to an endorsement of AGW as a rejection of AGW. There are almost no papers published these days that would reject AGW.

      Delete
  3. It seems that lately more and more comments at WUWT are kicking against the drivel that gets put up there. Could it be that the blowhard consensus is turning?

    ReplyDelete
    Replies
    1. I wouldn't take the comments here to indicate any change of mood at WUWT. It's just that they have Richard pegged as an "alarmist". Poor Richard finds himself in no man's land - ridiculed for antics such as this by people who accept the science and shunned by those who don't.

      I suspect he's where he wants to be - out on his own. I think he sees himself as a maverick. It's not how the rest of the world (ie anyone who's heard of him) would describe him.

      Delete
    2. After Tol's 91% and the BEST fiasco I think the Denialati are firming up against those who toe the line of academia, which is to say are constrained not to just make stuff up. All those rules about sourcing data, describing methods, rational inference and plausible conclusions would wear anybody down, which is the intent. In contrast someone like Monckton can simply claim to have a done a better study finding that only a trace percentage of scientists accept AGW (Oregon Petition algore Benghazi fnord) and have it legitimised in a Wall St Journal editorial. That's the sort of stuff the Denialati need. And the insects guy, he was great, there should be more from the likes of him.

      Play the Establishment game and the Establisment will win. There's a perfectly good old poicy on peer-review, and that's to say it's broken. That's all. Simple and unassailable. "We don't do peer-review because it's broken." Nobody could tol that up, surely.

      Anyhoo, Tol joins Muller in the shadowlands between two realities; the one they really want to be part of and the one with the liberal bias.

      Delete
    3. Sou has it right. You don't draw attention to yourself by agreeing with what everyone already knows. By disagreeing you stand out from the crowd, even if you're wrong.

      Delete
    4. Possibly wishful thinking on my part :)

      Delete
  4. So by Tols bad math, his assumption of a uniform error distribution, he claims that there are ~300 more papers rejecting the consensus of AGW in that 12,000 paper sample? When the Cook et al analysis found a total of 73, Tol thinks there should be 373?

    Surely if there 5x the number of rejecting AGW abstracts than Cook et al found, Tol should have no problem whatsoever finding a few of them as examples?

    Appalling work by Tol - he should be ashamed.

    ReplyDelete
  5. Strange. From my knowledge, this is the first time I see such a strong rejection from WUWT regulars for something that was introduced by Anthony Watts in positive terms.
    Even Watt's dog whistles against Cook didn't work, people attacked Tol.

    I would almost pity the poor Tol. Almost. It looks as if regulars feel they must entrench themselves deeper and reject the "traitors", even if the latter attack the "enemies".

    ReplyDelete
    Replies
    1. In my recent experience, the pseudo-skeptics are fracturing into the crazies who will never change, and the ones who are trying to reposition themselves to accept mainstream science but focus on costs of adaptation vs mitigation. And then there's Tol.

      I think the latter are looking for a soft landing; they want people to associate their names with "genuine concern about the cost of the US responding unilaterally, etc. etc.", not their 10-20 years of deliberate obfuscation. Start archiving threads.

      Delete
    2. I think one cause of the fracture is that the pro shills are positioning themselves to ride out a possible El Nino (and its associated temperature hike) with minimal damage to their credibility. The crazies just don't care.

      Delete
    3. PL, this is what I feel is happening too. As the science continues to pile up it is becoming increasingly difficult for the pseuds to deny it. Cost concerns are really all they have left.

      Delete
  6. The sad thing about Tol is that if one looks at his CV and takes his publications as a prediction, one would not expect to find him writing a paper so exactly in character with the crank rubbish found on WUWT. The quality of Tol's screed on Cook et al is perfectly congruent with WUWT's usual fare and that's an aberration for Tol.

    What happened to this guy, that he'd debase himself so? Not to knock Cook and crew in any way and with a tip of the hat to their meticulous treatment but the methods employed in creating C13 were not rocket science by any stretch of the imagination. Tol's willingness to make himself the butt of jokes over such methodological triviality is very strange. If Cook et al were forming their conclusions by using an untested and controversial system of analysis this might be a matter for intense academic debate, but they were not.

    So why play the crank, over this of all things?

    ReplyDelete
    Replies
    1. "So why play the crank, over this of all things? "

      I'd say that his dignity is stapled to his ideology, which was nudged in the direction that it's taken by his ignorance of systems science outside the scope of his own dismal field.

      Or, to put it another way, he painted himself into a corner using a lovely shade of Stupidity and a D-K brush, and all he can do now is to keep painting as if he knows what he's doing even if others can't fathom it.

      Delete
    2. Well, it seems here we're perhaps seeing an autoimmune disorder of dignity; the very instinct to preserve dignity is destroying it.

      It's just as dismaying to witness as the business w/Curry. Once the gulf of understanding becomes too wide, there's no crossing it, no path back. Doesn't have to be that way, but as they say, "pride is expensive." Expensive for everybody, in this case.

      The climate "debate" degrades everybody it touches.

      Delete
    3. "autoimmune disorder of dignity". After his Guardian comment, this is a good diagnosis. The best quote

      "I show that the 97% consensus claim does not stand up.

      At best, Nuccitelli, John Cook and colleagues may have accidentally stumbled on the right number."

      http://www.theguardian.com/environment/blog/2014/jun/06/97-consensus-global-warming

      Delete
  7. dbostrom's question is very pertinent. From where I sit it seems Tol is playing the man (i.e. Cook) with no interest in the ball at all. Several blogs now carry quotes with him agreeing that the consensus is well north of 90%. So why go after Cook13? He has stated that he does not have the resources to do his own analysis, so why do a seemingly botched job? Is it because some of his previous work has been called into question due to errors that he has to seek errors in the work of others as some way to salvage his reputation? The whole thing is like watching a train wreck in slow motion, you know the outcome is going to be bad but you just can't turn away.

    ReplyDelete
  8. Images?
    train wreck in slow motion,? The image that comes to mind is Jon Voight, in the ending sequence (4 minutes video) from Runaway Train

    or perhaps if one prefers surfing, Patrick Swayze in the Bells Beach (Victoria) finale of Point Break. (start 2:50, key line is at 7:20: he's not coming back.)

    ReplyDelete
  9. If that 24 Errors paper had been directed at me I would never go out in public again. I just couldn't.

    Tol's behaviour has intrigued me since the start of this. When he determined to attack C13 he must have known it wasn't coming from naifs. They knew the usual lines of attack and covered them. Tol should have been psyched up for a challenge, but instead he just blundered in with ill-thought-out claptrap and has been trying to recover himself since.

    His suggestion that if more right-minded people were researching climate change different science would be uncovered is batshit crazy. This is just the old nonsense about there being pro-AGW research and anti-AGW research and the latter can't get funding because algore. The man's just dribbling.

    He has been given a good working-over recently, but he set himself up for it. And now, with the 91%, he's lost the friends he used to have and done the cause no end of harm. It's BEST all over again. Love it to death.

    ReplyDelete
  10. I can't say that I'm not enjoying every part of this Tol fiasco. I'd go on, but everything has been said in the article or the comments.

    ReplyDelete
  11. I used to find the denier comments on sites like WUWT amusing but now they just
    depress me. You see these kinds of statements parroted all over the web by people
    who are apparently impervious to independent thought or reason.

    Sad........

    ReplyDelete
  12. It's purely a happenstance enthusiast matter for a lot of so-called deniers; the particular issue isn't even that important. If they were not in the stands doing the rah-rah for Anthony Watts, many deniers would be looking for stimulation somewhere else. We see this all the time; ordinary folks go into mortal combat over football teams. They willingly debase themselves for millionaires, for nothing in return but some adrenalin.

    For others I think Sou has it right; it's fear we hear. Ordinary people who feel as though they've no control over their lives and no sense of security find AGW to be the cathartic punching boy for their anxieties.

    I'm fairly certain that with a few of these people the anger we hear isn't even about the issue at hand. Their rage is because they feel compelled to express themselves yet they sense they can't put together a coherent sentence let alone a whole paragraph and are an embarrassment. Like kids learning how to tie their shoelaces; faces turn red and tears flow.

    ReplyDelete

Instead of commenting as "Anonymous", please comment using "Name/URL" and your name, initials or pseudonym or whatever. You can leave the "URL" box blank. This isn't mandatory. You can also sign in using your Google ID, Wordpress ID etc as indicated. NOTE: Some Wordpress users are having trouble signing in. If that's you, try signing in using Name/URL. Details here.

Click here to read the HotWhopper comment policy.