Anthony has probably spent a heap of time copying an image of The Consensus Project, trying to argue that an opinion survey of members of the American Meteorological Society can be equated with a survey of scientific literature by Cook et al. And this after goodness knows how many WUWT protests that "opinions don't count". (Anthony's probably been working on this article for a while because the paper came out a little while back.)
Anthony Watts hasn't even read the abstract of the Cook13 97% consensus paper
What is really weird is that Anthony Watts hasn't even read Cook13, the paper that he's made umpteen protests about. How do I know that? Simple. This is what Anthony has written - at least four times. And he's wrong each of those times:
You see, it turns out that Cook simply employed his band of “Skeptical Science” (SkS) eco-zealots to rate papers, rather than letting all authors of the papers rate their own work
And again:
Most people who read the headlines touted by the unquestioning press had no idea that this was a collection of Skeptical Science raters opinions rather than the authors assessment of their own work. Readers of news stories had no idea they’d been lied to by John Cook et al².And again:
...The result was that the “97% consensus” was a survey of the SkS raters beliefs and interpretations, rather than a survey of the authors opinions of their own science abstracts.
And again:
Had Cook actually done an honest survey, we’d have the opinions of the authors about their papers, not the opinions of the SkS pal review squad in place of those opinions.
Anthony Watts has always had a tendency to leap first look later. In the case of Cook13, Anthony has leapt about twenty times or more and looked not once. It looks as if he "read the headlines" and not the paper.
Although he has no excuse for not reading the paper itself because it's open access, Anthony wouldn't even have had to read the whole paper to know that he was wrong. He could have just read the abstract - from Cook13:
. ..In a second phase of this study, we invited authors to rate their own papers. Compared to abstract ratings, a smaller percentage of self-rated papers expressed no position on AGW (35.5%). Among self-rated papers expressing a position on AGW, 97.2% endorsed the consensus.If Anthony Watts had bothered to glance at the full paper he would have seen (my bold italics):
To complement the abstract analysis, email addresses for 8547 authors were collected, typically from the corresponding author and/or first author. For each year, email addresses were obtained for at least 60% of papers. Authors were emailed an invitation to participate in a survey in which they rated their own published papers (the entire content of the article, not just the abstract) with the same criteria as used by the independent rating team. Details of the survey text are provided in the supplementary informationIn Anthony's eagerness to embrace the opinions of meteorologists, particularly those who've never studied climate or done any research on climate, he shows once again that he is:
- is a stubby short of a six pack
- has a few roos loose in the top paddock
- is thick as a brick...
You get the idea.
Acceptance of AGW increases with increased knowledge
Anthony is delighted that he's found a paper showing that there are quite a few meteorologists who don't do any scientific research and who don't think that humans are affecting the climate to any great extent. (AFAIK in the USA, even a television weather announcer can call themselves a "meteorologist", and they are likely to have had some post-secondary education in the science of weather at some stage. AMS members these days would probably mostly have at least a bachelor degree in science. Maybe a reader from the USA can shed some light on the situation.)
Most of the AMS members who have done research say they know that climate change is real and that humans are causing it. But that latter finding just supports Anthony's conspiracy theory. He's a nutter of the 'climate science is a hoax' variety - and getting more entrenched as time goes by. Anthony twists deeper knowledge of climate science into "fudging results" to suit some supposed funding body objective. Why any government or funding body would want to fabricate such a problem is answered by the "new world order" conspiracy theory, which merges with the anti-semitic "evil banker" conspiracy theory and the "fiat money/someone stole all the gold in Fort Knox" conspiracy theory.
It's a tangled web of paranoid conspiracy theories in climate science denier land.
WUWT is "unreliable" - changing articles and censoring comments
As the thread grows, Anthony gets increasingly stroppy with anyone who points out where he went wrong. Funny thing is that when people do point out in the comments that the opinions of scientists on their own work was obtained, Anthony backtracks and edits his original article changing this (archived here):
You see, it turns out that Cook simply employed his band of “Skeptical Science” (SkS) eco-zealots to rate papers, rather than letting the authors of the papers rate their own work. The result was that the “97% consensus” was a survey of the SkS raters’ beliefs and interpretations, rather than a survey of the authors opinions of their own science abstracts.To this where he added the bolded italics sentence (archived here):
You see, it turns out that Cook simply employed his band of “Skeptical Science” (SkS) eco-zealots to rate papers, rather than letting all authors of the papers rate their own work (Note: many authors weren’t even contacted and their papers wrongly rated, see here). The result was that the “97% consensus” was a survey of the SkS raters beliefs and interpretations, rather than a survey of the authors opinions of their own science abstracts.
Aha - so he's now tentatively admitting that authors were contacted - but he's not properly correcting his article or comments.
From the WUWT comments
It didn't take long before someone brought up another "opinion poll" - the fraudulent "petition" known as the Oregon Petition. . dbstealey says:November 20, 2013 at 11:11 am
As I’ve often said, you couldn’t get 97% of Italians to agree the Pope is Catholic.
Anyone who believes that 97% of scientists think human activity is the cause of global warming appears to be ignorant of the OISM Petition, in which more than 30,000 American scientists co-signed a statement saying that more CO2 is harmless, and beneficial to the biosphere.
There is nothing comparable from the climate alarmist crowd, whose numbers are much smaller than generally assumed.
itooktheredpill is a fake sceptic who didn't bother reading Cook13 either, not even the abstract, and says, quoting Anthony Watt's fib:
November 20, 2013 at 11:14 am
You see, it turns out that Cook simply employed his band of “Skeptical Science” (SkS) eco-zealots to rate papers, rather than letting the authors of the papers rate their own work. The result was that the “97% consensus” was a survey of the SkS raters’ beliefs and interpretations, rather than a survey of the authors opinions of their own science abstracts.Those who can’t handle the truth,
try to silence those who speak it.
Cook effectively silenced the authors’ beliefs by replacing them with his SkS raters’ beliefs.
Cook can’t handle the truth.
Anthony wouldn't brook any facts in the comments. He got very irate when Dumb Scientist asked some pertinent questions. Dumb Scientist says:
November 20, 2013 at 11:28 am
Anthony Watts: The 97% consensus myth – busted by a real survey … We’ve all been subjected to the incessant “97% of scientists agree …global warming…blah blah” meme, which is nothing more than another statistical fabrication by John Cook and his collection of “anything for the cause” zealots … Cook simply employed his band of “Skeptical Science” (SkS) eco-zealots to rate papers … a lie of omission … they’d been lied to by John Cook et al². … we’ll be fighting this lie for years … blown Cook’s propaganda paper right out of the water.==================================
Isn’t a survey of opinions different from a survey of scientific abstracts? If so, how can an opinion survey show that a survey of scientific abstracts is a “lie”?
Note that 78% of meteorologists who publish mostly on climate agree that the warming is mostly human-caused. Only 5% of all meteorologists claim that the warming is mostly natural, and only 4% claim that the warming isn’t happening.
If we can agree about these facts, that’s great news!
True to form Anthony demonstrates bullying and cowardice. First off, Anthony doesn't give any real sceptic the courtesy of using his screen name. He figures 'naming and shaming' people will intimidate them and scare them away. (After all, what self-respecting person would admit to visiting a site like WUWT!). It's not as if he's a dinky di fake sceptic like itooktheredpill above.
REPLY: Bryan, This response suggests you are simply concern trolling. Had Cook actually done an honest survey, we’d have the opinions of the authors about their papers, not the opinions of the SkS pal review squad in place of those opinions.Concern trolling? The questions and statements were matter of fact, on point and courteously expressed. Anthony must really be feeling under the hammer, mustn't he.
For more shenanigans related to SkS, you might look up the sort of pea and thimble switcheroos (they didn’t survey skeptical blogs) and statistical techninques (populations of N=0 are allowed, add your own interpretation) employed by the gang that couldn’t shoot straight when it came to their published opinions on skeptics and their supposed belief in “faking the moon landing”. What you have here with SkS is an organized propaganda team. They aren’t interested in science.
– Anthony
Anthony couldn't cope with much more from normal people. Heck. That's not what WUWT is all about. WUWT is for people who don't read abstracts. People who don't check up on Anthony's fibs. People who embrace wacky conspiracy theories and all the utter nutter fake sceptics. There is no room for reason, fact-checking or science on WUWT.
Since bullying didn't work, Anthony takes the coward's way out. Anthony got out his censorship keyboard and went haywire any time he saw a comment that pointed out where he got it wrong. For example. when Dumb Scientist suggested that WUWT do its own survey, Anthony "snipped" it. At least he left the follow up comment though not without another whine:
Dumb Scientist says:
November 20, 2013 at 12:55 pm
[snip - not interested in your characterization of me - Anthony]
==========================================
Just a few days ago, dbstealey and Ferdinand Engelbeen drew my attention to the WUWT sidebar which criticizes SkS for deleting user comments and noted that this means SkS is “dishonest”.
So it’s disappointing that my comment was snipped, but even more disappointing that Anthony claims it was because of a “characterization” after he accused John Cook and other SkS authors of dishonesty.
I still think it’s possible that Anthony has the integrity to not snip this comment, so I’ll repeat my challenge that got snipped earlier: “I’d be very interested to see WUWT read through 10,000 scientific abstracts and rate them. You could show the world how to do a proper survey… right?”
REPLY: Oh please. Bryan for the record, I don’t give a rats ass about what you think about comment policy (see here). You put words in my mouth in the last comment, I snipped it because of that. Get over yourself. Why don’t you get your peers at JPL to do it, if it is so important to you? After all, you’ve got millions of dollars of government money at your disposal there and we have next to nothing.
The whole consensus chasing is a waste of time in my opinion, Mother Nature will be the final arbiter of the AGW issue- Anthony
For a "waste of time" Anthony has sure spent a lot of time and written an awful lot of articles in protest :) And he complains he can't be bothered surveying the literature because he wouldn't get paid for it! I guess the Heartland Institute don't see it in their best interest to have Cook13 confirmed yet again. John Cook and his team looked at more than 10,000 abstracts in their own time without being paid a penny. But Anthony Watts isn't interested in doing it because, as he knows, he'd end up with the same result as Cook13.
There are a number of people who've noticed that although the uninformed (non-research) meteorologists are less likely to attribute climate change to mostly human activities, only 5% have said that "it's natural".
I'm on the road so won't write any more on this right now, but I see there is lots more fun to be had with articles put up at WUWT in the past few hours. So I'll be back soon.
Meanwhile, you can read the various WUWT archives on this zany article here in order: First, second, third, fourth. The comments are a mix of the rational through to the utter nutters. Anthony Watts is leaning towards the utter nutter end of the spectrum but there are quite a few who surpass him.
Neil Stenhouse et al (2013), Meteorologists' views about global warming: A survey of American Meteorological Society professional members, Bulletin of the American Meteorological Society 2013 ; doi: http://dx.doi.org/10.1175/BAMS-D-13-00091.1
That survey had a 26% response rate - hardly representative of anything. And in other fields (e.g. health, where systematic reviews like Cook13 are standard practice) it is completely normal for the authors to rate papers. The illiterati at WUWT really don't have a clue about anything ...
ReplyDeleteAnonymous ignoramus.
Delete1200 responses pertaining to 2100 paper from 8500 requests is very much a sufficient sample size for adequate representation unless there is a peculiar statistical bias, and the congruence with the Cook et al results is strong evidence that there is no bias.
If you have a statistical justification that supports your claim please detail it.
Bernard J.
Bernard, I think you have read the comment wrongly. Anonymous was referring to the AMS survey which had a response rate of 26% out of 7062 valid email addresses. Only 14% responded to the Cook survey out of 8547 emails sent. Cook13 doesn't say whether all email addresses were valid. If, as is likely, many weren't valid then the percentage responses aren't comparable. (Cook13 went back 20 years so it's highly likely there were a lot of invalid email addresses in the thousands they compiled.)
DeleteIdeally you'd get more responses, but there are two measures I'd typically use - one is the total number of responses - more than 300 and you should have a reasonable sample. Both scored well over that number of responses. The other is the percentage. Anything over 10% response is probably okay, although 30% would be better.
Ah, I see. Unreserved apologies.
DeleteI guess that I am distracted by the events that just unfolded in Warsaw.
:-(
Bernard J.
I was speaking of the AMS thingy (sorry that wasn't clearer). 14% in Cook13 is pretty poor (I didn't know about this,btw) but that's a completely different kettle of fish - their survey was part of a bigger exercise (systematic review) that has a great deal more weight, and as Sou observes, that "14%" is actually a much bigger percent of a smaller pool of still-functioning email addresses.
DeleteLoss of validity due to poor response rates is not a statistically provable thing. It seems fairly obvious to me that the only people who would respond to the AMS survey would be the politically engaged, leaving a huge horde of unengaged accepters of the status quo who had better things to do than return the questionnaire - and those non-responders would, I guess, be largely accepting of the scientific consensus.
You won't get into the Lancet with a 26% response rate, that's for sure ...
Yes, the numbers I quoted (300 minimum responses, 10% or better response rate for the whole population) were for an indicative market research type survey where you're often lucky to get that, depending on the size of the total population you want to measure.
DeleteIf it's an entire nation you'd go for more than 300. Polls on political issues, for example, involve maybe 1000 people out of 300 million plus in the USA. On the other hand, Australian Bureau of Stats might involve phone calls to 10,000 or more people in some of their statistical surveys such as labour market research. There is more detail needed for the ABS surveys and more precision, too.
I don't know what sort of survey's they'd do for Lancet. Are they opinion surveys? I don't imagine a survey of medical practitioners would get much better than a 30% response rate on any normal issue where they are just asked questions out of the blue without it being pre-arranged. It'd be different if it's targeted research with people making a commitment to respond.
The Lancet will publish things like household surveys where people do house to house visits, taking blood and doing blood pressure readings etc (the most famous of late is probably the Iraq survey). Those kinds of surveys often have 98% response rates. The issue is not the final sample size, but the reason that people don't reply, e.g. opinion polls done by phone almost always under-sample young urban singles, because they don't have a landline and don't do surveys if they're called. There are ways around that (over-sampling etc.) but something like the AMS survey can't adjust for that (since they only have the names they have, they can't just expand the sampling frame).
DeleteThis is why I think the systematic review part of Cook13 would carry more weight - there is no non-response bias with a survey of the literature! (Though I'm sure if Watts ever got off his arse to try and replicate Cook13, he'd find a way to miss a whole bunch of papers...)
In a real sense though a house to house survey would only sample people who are at home wouldn't it. Even if they called back once or twice I don't imagine that they would ever get, say, 98% of a population living in a particular area. The 98% would be more likely a return of people sampled than 98% of a population as such, wouldn't it?
Delete(I cannot imagine anyone getting 98% of people asked to agree to give a blood sample. Does this actually happen? Even if there were a captive population in a hospital who were asked permission to take a sample, would you ever get 98% agreeing unless there were some massive incentive - like all the people had a disease for which a cure was being sought.)
It happens all the time in developing nations. One of the criticisms leveled against the Lancet mortality surveys by right-wing pro-war loonies was that the response rate was too high, but it's pretty normal to have very high rates - even with blood tests. See e.g. the Demographic and Health Surveys. I don't think people understand exactly why people in low income countries are so much more cooperative than rich westerners, but it's a thing ... in Africa, for example, they can get very good response rates (not quite 95%, granted) for surveys that do HIV tests...
DeleteYou're right that the 98% is on households sampled, not people in the area, but with good call backs and persistence it can be close to teh same thing. In that sense house to house surveys are better than phone polls - everyone has a house (usually), with a door; not everyone has a phone, or an accessible one at least.
And postal surveys are notoriously biased because you can't control any of this stuff...
Well, I've learnt something new out of the blue today :)
DeleteThanks for that info. I expect it must also be a credit to the people taking the samples. Building up trust in the community etc.
(I meant, normal for the authors of the review to rate the papers they are reviewing, without consulting with the authors of the papers).
ReplyDeleteI recommend the following paper for a laugh:
Deletehttp://onlinelibrary.wiley.com/doi/10.1002/jmr.1004/abstract
("Michael Jackson data")
And this one to show a rating by others:
http://onlinelibrary.wiley.com/doi/10.1002/jmr.1138/abstract;jsessionid=AA26C535D2E4BBFE7292B50721BD8B0E.f03t03
Not quite the same as Cook et al 2013, though...
Marco
Elevator doesn't go to the top floor.
ReplyDeleteLights are on but nobody's home.
One taco shy of a combination plate.
One grasp short of a fact.
Dang, I forgot: Blind in one eye and can't see out of the other.
DeleteYou missed the main one ... effing idiot.
DeleteI'm suspect Watts merely understands well that if he asserts something the flock wishes to believe frequently enough, that is what they will believe.
ReplyDeleteThis would all be embarrassing or a cause for consternation in the reality-based community, in Watts' world it's just Wednesday...
I also noticed the change Watts made to his blog post when I was writing my response:
ReplyDeletehttp://www.realsceptic.com/2013/11/21/97-climate-science-consensus-reality/
Why would he make an accusation that's so obviously false with him subsequently ignoring remarks that he got this wrong? The change he made to the article I don't even see as a correction as it doesn't remove the false accusation. It also doesn't provide an apology for incorrectly representing Cook et al.
I also find this quite hypocritical considering how Watts responds to critics when he thinks they are falsely accusing him.
Australia's shame, shame, shame:
ReplyDeletehttp://www.theguardian.com/global-development/2013/nov/20/climate-talks-walk-out-compensation-un-warsaw
Bernard J.
Once again, thanks for documenting these interesting times we live in, Sou.
ReplyDeleteI would point out that the 26% AMS and 14% Cook et al 2013 response rates are entirely reasonable, from what little I know of response rates.
ReplyDeleteThis source, for example, states that:
"Internal surveys (i.e. employee surveys) generally have a much higher response rate than external surveys (e.g. customer satisfaction surveys). Internal surveys will generally receive a 30-40% response rate or more on average, compared to an average 10-15% response rate for external surveys.
Response rates can soar past 85% (about 43 responses for every 50 invitations sent) when the respondent population is motivated and the survey is well-executed. Response rates can also fall below 2% (about 1 response for every 50 invitations sent) when the respondent population is less-targeted, when contact information is unreliable, or where there is less incentive or little motivation to respond."
The AMS survey was closer to an internal survey, although diluted by being a professional organization rather than a business with people directly working together. The Cook et al survey received a very respectable response rate for an external survey, particularly if the failed percentage includes unworkable email addresses.
People denigrating such results based on response rates (and there are a few) don't appear to be very familiar with actual surveys.
KR
Your numbers are consistent with the rule(s) of thumb I've always used when conducting surveys of various kinds, KR. (I haven't ever conducted a survey where response rate was less than 10%. I think about the lowest I've got would have been 25-30%. But that's in part because of the type of surveys I've done. Also they've been managed carefully.)
DeleteMotivation is key (importance as perceived by the participant, including 'trust', relevance and priority) and more important than any external incentive such as a freeby giveaway to participants. Though an incentive can help when motivation would otherwise be expected to be very low. The next most important thing I've found is follow-up reminders/requests to solicit more responses.
KR
ReplyDeleteI think the problem with the AMS survey isn't the percentage response but that the questions were poorly worded and the response is skewed. Tom Curtis goes into the detail over at Wotts'.
Although Tom raises some relevant points, I think it's nitpicking. His objections come across as trying to discredit the work because he doesn't like the results. I know Tom wouldn't deliberately act like that. He has a lot of integrity. But that's how it comes across.
DeleteThe responses are what they are and given the response rate I doubt that any other AMS survey would have resulted in markedly different results.
The different issue is what is actually happening with the climate and what is causing climate to change. Whether or not there are pockets of the AMS members who don't accept the science, it doesn't change the facts about climate. Hopefully the survey results will help people who are educating the up and coming meteorologists, as well as AMS itself as far as it takes on the role of informing its members about the science.
I think it was a useful survey and shows up the fact that there is a proportion of AMS membership who aren't informed about climate science. Mostly the people who don't study it. BAMS has more than enough good science to keep its members informed. So it's not for lack of information. I'd guess many of those who reject or don't accept the science don't bother to read BAMS.
The research pointed to the reasons why a proportion reject science:
In a survey of American Meteorological Society members, perceived scientific consensus was the 54 strongest predictor of global warming views, followed by political ideology, climate science 55 expertise and perceived organizational conflict
Some of the same reasons as the general public.
I must admit that was my take, Sou. The paper is explicit: contrarianism is strongly correlated with rejection of the scientific consensus because of poor topic knowledge and political stance. Not exactly news, and nothing to do with the physical basis for forced climate change.
DeleteAnthony Watts can't see that he's doing his 'cause' a disservice by publicising the AMS survey. The survey shows unequivocally that those who reject human-caused climate change tend to be uninformed or misinformed. His blog post and the comments that follow it support this conclusion and clearly demonstrate how one follows from the other.
DeleteWhile Nero Watts fiddles...looks like methane levels thru the roof...
ReplyDeletehttp://arctic-news.blogspot.com.au/2013/11/methane-levels-going-through-the-roof.html