Friday, April 25, 2014

No need for fury: Ethics and internet research


After all the fuss about Recursive Fury, I did some reading on ethics as it relates to internet research. There is quite a bit of material available. Below is some of what I found. These are from international literature and I'm not claiming they would apply in all jurisdictions.

The short answer is, however, that if what I've read is any guide, there was no breach of ethics by the authors of Recursive Fury in preparing or writing the paper, and none by its publication. (I can't say the same for its withdrawal by Frontiers. I didn't research that aspect however the behaviour of the "Chief Editor" did not come across as very professional to say the least.)

No diagnosis


Complaints about Recursive Fury were largely centred on two themes.  Firstly, that the study "pathologised" people who made comments.  This took the form complaints such as:

Yet the paper itself contained no diagnoses of individuals, let alone one that attributed a "mental disorder" or "psychological affliction". (Not that I've noticed Anthony Watts being in absentia, nor would I think too many people would consent to having a psychological affliction.)


No human subjects


Another objection raised related to whether informed consent was necessary.  Whether or not informed consent is required from an ethical standpoint, the first condition is that the research relates to "human subjects".  The National Science Foundation has a policy: 45 CFR Part 690: Federal Policy for the Protection of Human Subjects, in which it defines "Human Subject" as follows:
 (f) Human subject means a living individual about whom an investigator (whether professional or student) conducting research obtains 
(1) data through intervention or interaction with the individual, or
(2) identifiable private information.
Intervention includes both physical procedures by which data are gathered (for example, venipuncture) and manipulations of the subject or the subject's environment that are performed for research purposes.
Interaction includes communication or interpersonal contact between investigator and subject. 
"Private information" includes information about behavior that occurs in a context in which an individual can reasonably expect that no observation or recording is taking place, and information which has been provided for specific purposes by an individual and which the individual can reasonably expect will not be made public (for example, a medical record). Private information must be individually identifiable (i.e., the identity of the subject is or may readily be ascertained by the investigator or associated with the information) in order for obtaining the information to constitute research involving human subjects.

In the case of Recursive Fury, although some of the textual material came from the blogs of the researchers the comments arguably were not as a result of any such communication or interpersonal contact but rather were because of the publication of the "moon landing" paper, LOG12.  The research certainly did not meet the definition of "intervention". There was no venipuncture or any manipulation of the subject or the subjects' environment. Nor did it meet the definition of "private information".  The blogs from which the information was gathered were not merely public blogs, rather than private, they were much publicised public blogs and, according to Alexa rankings, were the more popular blogs in their sector of interest and, indeed, on the internet as a whole.  Blog owners did not require readers to log in.  There was no expectation of privacy. In addition, it is obvious from reading the blogs that many of the blog owners (and commenters) crave publicity, promoting new blog articles on Twitter for example.  One blog owner boasts of wide readership.

Therefore the complaint that prior consent was required fails before passing the first hurdle, that of "human subjects".


Waiving requirements


Given that the textual analysis did not involve human subjects as per the definition above, the following waiver does not apply.  Still it's worth reading.  This is from the same National Science Foundation policy:
(d) An IRB may approve a consent procedure which does not include, or which alters, some or all of the elements of informed consent set forth in this section, or waive the requirements to obtain informed consent provided the IRB finds and documents that: 
(1) The research involves no more than minimal risk to the subjects;
(2) The waiver or alteration will not adversely affect the rights and welfare of the subjects;  
(3) The research could not practicably be carried out without the waiver or alteration; and  
(4) Whenever appropriate, the subjects will be provided with additional pertinent information after participation.  
(e) The informed consent requirements in this policy are not intended to preempt any applicable federal, state, or local laws which require additional information to be disclosed in order for informed consent to be legally effective. 
(f) Nothing in this policy is intended to limit the authority of a physician to provide emergency medical care, to the extent the physician is permitted to do so under applicable federal, state, or local law. (Approved by the Office of Management and Budget under Control Number 9999-0020.)
A strong argument could be mounted that requirements (1) and (2) would have been met in this study and (3) would arguably apply as well. That's if the policy had applied to this research, which it didn't because the entire policy only relates to "human subjects".


Public vs Private

Here's a snippet from a paper by a PhD student in Sweden, which discusses the distinction between public and private in the context of internet research ethics (my paras and bold italics).
Without reducing the complexity of problems and previous arguments, it should be noted that the private/public distinction is a recurring theme in IRE [Internet Research Ethics] debates. More precisely, in online contexts, the boundaries between private and public appear blurred (Bromseth, 2002; Löfberg, 2003; Mann, 2003; Thorseth, 2003; Sveningsson Elm, 2009). This makes it difficult for researchers to assess the sensitivity of information and situations, It also makes it difficult to determine when research requires informed consent.
Attempts to solidify the private/public status of online phenomena have been made from various ontological and epistemological viewpoints, but arguments tend to follow one of two lines of reasoning: online phenomena can be considered public either
(1) if publicly accessible or
(2) if perceived as public by participants (Bromseth, 2002; Chen, Hall, & Johns, 2004; McKee & Porter, 2009; Sveningsson Elm, 2009).
According to the first argument, online phenomena are essentially public if they can be accessed by anyone with an open Internet connection. Moreover, public discourse must always be open for scholarly analysis and critique, and, in lack of restricted entrance, there is no need for consent or even anonymizing. The second and often counter-posed view holds that, though something may be accessible, the general public (including researchers) may not be the intended audience.
Researchers must therefore base their ethical decisions on a community’s purpose and participants’ expectations of privacy. Without taking consideration to personal privacy, researchers might instigate feelings of intrusion and exposure, or attract unwanted attention to online communities. My description of the above positions is, of course, a simplification.

In the context of the above, the material for Recursive Fury was public in that there was no restricted entrance nor was there any expectation of privacy on the part of people commenting.  Quite the contrary. An examination of comments shows that some of the same people posted their ideas on multiple public high profile blogs, indicating the the authors wanted as many people as possible to read them.  Indeed since Recursive Fury was published, some commenters have been posting their objections / conspiracy ideations on every site they can find which mentions the words "Lewandowsky" or "Recursive Fury".  (In some cases, so quick is their response that one might surmise they have a Google alert notification set up for these words.)


Rosenberg, A. "Virtual world research ethics and the private/public distinction." International journal of Internet research ethics 3, no. 12 (2010): 23-36.

Lewandowsky, Stephan, Klaus Oberauer, and Gilles E. Gignac. "NASA Faked the Moon Landing—Therefore,(Climate) Science Is a Hoax An Anatomy of the Motivated Rejection of Science." Psychological science 24, no. 5 (2013): 622-633.

Lewandowsky, Stephan, John Cook, Klaus Oberauer, and Michael Marriott. "Recursive fury: conspiracist ideation in the blogosphere in response to research on conspiracist ideation." Frontiers in psychology 4 (2012): 73-73.

26 comments:

  1. One of the many inconsistencies that strikes me in this matter is that the Denialati from WUWT are vociferous in demanding data and private correspondence from scientists but when it comes to their own very public (as reinforced above) opinions and statements it's time to slam the barn door.

    It would be impossible to invent such hypocritical self-parody, even if one was conspiring to perpetrate a scientific conspiracy in support of the Sinister Global Climatati...

    ReplyDelete
    Replies
    1. As you say, Bernard, that's just one of the many inconsistencies :o

      Another is *falsely* attributing comments *stolen* from a *private* forum to one of the authors. And then falsely complaining about ethics - and wailing that their *very public* comments that they wanted all the world to see and take notice of were actually taken notice of. That's ... well, try to count the multiple inconsistencies in that..

      Delete
  2. I do have some sympathy for the editors at Frontiers who caved: one thing climategate showed us is that deniers are very nasty people. And they have the backing of a trillion dollar industry, which makes threats from them rather scary.

    So, as somebody who posts anonymously, I will reserve any criticism and simply state my admiration for people like Lewandowsky who are prepared to take them on.

    ReplyDelete
  3. Did Lewandowsky et al seek advice from an Institutional Review Boards (IRB) as to whether the methods to be used in their study constituted human subjects research and would therefore require IRB approval? I have quickly looked at their methods section but could not find anything relating to the involvement of an IRB. I'd be interested to see how an IRB would handle this type of research proposal. Its not the conventional sort of human subjects study but perhaps the authors might have been better served by coding the websites etc so that no names/sites could be easily identified. I think much of their data would have survived such treatment. Of course their methods would need to be written in such a way as to not allow an outside party to guess the identity of website and/or individuals; being able to identify themselves is something that I'm sure would enrage the pseudoskeptics.

    ReplyDelete
    Replies
    1. The authors got all the relevant approvals, obviously. Mike. They would know the relevant ethical guidelines much better than I.

      This article was not to second guess the authors or the funding body or the UWA who all approved the project.

      It was to counter the claims of denialists that the work or the paper was unethical, that the deniers should have been asked first, and that they were diagnosed with a dreadful psychological disorder - none of which are true.

      Delete
    2. Mike, I addressed this subject at the end of March.

      The short answer is that there are both institutional and funding body mandates requiring ehtical/procedural review of this project, and such reviewing was done.

      This is one reason why the UWA lawyer is so confident about the paper - he knows that the process is kosher.

      Delete
  4. This is a silly post that doesn't deal with the serious issues raised about Fury at Lucia's and Climate Audit. There was extensive interaction with the "human subjects" by the authors of the paper and in fact a lot of taunting of them. Please, are you the mirror image of WUWT or an actual adult? Frontiers is right to have retracted this paper as its methodology was flawed and its ethics questionable.

    ReplyDelete
    Replies
    1. I've no idea what "serious issues" you are talking about. I rarely visit denier websites (except for WUWT). Going by the silly comments full of some of the best unfounded conspiracy ideation, which I've read at Frontiers and Shaping Tomorrows World, I'm sure they could be classed as a mix of persecuted victim, nefarious intent, something must be wrong - at the very least.

      Your claim doesn't hold true. You are simply making it up. Blog articles about LOG12 and a couple of clarifying comments to the public at large does not constitute "extensive interaction" or interaction of any kind. That would be the same as saying that The Age or the New York Times "interacts" with you whenever you read an article in the paper.

      As for "taunting" - if you are trying to say something along the lines: "Stephan Lewandowsky explained ln his blog that our conspiracy theories about LOG12 weren't true and had no basis in fact" as taunting, then are you trying to argue that he made you "not believe" him even more because of course he had some evil motive and was trying to deceive you?

      Poor little *persecuted victim* you, you wrongly reckon you were being "taunted" by someone having *nefarious intent* because *something must be wrong* - sounds like you have an *over-riding suspicion* and using *self-sealing reasoning* to my way of thinking.

      Delete
    2. Oh, and as for ClimateAudit - I have read quite a bit of dozens of articles full of false allegations that Steve McIntyre made about Professor Lewandowsky in particular (the lead author). (A good example of the application of the Serengeti Strategy by Steve McIntyre by the way).

      For McI to then take offence that his very public comments (arguably highly unethical) that he wanted all the world to see and take notice of were taken notice of, strikes me as the height of hypocrisy.

      And was it Lucia's blog or was that somewhere else, where people were falsely attributing comments made by someone else to one of the RF authors - comments that were stolen from a private website?

      And you think you or those blog owners understand ethics or have any regard for ethics?

      If you don't believe me that neither the paper nor the research breached any ethical standards, take it up with UWA lawyers who agree they were in compliance with ethical guidelines and standards.

      Delete
    3. Lett us consider Bradley Copies Fritts, a dandy case of a blog post whose only plausible purpose was to damage Ray Bradley's reputation in the blogosphere. Google that title. It spread fast.

      McIntyre painstakingly showed the Bradley had plagiarized Harold Fritts by including many of his images ... but had "neglected" to read the pages of dense Acknowledgements/Permissions that covered every one. he also showed himself totally clueless about the construction of textbooks. Fritts was furious, but not at Ray.

      Delete
    4. [Correction]Although Lucia did quote comments stolen from the private website, it was Steve McIntyre who was quoting comments made in private by *someone else* as some sort of evidence that one of the RF authors, John Cook "sanctioned vile commentary against both me and Anthony Watts" - comments that were stolen from a private website.

      This despite the fact that it was not John Cook who made the comments. It was on a private website, not in the public domain. And the comments were stolen!

      And it gets worse, this was after Steve's myriad articles full of his own "vile commentary" in which he arguably libeled the authors of LOG12 and Stephan Lewandowsky in particular. As Graham Readfearn discovered:

      In McIntyre's complaint letters (seen as item numbers 95 and 99 on the FOI document release), the Canadian blogger uses quotes hacked from a private forum of the Skeptical Science, founded University of Queensland academic John Cook and co-author on the Recursive study.

      McI even posted a link to this claim on his own website.[End correction]

      Delete
    5. "...and in fact a lot of taunting of them."

      Oh please...

      Is that the best you can come up with? You may as well fart in our general direction.

      Delete
    6. I am going to be a bit contrarian on this (and not for the first time): in his posts on STW Lewandowsky taunted several of the people who criticized LOG12 by snide remarks. Note that I do not mind the remarks as such, he has every right to mock them. However, such mocking does tend to fuel the fire, and therefore it was inappropriate, in my opinion, for Lewandowsky to do the analysis of the response to LOG12.

      So, yes, Bernard, in my view the "taunting" is actually a strong argument against Recursive Fury, as it violates, if not the letter, at least the spirit of the ethical approval and common morality in doing this type of research.

      Delete
    7. The next paper will be able to research a new category of conspiracy ideation: "they made me do it" = TMMDI.™ (refer comment at 17 Apr 2014 at 07:25pm)

      By explaining the facts and showing the various conspiracy theories were wrong and contrary to the facts, the authors forced the deniers to come up with new and innovative conspiracy theories. After all, the authors had "nefarious intent", the deniers were "persecuted victims" therefore "something must be wrong" - and, by golly, the deniers were going to ferret out what it is by hook or by crook.

      Delete
    8. Marco, I respectfully disagree. Other bloggers may have taunted (and who could blame them), but reading the text of Stephan's article's they included some wry commentary at worst, not taunting. And they signified effort on his part to quell the conspiracy ideation and combat the claims of "fraudulent", "scam", "fake" etc that McIntyre in particular was making. Stephan chose to write a series of blog articles rebutting each false allegation as it emerged - on his own blog - rather than engage in any direct slanging match, for example, on McIntyre's dreadful blog. The latter would have served no purpose and would not have squashed the silliness going on elsewhere in the deniosphere.

      3 September

      4 September

      6 September

      7 September

      9 September

      10 September

      12 September

      More here.

      This one from 19 September is probably the article that could be described as "taunting" at a pinch, but only if one wasn't aware of the myriad allegations of deception and scam that McIntyre had been flinging about with gay abandon, where he pointed out the flaws in McIntyre's "stats". But is it sufficient to exclude Professor Lewandowsky from doing the research? I don't believe it is.

      Delete
    9. I think we are going into semantics now on what "taunting" really means. Is the following just "wry", or does it contain a "taunt"?
      "As we are awaiting the decision about release of the names, just a matter of general principle, there can be no harm if those folks were to again check their inboxes (and outboxes) very carefully for correspondence from my assistant at UWA in August and September 2010. I know how difficult it is to locate individual emails among thousands received in a year, and a double check may therefore be quite prudent. (Who knows, it might even prevent some overly trigger-happy and creative people from floating a conspiracy theory about how I just made up the fact of having contacted those blogs, similar to the way NASA faked the moon landing.)"

      Or this:
      "Should any others want to continue searching their correspondence, it might be helpful to know that my assistant has just re-read old correspondence from some time ago (e.g., from Thu, 23 Sep 2010 08:38:33 -0400) with considerable amusement in light of the frivolous accusations flying about the internet that we may not have contacted those blogs with a request to post a link."

      Or this:
      "Oh how nefarious! I reported data only 3 days after contacting a blogger to collect data!
      Never mind that the first theory claimed I never contacted anyone. That's sooooo 2011. Let's move on to the next conspiracy.
      Only 3 days and I reported data from 1100 subjects. The travesty of it!"

      Or this:
      "But don't let me stop anyone staring at that shiny object, it's been approximately 666 + 45 days since Mr McIntyre ignored my email, and the cube root of 666+45 is, after all, 8.925307759554336."

      And the most obvious taunt:
      "This leaves us with at least two further theories. Both are still in their infancy and it may be advisable to let them grow a little more.
      I will therefore tread lightly and speak softly to provide them both with the nurturing environment they deserve."
      and the rest that follows that (all in the September 6).

      I'll stop here, although there are more examples, but I cannot call this just "wry" commentary.

      Delete
    10. Marco, it might depend on the thickness of one's hide and one's sense of humour :) (Those who dish it out most virulently have remarkably thin and tender skin, which bleeds copiously at the merest touch.)

      Remember too that the first comments you quote are before Stephan had received the okay from the ethics people that he could release the names of the blogs he invited. He was giving them another chance to not look quite so foolish.

      I'll refrain from a "you said" / "he said", except to say that what Stephan wrote was a tender loving whisper compared to the bile that was being spewed at Climate Audit and elsewhere - including by deniers on STW.

      Delete
    11. Sou, I found it quite funny myself, seeing the deniosphere wriggle around like crazy, but in the end the result is the same: Lewandowsky was egging them on, literally asking them to come with more conspiracies. You could say it is their (=pseudoskeptics) own fault to then actually deliver, but then we have a direct interaction between observer and study object, and thus an impact of the observer on the study object. And that was not part of the approved study protocol...

      Delete
    12. I don't expect you're the only person who thinks that way, Marco. And climate scientists are, in the main, a polite and retiring bunch of people - extremely polite. Professor Lewandowsky is from a different discipline and he isn't exactly the shy and retiring type :)

      Then again, who'd seriously expect anyone to bite when someone writes:
      "This leaves us with at least two further theories. Both are still in their infancy and it may be advisable to let them grow a little more."?

      Out of all the people who read that, there wouldn't have been too many people who would have meekly done his bidding. Surely not the very same people who were protesting everything else he'd written :(

      Seriously though, while the headlines were cheeky, and with some exceptions, the tone and content of Stephan's articles were on the whole very restrained and disciplined, given the circumstances.

      Delete
    13. Marco:

      "...we have a direct interaction between observer and study object..."

      That interaction commenced the moment that the Denialati started with their conspiratorial responses to LOG12, and became by very definition a part of Recursive Fury. In such an instance some interaction is an inevitable part of the whole matter.

      Further, interaction is a recognised and to some extent an unavoidable part of human studies - white coat effect, placeboes and noceboes anyone? The skill is to identify and account for any investigator/subject interaction.

      When all is said and done if Lewandowsky did anything at all it is pretty much in line with something I raised over 18 months ago on pretty much the same subject...

      No one asked the bankers to stand.

      Delete
    14. The interaction wasn't direct any more than a newspaper article is direct. Stephan didn't have a normal dialogue with anyone. He posted a blog article. And apart from eventually listing the blogs that were invited to post a link to the LOG12 survey (after much haranguing from deniers), the only person named in the blog articles AFAIK was Steve McIntyre, who kept putting up nonsense stories. Stephan didn't interact directly with him, he just put up evidence countering what he wrote about "scamming" and explaining why McI's stats was up the creek.

      Delete
    15. I disagree with the idea that the interaction wasn't direct any more than a newspaper article is direct. It showed Lewandowsky directly reacting to blogs and blog comments elsewhere. Newspaper articles don't do that. It was clear who was being addressed. Call it a dog whistle.

      Delete
    16. Bernard, in many studies interaction with the study objects is impossible to prevent, but then you do everything you can to minimize that influence or control for that influence as you correctly point out. Hence the use of double-blind placebo controlled trials. Lewandowsky actively sought the interaction and I did not see any attempt in Recursive Fury to take that into account.

      It's like deliberately staring at a dog that shows some potentially aggressive behavior. You know you are provoking actual aggression.

      Delete
    17. Marco, the easiest way to resolve this is not to speculate amongst ourselves about Lewandowsky's motivation and thoughts, but to simply ask him directly.

      I've found Stephan to be easily approachable and I'm sure that he'd offer his own perspective if asked politely.

      Delete
  5. Good collection, thanks Sou.

    Of course, for legal purposes, as in defamation cases, blog posts are certainly considered public speech, which would make it absurd that they not be subject to critique or scholarly analysis. There are few cases in progress right now, with lots of amazing claims, but as far as I know, not including the idea that blog comments would not be publicspeech.

    I have never been able to figure out why someone who posts as Anonymous, especially on a blog that says:
    "Instead of commenting as "Anonymous", could you please comment using "Name/URL" and your name, initials or pseudonym or whatever. "

    should expect that anybody should even bother reading what they write, much less ascribe any weight to their opinions. This seems one of the bizarre expectations of the virtual world that do not occur so much in the real world, where people actually give different weights to different people's opinions. At least, on the Web, if someone has a website and uses a consistent, unambiguous handle, readers can assess the credibility of opinions.

    ReplyDelete
    Replies
    1. Thanks, John. That's a very good point you make about defamation, blogs and blog comments and public speech.

      In Australia, that applies also on blogs that are public blogs (in that there is no barrier to people subscribing) but that require readers to log in to read or make comments.

      Delete

Instead of commenting as "Anonymous", please comment using "Name/URL" and your name, initials or pseudonym or whatever. You can leave the "URL" box blank. This isn't mandatory. You can also sign in using your Google ID, Wordpress ID etc as indicated. NOTE: Some Wordpress users are having trouble signing in. If that's you, try signing in using Name/URL. Details here.

Click here to read the HotWhopper comment policy.