There's an article at WUWT today (archived here) which is edited by that Master of Delusion himself - Anthony Watts. Anthony is portraying himself as a super-hero (again). He is telling his readers that blogs are better than scientific journals. In particular, his climate conspiracy blog is better - oh boy!Anthony thinks that a blog article he came across via a tip from Judith Curry (of all people) means that WUWT is better than Nature and Science when it comes to scientific articles. (Remember, Anthony is also under the delusion that his climate "hoax" conspiracy blog WUWT is a science blog.)
Anthony's contribution to the article is a sentence at the top of his copy and paste, some comments in brackets in the copy and paste, and some deluded comments at the bottom. (He's not one for writing much these days. He gets himself into too much strife.)
The article he links to is by experimental psychologist, Daniël Lakens, who is arguing (I think) for an improvement in the quality of social science and/or psychology journals. Dr Lakens does this by putting forward (but not testing) an hypothesis that blog articles are better than science journals.
Are blogs of higher quality than journal articles?
One interesting thing is that Daniël Lakens doesn't provide much by way of data or analysis (let alone code) to support his hypothesis. Before getting into that, though, lets look at what he is arguing. He wrote:
In this blog, I will examine the hypothesis that blogs are, on average, of higher quality than journal articles. Below, I present 5 arguments in favor of this hypothesis.His five arguments are as follows:
- Blogs have Open Data, Code, and Materials
- Blogs have Open Peer Review
- Blogs have no Eminence Filter
- Blogs have Better Error Correction
- Blogs are Open Access (and might be read more)
One big problem I have with his hypothesis as constructed is that it does not distinguish between blogs. He doesn't place a restriction, for example, that the blog have any internal quality control or credibility. He does restrict journal articles to peer-reviewed journals. Perhaps he meant his readers to assume some miminal level of competence (and ethics) by the author of the blog subject, although two of his five arguments suggest not.
On the five arguments above, I offer the following comments:
1. Open data, code and materials. Now open access to data is extremely important and these days with computer storage quite cheap, there's little excuse not to provide it, even if the data sets are gigantic as they often are. However there are also very good reasons for not sharing raw data with every Tom, Dick and Mary in some instances. For example: if you are collecting personal data from people on the understanding that their private information will only be used by the researchers and not aired to the entire world, you'd have a hard time (as would everyone in your field) getting any more people to participate if you broke that commitment. You'd also be violating ethics codes of your profession and research institution. You'd probably also be breaking the law. There are privacy regulations in many countries.
In the field with which I'm most familiar (climate science), the majority of blog articles written by climate specialists are derivative in that they are discussing work that has been published in a scientific journal. It's the journal that has the data, code and materials. These days, with storage much cheaper than it used to be, most though not all journals provide that (and most insist upon it). Therefore the issue of open data, code and materials is much less of an issue than it used to be, and the reference to it in blog articles is often to the supplementary information in the journal - or to the data held on a climate site (such as temperature data).
2. Blogs have Open Peer Review. This may be so to a point. However, as Daniël Lakens points out, "The quality of the peer review process is as high as the quality of the peers that were involved in the review process."
On blogs that have no quality or peer filter, it is very difficult for the casual reader to determine the quality of comments. It may not be possible for the casual reader to determine if there is a filter in place at all, or whether the people commenting are peers of the author, or have any expertise in the subject.
Blog owners can place restrictions on who can comment on their blog. That is done to a greater or lesser or no extent on climate blogs. The main problem with blogs when there is open slather for comments, is that the lay person can become very confused and find it difficult to distinguish between knowledgeable comments and utter nutter comments such as you'll see at WUWT or relegated to the bore hole at RealClimate.org. Further confusion can arise when amateur or professional science disinformers add their bit, with the intent of confusing readers. (This happened at NYTimes DotEarth a lot, where there were a couple of regular disinformers who took over the comment threads.)
At WUWT, Anthony Watts has banned most scientists and knowledgeable climate hawks from commenting. The result is that the comments under his articles are commonly of the type "climate science is a hoax" and "it's not warming" (regardless of what the article is about). The "peers" in his case are disinformers, wacko climate science deniers, conspiracy theorists and fully paid up members of the scientific illiterati. Judith Curry's blog is little better. She panders to climate science deniers and the illiterati, which is reflected in the comments.
3. Blogs have no Eminence Filter. Daniël Lakens wrote: "It is an egalitarian and democratic medium. This aligns with the norms in science". I think his argument is around the author of the blog in this case, not the people who comment on the blog article.
In scientific publications there are all sorts of filters put in place, some desirable some less so. There are some classic examples of papers that were rejected which turned out to be ground-breakers in science. However this is the exception, not the rule. Newcomers to a field have a lot of hurdles to jump in order to make a name for themselves. They will often break in by attaching the name of an eminent scientist as co-author of their papers. (It could be interesting to compare the papers that get rejected with the papers that get published. I'll bet the rejections include some real doozies.)
On the other hand, having no eminence filter on a blog article means one of the usual guides for the non-expert is missing. If the blog article gets endorsement from an eminent person in the field (in the comments or via Twitter or by a referral from another blog article), then that can indicate to a person who has no expertise, that the article has merit. (People with expertise, eminent or otherwise, can judge the article on its own merits, of course.)
One more thought - readers can apply their own eminence filter to blog articles. People go to Isaac Held's blog because he is "eminent". People go to realclimate.org because the owners are eminent in their field. People go to WUWT because they are anti-science or they want to find out the latest conspiracy theory doing the rounds in the deniosphere. (I've no idea why people go to Judith Curry's blog. Perhaps to see what fluff or outrageous nonsense she came up with today?)
As an example of providing an "eminence" filter, Anthony Watts introduced Daniël Lakens as "an experimental psychologist at the Human-Technology Interaction group at Eindhoven University of Technology, The Netherlands". The intention was, most likely, to draw attention to his academic credentials, or "eminence" :) (He is in the Industrial Engineering & Innovation Sciences Department, and has published some work on p-hacking among other things, most probably making a valuable contribution to his field of work.)
4. Blogs have Better Error Correction. This seems to me to be a case of argument by assertion. The only example Daniël Lakens provides is when a journal he published in didn't put the link to his error correction above his paper. It's the norm in climate journals that I read to put a prominent link to corrections at the top of the paper. I can't say if it's always done or not. Mistakes happen, even in scientific journals. In Daniël's case, that's a problem for the journal but not an argument for blog science.
Daniël says that he can much more easily correct an article on his own blog. That's so. He has full control there. Not everyone is as thorough. How does the casual reader know if a blog article contains errors or not? A scientific journal has processes for error correction. If the error correction is made by the author it is published and (should be) prominently displayed above the original article. If another person submits a comment that disputes something in a paper, it is published as a comment for everyone to read, together with the reply by the original authors. In the extreme case, an author or the journal itself can retract a paper. In some cases, errors in journal articles are first discussed on blogs.
What about blog articles? There is no guarantee or standard process for error correction on blogs. Some of the few means a non-expert person have to judge a blog article is by the reputation of the author and the reputation of those adding comments (those that are let through the blog owner's filter). They may also look around to see if there are other blog articles or scientific papers on the topic, but that takes effort and knowledge. This brings us back to the eminence filter and the quality of the peers argument. Since no-one can be an expert in all things, judging the quality of an article often comes down to trust and credible authority.
Incidentally, in my experience, science bloggers are much more likely to correct mistakes than are science disinformers, who only survive in the blogosphere because they don't correct disinformation. Their target audience is people who don't want to "believe" science and who would leave their blog in a jiffy if disinformers started to publish fact instead of fiction.
5. Blogs are Open Access (and might be read more). There are also open access journals, but I'll put that to one side. In climate science, as I wrote in the first argument, many blog articles are about papers that have already been published in a scientific journal. They are a way of bringing science to the general public. They may be review articles about a general topic and refer to a number of papers, or an article explaining just one or two papers on the same topic. They might also be an article about something that's news (with no published science yet), or some nonsense doing the rounds on denier blogs. In any case, they will normally be supported by links and references to scientific data and papers in scientific journals.
Being read more does not provide any guarantee that what is being read is credible, sound or robust. Consider the number of people who still think that climate science is a hoax, because someone on a denier blog said so.
No hypothesis testing
As part of his conclusion, Daniël Lakens wrote:
Before this turns into a ‘we who write blogs recommend blogs’ post, I want to make clear that there is no intrinsic reason why blogs should have higher scientific quality than journal articles. It’s just that the authors of most blogs I read put some core scientific values into practice to a greater extent than editorial boards at journals. I am not recommending we stop publishing in journals, but I want to challenge the idea that journal publications are the gold standard of scientific output. They fall short on some important dimensions of scientific quality, where they are outperformed by blog posts. Pointing this out might inspire some journals to improve their current standards.My conclusion is that Daniël Lakens' blog article is not a good example of blog articles that have "higher scientific quality" than journal articles. It is not supported by data other than a few anecdotal illustrations of particular points he wants to make. The "blogs I read" comment in the paragraph above suggests that Daniël Lakens applies a filter to the blogs he reads, maybe an "eminence" filter😏. (I don't know how often he gets around to reading other blogs.) He did not report that he gathered the necessary information from blogs, or blogs that purport to be scientific blogs, to test his hypothesis. There is no attempt to collect or compare the quality of blog articles in general with the quality of journal articles. In other words, there is no rigorous testing of his hypothesis. (See the dissertation and paper by Paige Brown Jarreau in the references section below.)
I expect that was not Daniël Lakens' intention. I figure his blog article was merely meant to be provocative, not scientific. However the implicit irony struck me.
The value of scientific blog articles is that they are not as restricted as journal articles. People can write in plain english for the layperson. If written for the general public, they are less likely to be overloaded with scientific jargon (the language of the experts). Blog authors are free to hone in on a particular point and can refer the curious reader back to journal articles for further information. There is no length limit or graphics limit in a blog article. One is less likely to find a well-written blog article so full of acronyms that too much time is spent figuring out what they all mean.
Blog articles are a very important means by which scientists can bring science to the public. In my view, they are extremely valuable as an addition to scientific papers, not a replacement for them. Blog articles are not subject to the same rigorous review before or after publication. While they can be great as a way of explaining science and stimulating ideas for further research, they cannot be a substitute for the peer-review process of scientific publications.
Anthony Watts uses a blog article to justify his climate disinformation
Now you won't be surprised that Anthony Watts is very pleased to use Daniël Lakens' article as an excuse to avoid publishing his nonsense in scientific journals. He delights in promoting the notion that pseudoscience and conspiracy theories on climate disinformation blogs is every bit as good as quality research published by career scientists.
Anthony Watts, showing delusions of grandeur, wrote:
It has been said to me by a few people that WUWT has changed the world. I think it has, but I view it as a collective effort with other climate blogs. If climate blogs didn’t exist, there would be no exposure of Climategate, no exposure of the [IPCC’s] horrid messes in AR4 and AR5, among other issues.Seriously? WUWT has changed the world? How? His "exposure of Climategate" was a non-event that at best. A few of the stolen emails showed scientists have human feelings. They are not automatons or robots. "Climagegate" did not change the science in any way other than to strengthen it (through the resulting enquiries). There were no "horrid messes" in the IPCC reports. These reports had way fewer errors than your average telephone White Pages (or a dictionary), whereas WUWT is nothing more than one big error.
From the WUWT comments
There was an early diversion into the merits and otherwise of the Prius, which had nothing to do with the topic of blog articles. There were some comments about what misinformation readers have "learnt" at WUWT (eg that climate science is wrong). Then there were some classics.
eyesonu has never entered the hallowed grounds of a university, from what I can tell:
April 15, 2017 at 9:14 am
I have said it before and will say it again: WUWT is a University experience. I’m sure others agree.
I follow some other blogs and follow links but get most of my ‘classes’ here. A lot of years invested so far.
Latitude points out that Anthony Watts' habit of banning knowledgeable people means that his blog comments are one-sided (and predominately of the "stupid" variety):
April 15, 2017 at 9:15 am
…all true, except when blogs are heavily censored, people are banned that disagree
William Evans started his "thought" with this, and it got worse, much worse:
April 15, 2017 at 1:46 pm (excerpt)
Luke warmers are the ignorant hicks who made ”Yah but thuh.. basick signts is SoWND, Ya’W” a global statement.
If it’s not able to discuss gas mechanics freely and explain itself, yet thinks magic gais is areal signts.
It’s a lukewarmer. Or a believer.
Steven Mosher threw a spanner in the WUWT works, writing:
April 15, 2017 at 9:26 am
Funny skeptics posting here are the only people who have ever denied me requests for code and data.
Scafetta. Monkton. Amongst others.
Here’s a challenge Anthony. .. demand an archive before people post their articles LOL. …PLOS does that.
John Robertson doesn't seem to want to acknowledge that almost the only science you'll find at WUWT are his copies and pastes of press releases about scientific papers, which Anthony usually republishes under a headline starting with the word "claim". (That's the dogwhistle to his conspiracy clan that they are not to believe it.) He probably prefers the pseudo-science that Anthony's guest disinformers dish up, and the comments from WUWT's hard core conspiracy theorists.
April 15, 2017 at 9:42 am
Yes Blogs are now the best venue.
I get immediate access to the actual science offered up and get to draw my own conclusions as opposed to reading a breathless Press release and being directed to a paywall.
The imposition of ill-informed and or activist interpreters by most of the “science magazines” was the beginning of their demise.
The added benefit of instant criticism and points of view I would not naturally arrive at, do much to improve my learning.
Universities are doomed.
The School of Hard Knocks now has an active open learning wing.At a price the working man can afford.
son of mulder reports one of the many major deficiencies of WUWT blog articles. They generally don't adhere to argument 4 in Daniël Lakens' hypothesis:
April 15, 2017 at 9:56 am
The weakness of blogs is that after all the comments there is no revised version of the original article. It’s like an audit trail without the final accounts. At least authors could coment on whether significant revision is required or whether they are happy after the comments have dried up and at best revise their submission. And yes I don’t underestimate the additional work involved but it would leave a better than submitted article where appropriate or withdrawal.
Writing Observer is obviously not able to judge expertise. I doubt he or she has ever read a scientific paper much less met a traditional academic (whatever that is).
April 15, 2017 at 10:31 am
Speaking of correction within minutes… It’s “IPCC,” not “IPPC” in the last sentence of the next to last paragraph :>.
The other advantage of blogs is that one like WUWT assembles a “team” of high-quality experts in everything – theorists, observationalists, mathematicians, and even lowly copy-editors. Traditional academia rarely puts together such teams; there are always weak links somewhere.
Phillip Bratby prefers to swim in the denier fish bowl, and can't hack it when his silly notions draw criticism from real scientists.
April 15, 2017 at 10:44 am
Does RealClimate still exist? I haven’t been there in years; not since they didn’t like my criticism of one of their posts.
Aphan is under the delusion that there are conflicting opinions among scientists about the fundamentals of climate science. He or she doesn't know that it's the very fact that it's the consilience of all the evidence from the wide variety of different disciplines that is the strength of climate science.
April 15, 2017 at 11:11 amThat's enough. You can read the rest here.
I’ve often fantasized about “kidnapping” (if necessary) a group of the world’s “experts” in every area that affects the climate, and locking them in a building until they all come to an agreement on things. FORCING them to network, compare knowledge, explain each individual part of the planet to each other, and form a coherent, correct, cooperative platform upon which to build and understand from that point going forward.
Making the “atmospheric” scientists listen to, and validate what the geologists have to say. Making the computer jockies listen to and validate what the field researchers have discovered. And getting them ALL to sign a statement that lays it all out for the public in easy to read and grasp format. Something like “Here’s what we KNOW and can demonstrate. Here’s what we THINK. Here’s what we ASSUME. Here’s what we don’t know. ”
They all just do not mingle, share, or ask questions outside of their own specialties. And until they DO, and do it in an honest, open, clear way…we’ll never understand this planet completely or accurately.
References and further reading
Five reasons blog posts are of higher scientific quality than journal articles - blog article by Daniël Lakens at The 20% Statistician
Who's afraid of Open Data - blog article by Dorothy Bishop at BishopBlog, November 2015
All the Science That Is Fit to Blog: An Analysis of Science Blogging Practices - Doctoral dissertation by Paige Brown Jarreau, 2015
Jarreau, Paige Brown. "Science bloggers’ self-perceived communication roles." JCOM 14, no. 04 (2015): A02-2. (pdf here)
Possibly, Dr Lakens would not consider fossil fuel industry funded websites to be genuine blogs if he was aware of them.
ReplyDeleteThat may be so. (There are also anti-vax blogs and all sorts of weird and awful blogs posing as science blogs.)
DeleteIt's clear in his article (and comments) that he's writing for an audience of his professional peers to get them thinking. He's not an advocate for climate conspiracy blogs :)
Did anyone notice Watts' response to Mosher's post?
DeleteIn part here:
"But’s here’s a challenge for you: don’t be an arrogant SOB and reply to comments with one word “wrong” like you did yesterday, and have many times in the past, without offering anything other than your opinion."
That of course only applies to posters that defend science.
There are many there that do much worse that go unchallenged.
In response to such I received a "rebuke" from the great one, along the lines of "I'm fed up with moderating your arguments".
Like I'm supposed to give up and/or agree with he reams of bollocks posted on there when they "argue" (often via assertion) with me.
No longer post BTW - just look out for a few sensible posters there.
Heh. I gave up posting on WUWT when I was assured by the infamous attack dog Richard S. Courtney that Greenland couldn't possibly melt because the earth underneath the ice cap was shaped like a bowl, and all the meltwater would just collect there.
DeleteThen I asked him if he'd ever heard of moulins.
I can't quite remember what the response from Richard was but you can bet that it was dismissive and condescending. I stopped posting right after that episode. Complete waste of time and effort.
Metro:
DeleteI had an "discussion" with Courtney.
Twas about atmospheric CO2, and how much was anthro.
Maintained that the increase was natural, and dismissed the C13 v C12 decrease via appeal to a paper - err - his own, was "peer reviewed" before being published in E & E !
With one bound, eh?
Got down to the usual "lacking respect" for me and accusations of ad hom etc.
(Usual rabbit-hole stuff).
I actually gave up posting after one of my posts (perfectly reasonable, as ever) was [snip] 'd.
Presumably to end "moderation" of one of my "arguments".You need to have self control beyond which I can muster to be a regular science defender on there.
Much kudos to Nick Stokes and a (very) few others on there.
You are correct it's a waste of effort.
There may be the odd neutral that twigs from such posts, but denizens are so far gone, they'll just answer the dog whistle and sad cases like Worrell, Ball, Eschenbach will get the hugs and kisses they crave for.
don’t be an arrogant SOB
Delete
DeleteMaintained that the increase was natural, and dismissed the C13 v C12 decrease via appeal to a paper - err - his own, was "peer reviewed" before being published in E & E !
There's an even easier way to debunk the "increase is natural" claim -- unfortunately, it requires skills that your average denier doesn't possess, such as being able to read a periodic table of the elements, count protons and neutrons (to compute the molecular weight of CO2, N2, and O2), convert metric tons to grams and grams to moles. Basic high-school stuff.
Ask deniers who claim that most of the CO2 increase is natural to do the following:
1) Given that humans have put approximately 2 trillion metric tons of CO2 into the atmosphere since the beginning of the Industrial Revolution, convert that 2 trillion metric tons into moles of CO2.
2) Look up the total atmospheric mass and calculate the total number of moles of molecules in the atmosphere. (The denier will have to compute an estimate of the average molecular weight of the atmosphere to do this).
3) Using (1) and (2), calculate the CO2 PPM change assuming all of the 2 trillion metric tons of CO2 stayed in the atmosphere.
4) Compare the answer computed in step (3) with the observed 120 PPM increase since the beginning of the Industrial Revolution.
Here's what those deniers will most likely do instead:
Respond with a dumb look,
***********or***********
respond with an insult,
***********or**********
respond with a dumb look *and* an insult.
It's just like when the deniers complain they can't do any analysis because they don't have access to 'all the data'. Well that's because some countries charge non-academics for their data as it's expensive to collect and maintain.
DeleteBut then, of course, when previously unavailable data eventually does become available (like, all the GISTEMP data has been available for years), the deniers have already moved on to the next reason why they can't do anything (useful, that is. Like actually analysing said data that they were complaining they couldn't analyse).
"Blogs have Better Error Correction"
ReplyDeleteROFL! You can still download Watts-authored reports from SPPI & Heartland making claims about the surface record that even the great misinformer himself has since contradicted. As Tamino wrote of one of these:
"If you have any honor at all, you’ll set the record straight. You owe it to everyone, and especially to NOAA, to admit that you were wrong. And you certainly owe it to NOAA to apologize. You need to make a highly visible, highly public admission of error, and apology, for using falsehoods to accuse others of fraud.
Are you man enough?"
Imagine if a real scientist behaved like this…
Then there was Monckton's regular series of posts during The Pause That Never Was where he consistently misrepresented the IPCC's 1990 projection (He showed Scenario A, deleted B&C which were closer to reality) long after I'd pointed out the error.
Then there was Steve Goddard's nonsense on the Venus Greenhouse effect.
I could go on (and on).
In the UK we have the Bishop Hill blog run by accountant Andrew Montford. Recently he claimed that NOAA manipulated data to get rid of the Pause, based on the Bates non-event. The claim remains up even after Bates made it clear there was 'no data tampering, no data changing, nothing malicious.'
One does not need to look hard at BH to find other examples [though that's not a good use of anyone's time].
Anyone looking to Watts, Montford, Goddard or Monckton for reliable science is making a huge category error, IMHO.
I wrote the reply to Lakens back in 2015: No, blog posts cannot replace scientific articles
ReplyDelete"The science the public does not see, neither in the media nor on blogs, is also important for science. We will need a way to disseminate science that also works for the other 99.9% of science. Blogs and "blog review" won't do for this part."
Thanks Victor. Your article is a "must read". You provide readers with a depth of understanding of the research process in the context of publication, as well as giving us all insight into how scientists work together.
DeleteHi Sou,
ReplyDeleteI don't think I would totally endorse Daniël Lakens point but I think you need look at the type of blog I believe he is discussing versus Watts or Curry.
It is quite clear that Lakens' article is an opinion piece designed to be provocative and is addressing issues that are very common in psychology and associated fields which may not impinge on most areas of climate science or, perhaps, many of the hard sciences. He is not setting out to prove the thesis.
There is just no sensible comparison of a Watts' post and the subsequent "responses", most of which are Pavlovian, and a blog of the type Daniël Lakens is discussing.
Watts apparently lacks the knowledge to understand the difference and jumps on an interesting discussion piece to blow his own horn. And, clearly, most of his readers do not understand the difference between science denial and a discussion of actual scientific issues.
I am not a regular reader of Lakens' blog but it is very interesting and, from my rather casual reading, deals with a number of important methodological issues in the behavioural, social, and medical research areas. See for example http://daniellakens.blogspot.ca/2016/12/tost-equivalence-testing-r-package.html. Not perhaps light reading for non-psychologists but the issue is important.
The same can apply to a blog like Andrew Gelman's http://andrewgelman.com/ or, say Dorthey Bishop at Bishopblog http://deevybee.blogspot.ca/.
We are not talking anything at the same (extremely low) level of quality as WUWT or the Bishop Hill blog. From the climate science side you might like to think of these as roughly the equivalent of Tamino's blog compared with WUWT.
So while blogs are not likely to completely replace refereed journals they seem to have a very important place in some areas of science.
Daniël Lakens is an honest broker. He is discussing what educated honest people would do.
DeleteNot misinformers.
We could equally start a thread on why comedians in the US are more trusted with the news than the MSM news!
Bert
He is also good-spirited and courteous, and took the above article in the vein intended.
DeleteSo the claim that commenting on blogs is "open peer review" is a huge distortion of peer review. To have real scientific peer review you must have a scientific editor that is judging the reviews and the authors' responses and assessing the paper on objective grounds. On a blog there is no editor. Therefore there can be no peer review. It is in that transfer of responsibility from the authors to an independent editor/publisher that is the foundation of the trust we place (sometimes misplaced, yes, but real nonetheless) in peer-reviewed science. Too often the editors are forgotten, but they are the key.
ReplyDeleteYou are describing one type of peer review, one with its own strengths and weaknesses.
DeleteAcross disciplines there seems to be a great deal of variation on the dissemination of research findings and I suspect that conventions and standards regarding the peer review process vary by discipline.
One can argue that post-publication peer review is extremely valuable and that blogs can be one way of doing this. Or, as Lakens suggests, it could be a strong contender as a replacement in some cases.
To be blunt, there is no "peer" review if the reviewer is not a qualified peer. "Open", free-for-all review is not conducive to review by qualified peers especially in cases like evolution, climate change, etc. which impact on societal norms or large economic interests.
DeleteThis sounds elitist. But there it is. Science is generally performed by those who have worked decades and longer perfecting their craft to become elite.
To be even more blunt, some silly WUWT-er commenting "I don't understand it but it's brilliant" on an pseudo-science article by Willis Eschenbach, Bob Tisdale, Jim Steele Christopher Monckton or Tim Ball, is not peer review.
DeleteOn second thought - yes, it probably is :(
Delete@ jgnfld
DeleteIf you look at the blogs I reference you will find that there is a range of expertise but, in general it is by people who are experts or by people who are asking for explanations.
Science is generally performed by those who have worked decades and longer perfecting their craft to become elite.
Which in general describes the comentators on the blogs I mention.
@ sou
yes, it probably is
Indeed, yes
jrkrideau - name a scientific discipline where an independent editor is not the key to publication of peer reviewed articles. If reviews are done after publication, what is there to force the authors to make any changes? If the publishing venue is not independent, how does a reader know whether critical review comments have not just been deleted?
Deleterkrideau - name a scientific discipline where an independent editor is not the key to publication of peer reviewed articles.
DeleteNo, why should I?
First show me one where the editor has been key, as an editor, and then I might look and see if I can find one. I may or may not be able to do so.
I am pretty sure that I could show that Hugo Gernsback or Jim Baen have had a tremendous effect on Science Fiction; I am not sure I can find many editors of science journals can be shown to be as influential though I expect some were/are.
It is perfectly possible to cite a number of examples where editorial prejudice or incompetence or a simple lack of subject expertise has allowed the publication of dubious papers. Take the infamous Wegman saga for an example.
Historically we did not have editors or even peer-review. I may be wrong but I doubt that Archimedes or Galileo or Newton went through a blind review process and nor would have Bayes or Leibniz.
I don't doubt that in some or even many cases a top-notch editor and good peer review has very likely been of tremendous value to a journal and somewhat indirectly to a disciple, and, certainly, to individual researchers, but that does not mean that this is the only model we need to advance science.
At the moment I know of no research on the added-value of the editor-peer review model of scientific publishing. I believe it gives excellent value in many cases but I also believe it may not be the only model of scientific publishing that has value or that we need.
P.S. As a bit of a joke, one could argue that good pre-pub peer review might have helped in Galileo's case but the post-publication review did work. Still a post-pub review with a paranoid autocrat convening the “post-pub reviewers” may not be the optimal model.
I think the point was that the editor is key, not that any particular editor is key - though there are some notable editors, of course.
Delete@ Sou
DeleteOh yes, sorry if I was not clear but my point is that the thesis that one can show that an editor plus the need for peer review as it now exists "key", then there is no reason to assume that the current model (well mélange of models might be a better term) for scientific publishing and 'certifying' of research is not necessarily optimal. Heck, taking Daniël Lakens' article at face value, it may well be obstructive. In some cases, it may well be.
I may have to look up some information just how the current main editor + reviewers system evolved :(.
Just to throw out a couple of thoughts, journal space before the Internet was a very precious and commodity. so there was great value in having a (good) editor as a gatekeeper.
I, today, can write a paper that looks nice and sciency in a couple of hours with the aid of LaTeX and some decent graphing or imaging software.
In, let's say 1970, to produce the same paper that would like nice and sciency required skilled typesetters, expensive printing presses, elaborate formatting of materials to production standards plus time-consuming communications lags.
Much of the support that the editor + journal + organization + publisher provided may no longer be needed.
I think the point being made was about the editors' role in deciding whether to accept to the point of sending out for review, and then deciding whether to publish. (Don't all journals expect the authors to provide the artwork? I know some do. Layout is still done by the journal - print and electronic.)
DeleteThis is from the history page of the Royal Society. It has had an editor since 1665:
DeleteThe very first ‘learned society’ meeting on 28 November 1660 followed a lecture at Gresham College by Christopher Wren. Joined by other leading polymaths including Robert Boyle and John Wilkins, the group soon received royal approval, and from 1663 it would be known as 'The Royal Society of London for Improving Natural Knowledge'.
The Royal Society's motto 'Nullius in verba' is taken to mean 'take nobody's word for it'. It is an expression of the determination of Fellows to withstand the domination of authority and to verify all statements by an appeal to facts determined by experiment....
...The early years of the Society saw revolutionary advancements in the conduct and communication of science. Hooke’s Micrographia and the first issue of Philosophical Transactions were published in 1665 alone. Philosophical Transactions, which established the important concepts of scientific priority and peer review, is now the oldest continuously-published science journal in the world.
We published Isaac Newton’s Principia Mathematica, and Benjamin Franklin’s kite experiment demonstrating the electrical nature of lightning. We backed James Cook’s journey to Tahiti, reaching Australia and New Zealand, to track the Transit of Venus. We published the first report in English of inoculation against disease, approved Charles Babbage’s Difference Engine, documented the eruption of Krakatoa and published Chadwick’s detection of the neutron that would lead to the unleashing of the atom...
...Over time, the criteria for, and transparency of election to the Fellowship became stricter, and Fellows were elected solely on the merit of their scientific work.
Do all editors always make the right decision? No. Is the process more rigorous and reliable than the array of blogs on the internet? Indubitably :D
Do all editors always make the right decision? No. Is the process more rigorous and reliable than the array of blogs on the internet? Indubitably
DeleteBeall's list exists for a reason. :)
Authors are doing much of the work today that 30-40 years ago was done by the publisher and printer so dubious journals are now cost-effective.
BTW, I doubt that the Royal Society's screening and review in the 1680s was anything like its process now. Then all (almost all?) the members would know each other personally. Most had been friends and/or foes for years.
I'd question if there was anything formally like an editor, more likely it was, "So gentlemen, are we willing to publish Isaac's new work? He's calling it Principia Mathematica. We're agreed? Good.
Mr. Secretary would you make arrangements with the printer?"
Not quite sure what you're arguing, jkrideau, (and a bit disappointed to read the "I'd have to question" that seems awfully like an argument from ignorance).
DeleteAre you saying that the Royal Society erred in publishing Newton's work? That you don't believe its motto meant anything in practice or that it implemented the procedures it adopted, and which evolved to what they are today?
Are you saying that dubious (usually not peer-reviewed) journals exist, therefore blog review is better than the peer review process used by (real) journals?
BTW I don't think that authors are providing any more to journals now than they did 30 or 40 years ago, though it's easier to prepare now (and the artwork quality is better). Someone who submitted stuff in the 1970s and 1980s can tell me if they sent more to journals back then than they do now (in prepared words and charts).
Sorry Sou,
DeleteThe "I'd have to question" format is a discipline (and age?) format that says "There is no reason what-so-ever to believe that reviews in the 1680's had any relationship to modern-day peer reviews". Or in simple translation "prove it",
It is probably a national, disciplinary and age issue. Canadians of my generation (60+) are, usually, extremely reluctant to say "I don't believe you" or "You're a lier" and we may have evolved methods of intimating this without a direct attack.
My impression re the RS publishing is only based on general reading of the times and research activities of the times but it seems likely. I am not suggesting that the RS should not have published Newton's work, just that the review process would have been very different than a modern-day review.
I am arguing that we cannot attribute behaviours and requirements that we see in the 20th and 21st C apply to the 17th.
Again I would stress that Newton would have been personally acquainted with all or almost all of the other members of the Royal Society. Not just a few anonymous reviewers would be acquainted with his work. It is quite likely (although I have no evidence that I remember anyway) that several or many members may have read earlier drafts.
Are you saying that dubious (usually not peer-reviewed) journals exist, therefore blog review is better than the peer review process used by (real) journals?
No but I am suggesting that the existing model may no longer be as valid as we would like. You might like to look at http://andrewgelman.com/2017/04/19/my-proposal-for-jasa/. To be honest, I have only scanned it but it seems an interesting approach to some problems. At the moment, given my limited understanding of the issues and solutions it is addressing I just advance it as an alternative.
BTW I don't think that authors are providing any more to journals now than they did 30 or 40 years ago
I think we may be talking at cross-purposes here. I am just saying that I as an "author" can produce something that looks "sciency" a lot faster today than 30 or 40 years ago.
This does not mean that good researchers are submitting more articles just that it is faster and easier for dubious press to publish if the researcher is conned into submitting.
BTW in no case am I suggesting that blog review is better than conventional peer review processes. I just am not convinced that the current peer review processes are keeping pace with evolving technology and social demands.
Yes, there are certainly horrible "science" blogs out there, including scores of climate science denying ones, but their existence shouldn't serve to smear the ones that are doing good work.
ReplyDeleteFor example, blog forums such as the Azimuth Project allow full markup of equations and posting of charts and graphics so that you can see the sausage being made. There is nothing like that being done in the traditional scientific review cycle. Of course collaboration among scientists occurs, but it is rarely done in the open.
Too bad that the deniers are giving the dedicated scientific bloggers a bad name. It really is so easy to distinguish delusion from serious discussion.
Has our hard-dealing President found a buyer for Trump U in Tony Watts ?
ReplyDeleteEyesonyou April 15, 2017 at 9:14 am
I have said it before and will say it again: WUWT is a University experience. I’m sure others agree.
I follow some other blogs and follow links but get most of my ‘classes’ here. A lot of years invested so far.
Oh yes, Russell, WUWT is a university-level experience alright: Watts' University Would Tank.
DeleteJust... not my kind of university.