Research publication through the Internet and the role of review

Professor Hitoshi Mikada, Graduate School of Engineering, Kyoto University

In the PEPS Editors' blog in October 2015, I confessed that it was a laboratory policy not to refuse the favor to review papers submitted to scientific journals and that the consequence at the time was to review regularly double digits number of papers every year. Since 2016, the number of reviews has drastically decreased and is now far less than 10. I do not know why a sudden change has taken place but imagined an English translation of my text in the blog could make editors to feel a little sorry for inviting me for review. However, the number of peer-reviews to proceedings and abstracts for international meetings for the acceptance has become three-digits per meeting at the same time and the total time cost may become the same as before. When serving as a convener to a session of the Fall Meeting of the American Geophysical Union more than a decade ago, I confess that I rejected two submissions out of 40 to 50 and the rate of acceptance was about 95%. As the Society of Petroleum Engineers sets the ceiling in the number of acceptable papers up to 30% of the total number of submissions, peer-reviews of abstracts and proceeding papers for engineering international meetings are very important to reject submissions that do not deal with researches beyond a certain scientific datum. The reviews to submissions to scientific journals or engineering society meetings are to ensure the quality of research publications. The system of peer-reviewing is purely supported by anonymous reviewers responsible for maintaining the level of the academia, and I would like to deal with the review system as a topic from a standpoint a little higher than last time. The contents are from what I give to students in the class of engineering ethics in the Undergraduate School of Engineering, Kyoto University.

Seven years have passed since Mr. Jeffrey Beall of the University of Colorado at Denver (UCD) created the word "Predatory Journal." According to Butler (2013), Mr. Beall noticed a rapid increase in the number of direct emails to invite people to submit manuscripts to open journals around 2008 and numerous grammatical errors in the messages that triggered Mr. Beall to jump into the predatory journal problem. Mr. Beall used a set of criteria (Beall, 2012) to list up predatory journals and publishers and continued to publish and update what is called the Beall's List on a web site in UCD until the end of 2016. The number of predatory publishers in the Beall's List was 18 in 2011 has reached 923 on December 31, 2016. No researchers who were watching the site have not at all been noticed but the site was suddenly closed in January 2017 this year (Silver, 2017a) and seven months have passed since then. The readers may remember a lot of speculations for reasons of the sudden site closure and complaints when the Beall's List was suddenly lost. Since June this year, the list of predatory journal and publishers based on different criteria from the Beall's List was published for sale by a private company in the United States (Silver, 2017b). I feel that there really was a demand to the list of predatory journals and publishers in the academia.

In 2013, an article dealing with the results of a survey on peer-review of open access journals was published (Bohannon, 2013). Mr. John Bohannon assumed a false biologist Ocorrafoo Cobange of a false institute Wassee Institute of Medicine and wrote a paper to claim "a chemical substance extracted from lichens has shown anticancer properties." The manuscript he had prepared included mistakes and defective figures easily distinguishable by whom studied high-school level chemistry. Two months after the submission of the manuscript to 304 open access journals, 157 journals have accepted the paper for submission and 29 journals seemed to be derelict. Surprisingly, the rate of acceptance of his fake paper was 57% (52% even including 29 discontinued). Mr. Bohannon has included publishers in the Beall's list and it was also reported 82% of the predatory publishers have accepted the fake paper. The rate of paper acceptance by the publishers in the Beall's List have proven the applicability of the Beall's List. It was also surprising to see a journal published by Elsevier accepted the fake paper.

As is clear from the publication of the Beall's List and the findings of Bohannon (2013), researchers who want to have their papers printed and publishers who wants to earn what are paid by authors share the common objective to publish papers and the combination of the two risks research misconducts. There seems to be even cases in which fake impact factors are shown on the web site of such journals. Any changes in publishers such as in the intention for a reputation, for the presence, the succession of publisher's owners, or editor-in-chief, could influence the level of acceptance of submitted manuscripts, in which journals could risk becoming predatory. In academia, which is made up of maintaining the quality of research results, the number of papers and the impact factor of journals for which the publication of papers is done have been implicitly regarded as criteria for evaluation of researchers for a long time. However, it is interpreted that a whistle was blown to ask us if it is necessary to monitor the journals and publishers in terms of scientific values as a function of time by some means. At the same time, I think that it is necessary to recognize the importance of peer review process in maintaining academia's research level.

In Kuroki (2016), it is stated that Japan is the world fifth country in the retraction rate of papers. The rate of paper retraction has a big gap between the world fifth and sixth and it is an urgent task necessary to take prompt and effective countermeasures in countries down to the world fifth place including Japan. It is the Internet that cultivated the ground of predatory journals, but at the same time that founded a workplace for the disclosure of research misconducts. In order to gain the confidence to research integrity, I would like to propose editors and reviewers of PEPS to keep a sense of responsibility and enthusiasm and remind them how the peer review plays a big role in the advancement of science.

Beall, J., 2012, Criteria for Determining Predatory Open-Access Publishers (2nd edition), available at https://web.archive.org/web/20130603123106/http://scholarlyoa.com/2012/11/30/criteria-for-determining-predatory-open-access-publishers-2nd-edition/ (accessed on August 20, 2017).
Bohannon, J., 2013, Who's Afraid of Peer Review?, Science (04 Oct 2013), 342 (6154), 60-65, doi: 10.1126/science.342.6154.60.
Butler, D., 2013, Investigating journals: The dark side of publishing, Nature, 495, 433–435 (28 March 2013), doi: 10.1038/495433a.
Kuroki, T., 2016, Kenkyu-Fusei (Research Misconduct: translated by Hitoshi Mikada on August 22), Chuko-Shinsho 2373, Chuokoronsha, 302pp., ISBN: 978-4-12-102373-5. (in Japanese)
Silver, A., 2017a, Controversial website that lists ‘predatory’ publishers shuts down, Nature News (18 January 2017), doi: 10.1038/nature.2017.21328.
Silver, A., 2017b, Pay-to-view blacklist of predatory journals set to launch, Nature News (31 May 2017), doi: 10.1038/nature.2017.22090.

Professor Hitoshi Mikada, Graduate School of Engineering, Kyoto University
PEPS editor in the solid earth science section


PEPS固体地球科学セクション編集委員の三ケ田 均(みかだ ひとし)です.

2015年10月にこのBlogに駄文を投稿し,研究室として査読依頼は断らない方針であること,そして毎年二桁となる数の論文査読を担っていることを告白したところ,何と2016年から査読数が激減しました。英訳もついていたため, 海外のEditorの目にも留まり,不憫に思ってくださったのかもしれません.但し,気付けば論文ではなく,国際会議での発表可否を決める査読の要旨数は一会議当たり三桁になりました.十数年前,American Geophysical Unionの秋季大会のSession Convenerを務めた際,小職のセッションに投稿された40〜50の発表申し込みに対し,リジェクトした要旨は2編のみ,採択率約95%でした.しかし投稿数の30%を採択数と決めているSociety of Petroleum Engineersなど,工学系の要旨査読は少々シビアです.論文査読も大変ですが,過半数が不採択となる要旨査読も,細心の注意を要する集中した作業になります.どちらにしてもアカデミアのレベルの維持を担っているのは査読者が支える査読システムであることは間違いなく,今日はシステム全体をもう少し高い場所から眺める話題にしたいと思います.内容は,京都大学工学部の「工学倫理」で筆者が教えているものです.

コロラド大学デンバー校(UCD)のジェフリー・ビール氏が,Predatory Journal(黒木(2016)では,「捕食」ジャーナルあるいは「ハゲタカ」ジャーナルと和訳されている)なる言葉を生み出し,世界各国の数多の雑誌をこのPredatory Journalに分類し始めて7年が経過しました.Butler (2013)には,2008年ごろ急速にその数を増やしたOpen Access Journalからの投稿案内ダイレクト電子メールの文章に数多くの文法ミスを見つけたことが契機になり,ビール氏がこのPredatory Journal問題に気付いたとの記述があります.ビール氏は,Beall (2012)の基準を用い,2011年以降Predatory Journal及びその出版社を公開・更新し続けたリストは,Beall's Listと呼ばれ,研究者の世界では重宝されたことは言うまでもありません.最初の公開時には18社であったPredatory Journal出版社は,2016年12月31日のリストでは923社に達しましたが,今年1月に突然サイトが閉鎖され,ビール氏が分類作業やPredatory Journalに関する発信を突然終了(Silver, 2017a)してから7か月が経過しました.この間,突然Beall's Listが失われたことに対し,突然のサイト閉鎖に対する理由の憶測や不満の声が流れたことはご記憶に新しいかと思います.今年6月からは,Beall's Listとは異なる基準でリストアップされたPredatory Journal及び出版社のリストが,私企業の手により有料で公開(Silver, 2017b)されたのは,経済活動の活発な米国の健在ぶりを再認識させると同時に,アカデミアのPredatory Journalリストへの需要の大きさを物語っている気がします.

2013年に,Science誌上でジョン・ボハノン氏の行なった調査の結果(Bohannon, 2013)が発表されました.ジョン・ボハノン氏は,Wassee Institute of Medicine(ワシー医学研究所;実在しない)の生物学者Ocorrafoo Cobange(オコラフー・コバンジ;偽名)を騙り「地衣類から抽出したある化学物質に制癌作用がある」との内容の論文を作成し304社(このうち29社は既に廃刊状態であったことが判明)のオープン・アクセス・ジャーナル出版社に投稿しました.高校生レベルの化学を学習した人間には容易に判別可能な間違いや欠陥のある図を挿入したにも拘らず,投稿から2か月後には157社の雑誌で受理されたそうです.彼の偽論文の採択率は57%(29社を含めても52%)と過半数になったことにも驚かされましたが,送付先に含まれていたBeall's List中の実に82%の出版社が論文を受理したことでBeall's Listの信憑性が高まったのです.また,偽論文受理にElsevierの出版する雑誌が含まれていたことも物議を醸しました.

Beall's Listの公表や,ボハノンのサイエンス誌上の調査結果から明らかな通り,論文を数多く出版したい研究者と,論文を出版することで投稿料や掲載料収入を得たい出版社の思惑は一致します.論文数を増やしたい研究者が流れ着く先は,最後に印刷を引き受けてくれる出版社になるのかもしれませんが,掲載目的のためだけに成り立つオープン・アクセス・ジャーナル出版社が残念ながら存在します.中には偽のインパクト・ファクターを使っているケースもあるようです.さらに,出版社における論文数増加の意図の発生,出版社の経営者やEditor-in-Chiefが交代などを契機として,投稿論文が受理されやすくなりPredatory化する,あるいはPredatory判定の基準周辺で浮沈する出版社もあるようです.研究成果の質を維持することで成り立っているアカデミアでは,論文数や論文掲載雑誌のインパクト・ファクターが,長い間,暗に評価の基準とされてきました.しかし,どうもこの基準だけでは不十分であり,研究成果発表をする雑誌の科学的評価を時間の関数として捉えていく必要があるという警鐘が鳴らされたのだろうと解釈しています.また,同時にアカデミアの研究レベル維持における査読プロセスの重要性を認識する必要を物語っていると思います.

黒木(2016)では,論文の撤回率という物差しで見た場合,日本が世界第5位の研究不正大国であることが記載されています.第6位以下の国々の論文撤回率は第5位以上と桁が異なり,日本を含む世界第5位までの国々では迅速かつ効果的な対策を講じる必要があります.Predatory Journalの温床となっているインターネットですが,同時に不正の発見に成果の電子的公開という手段が寄与していることは皮肉です.研究不正の少ない日本という信頼を取り戻すためにも,我々研究者が担う査読がいかに大きな役割を果たしているかという観点で責任感と矜持をもち、今後も襟を正して査読をさせていただきたいと思っております.

Beall, J., 2012, Criteria for Determining Predatory Open-Access Publishers (2nd edition), available at https://web.archive.org/web/20130603123106/http://scholarlyoa.com/2012/11/30/criteria-for-determining-predatory-open-access-publishers-2nd-edition/ (accessed on August 20, 2017).
Bohannon, J., 2013, Who's Afraid of Peer Review?, Science (04 Oct 2013), 342 (6154), 60-65, doi: 10.1126/science.342.6154.60.
Butler, D., 2013, Investigating journals: The dark side of publishing, Nature, 495, 433–435 (28 March 2013), doi: 10.1038/495433a.
黒木登志夫, 2016, 研究不正,中公新書2373, 中央公論社, 302pp., ISBN: 978-4-12-102373-5.
Silver, A., 2017a, Controversial website that lists ‘predatory’ publishers shuts down, Nature News (18 January 2017), doi: 10.1038/nature.2017.21328.
Silver, A., 2017b, Pay-to-view blacklist of predatory journals set to launch, Nature News (31 May 2017), doi: 10.1038/nature.2017.22090.

PEPS 固体地球科学セクション編集委員 / 京都大学 大学院 工学研究科 教授 三ケ田 均


Visualization of peer review records

Hi, I'm Yuichi Hayakawa from CSIS (Center for Spatial Information Science), The University of Tokyo. 
I originally posted a Japanese version of this article, but there were some requests to provide it in English as well.
Thanks for giving me the opportunity to provide this article in English, regarding the visualization of peer review records. 

Supposing that many readers of this blog are researchers including students, I would like to ask you one question: 

As a researcher, submitting one's paper is an essential task.
On the flip side, reviewing papers of someone else is also necessary for publishing scientific papers. 

However, the process of peer review is often performed anonymously, and the action itself may not correctly be evaluated no matter how much a researcher takes the time to read and evaluate someone's manuscript drafts.

Even though we recognize that peer review is an essential process for the scientific community, this may lead to the disadvantageous feelings that reviewing is rather time-consuming, cost ineffective. 

To overcome such problems, there is a relatively new service, Publons

Publons, founded on 2012, provides a web service on recording and publishing peer reviews with necessary confidentiality. 
"A mission to speed up science - Publons" 
There are a variety of functions in Publons, including post-publication review. However, for simplicity, here I focus on the normal pre-publication review. 

A researcher can register the basic information of a peer review report that she/he performed, including title, journal name, date of review, and review report itself. An option of making it public or private is available for each review following the policy of the journal and the reviewer. 

Here is my case. 
An overview can be embedded in a web page like this.

In the Profile > Statistics view, the timeline of peer review records is shown (as of July 2017 – just 100!).

Monthly review records

For the old review history, I salvaged past e-mails and files to enter the records – somewhat annoying taking a long time... But finally, I can see my total review records like this. It should be better to record the current review in progress every time it is completed. 

The number of reviews has increased, where their frequency has also risen. 
It had been once every few months, but in recent years, it has become more common to review 3-4 cases per month.
This increase may be either as a result of the personal career development and the general increase in the number of publications.

Impact factor (IF) is often regarded as a standard evaluation index of academic journals, and the number of reviews for journals can be shown like this.

Impact factors of journals reviewed for

In my case, the mode is around IF = 2 to 3. This could be because I most often review for Geomorphology (Elsevier) as a board member.

Also, the trend can be compared with those in the field of earth and planetary science.
Researchers in this field seem to have many reviews conducted in journals with a bit higher IFs than me.

Another funny stat is the "weekly review punchcard" – you can see which day of the week you often submit your review report.

Weekly review punchcard

Though we can read manuscripts at any time, it is necessary to keep the time to summarize review results. The review reports, therefore, tend to be sent more frequently during the latter part of weeks where teaching classes and administrative meetings are relatively few, or on the weekend's evening (the midnight of Sunday – actually Monday).

Without doing anything, peer review is just over when it is returned – this should be, of course, a significant contribution to the scientific community. However, recording reviews would help researchers to motivate their review activities by the visualization of review records. 

By the way, we do not have, in fact, a standard lesson or training methods for reviewing so much. Some opportunities like seminars or workshops are being provided recently, but still not so many.
It is necessary to learn by themselves like an on the job training.
Preparation as a peer reviewer - PEPS Editors Blog (mostly in Japanese, but some references link to webpages in English)
Then, even by just looking at the peer review history of others, visualization of review records would help to know how to proceed with reviews.
It is not competition, of course, so the number or frequency of reviewing is not a big problem.
For the high-quality peer review, we can utilize this kind of system to enhance our motivation and to be evaluated in an appropriate manner. 

By the way, Publons has been very recently (last month) acquired by Clarivate Analytics
Also, for example, a major academic publisher like MDPI is collaborating with Publons on its peer review system.
It seems that the peer review evaluation services of Publons are becoming more popular.







Publonsの創設者にインタビュー:「科学加速化ミッション」 - Editage Insights 









IF=2-3 くらいの学術誌が突出して多いのですが、これはElsevier社のGeomorphologyの委員になっていることもあり、そこでよく査読を行っていることが影響していると思います。





査読者の心構え – PEPS Editors Blog

なお、ここで紹介したPublonsは、ごく最近、あのClarivate Analytics社に吸収されました。





PEPS大気水圏科学セクション編集委員の早坂 忠裕です。









PEPS科学セクション編集委員/東北大学 大学院理学研究科 早坂 忠裕


New Year Message from the Editor in Chief of Progress in Earth and Planetary Science

[Japanese text follows English. 日本語版は英語版の後に表示します。]

A happy new year to all

In 2016 PEPS received 60 manuscripts and published 37 papers. Since PEPS was only launched two and a half years ago we have not yet had time to finish our registrations with either of the main international research paper databases Scopus or Web of Science. In view of this I think we can regard the number of published papers as a modest sign of success. We have now completed our applications to both Scopus and Web of Science and these are currently being reviewed. I hope that I will be able to report sometime later this year that both applications have been accepted.

At the moment it takes on average 190 days from receiving a manuscript until it is published. The editorial team is focusing on “rapid review and publication” in order to reduce this time. The editorial office monitors the speed with which manuscripts proceed through the system and sends email notification to the relevant editors when there is any delayHowever the time to publication is largely dependent on the speed at which referees are able to review papers. I would therefore like to take this opportunity firstly to thank everyone who has or will review papers for PEPS for their hard work, and secondly to humbly ask that referees aim to complete their reviews as quickly as is reasonably possible.

We receive many manuscripts written by authors who are not native English speakers: for these manuscripts we have adopted a system of provisional acceptance. Under this system, manuscripts are sent out to referees and those by non native authors that are favorably reviewed are provisionally accepted and then sent to an English language proofreading company at the JpGU’s expense in order to correct any minor mistakes in expression or grammar. Although this all happens before the manuscript is formally accepted, in practice provisional and formal acceptance are nearly equivalent and we have so far published all of our provisionally accepted articles. The reaction to this process has generally been positive but we have had some complaints that manuscripts have been over edited, that the burden on authors is increased and that the system leads to delays in publication. There is some truth in these complaints. However it is our main goal that PEPS be regarded as a journal of the first rank: we are working hard to achieve this, and, I feel, have had much success in doing so. In order to maintain our status we have to consistently publish high quality papers where excellent scientific content is clearly presented and precisely explained. Unfortunately this requires, amongst other things, a certain level of English language competence. I apologize for the additional burden that this places on our non native authors, and hope that they will understand our position.

I have been aware for a while that Japanese institutions have been suffering from the rapid increase in online journal subscription fees. In the past few months there have been several international developments affecting this issue. At the end of last year negotiations between Elsevier and a group of German state funded universities and research institutions broke down with the result that from January 2017 researchers at these organizations will lose online access to many Elsevier journals. Similar problems have occurred in Taiwan and Peru (Nature News on 23 December 2016). Such problems serve to emphasize the important role that open access journals have to play in the field of academic publishing, and these developments further convince me that PEPS, as a high quality international open access journal covering all of the areas of Earth and planetary science, has an important role to play.

Recent years have seen the increasing popularity of letter journals, and indeed these offer a number of advantages: the compact presentation enables readers to quickly acquire an overview of material they are interested in whilst authors benefit from the relatively short time between submitting a paper and its publication. Despite these benefits, the importance of detailed exposition of new scientific ideas and results of course remains undiminished and this is why PEPS has chosen to focus on this area. We want our authors to explain their ideas in detail and we allow them up to 50,000 words per paper in order to do so (and, as I mentioned above, we are working to further reduce the time to publication of our papers).

The purpose of academic study is not only the creation of new knowledge. For it to be of any use such knowledge must be transmitted to the wider community, and we at PEPS are doing all we can to help with this. I would like to thank everyone for the assistance they have already given us and respectfully ask you all to continue to help, firstly by considering publishing new work in PEPS, and secondly by refereeing any articles we send you as quickly as is reasonably possible

Finally let me wish you all a happy, productive and prosperous 2017

Yasufumi Iryu
PEPS Editor in Chief

PEPS総編集長の井龍 康文です.2017年の年頭にあたり,皆様に御挨拶申し上げます.

2016年,PEPS37編の論文を出版しました.また,投稿を受け付けた原稿の数は60編でした.これは,PEPS創刊から28ヶ月ほどであること,Web of ScienceWOS)やSCOPUSという国際的な論文データベースに採録されていない現時点では,健闘している数値であると認識しています.なお,WOSおよびSCOPUSへの採録ですが,昨年,採録申請を行い,現在,審査を受けております.2017年に皆様に朗報をお届けできると期待しています.

現在,PEPSでは原稿受付から出版までに,平均で190日ほどを要しております.編集部では「Rapid review and publication」を心がけており,査読の遅れている原稿に対しては,編集事務局から編集委員の方々へ対応を求めるメールを送り,査読が迅速に進むようにしております.しかし,査読期間の長短は,査読者による査読の遅速にかかっています.査読者の御尽力に感謝するとともに,「Rapid review」達成のためにより迅速な査読を心がけていただくようお願いします.







井龍 康文