Visualization of peer review records

Hi, I'm Yuichi Hayakawa from CSIS (Center for Spatial Information Science), The University of Tokyo. 
I originally posted a Japanese version of this article, but there were some requests to provide it in English as well.
Thanks for giving me the opportunity to provide this article in English, regarding the visualization of peer review records. 

Supposing that many readers of this blog are researchers including students, I would like to ask you one question: 

As a researcher, submitting one's paper is an essential task.
On the flip side, reviewing papers of someone else is also necessary for publishing scientific papers. 

However, the process of peer review is often performed anonymously, and the action itself may not correctly be evaluated no matter how much a researcher takes the time to read and evaluate someone's manuscript drafts.

Even though we recognize that peer review is an essential process for the scientific community, this may lead to the disadvantageous feelings that reviewing is rather time-consuming, cost ineffective. 

To overcome such problems, there is a relatively new service, Publons

Publons, founded on 2012, provides a web service on recording and publishing peer reviews with necessary confidentiality. 
"A mission to speed up science - Publons" 
There are a variety of functions in Publons, including post-publication review. However, for simplicity, here I focus on the normal pre-publication review. 

A researcher can register the basic information of a peer review report that she/he performed, including title, journal name, date of review, and review report itself. An option of making it public or private is available for each review following the policy of the journal and the reviewer. 

Here is my case. 
An overview can be embedded in a web page like this.

In the Profile > Statistics view, the timeline of peer review records is shown (as of July 2017 – just 100!).

Monthly review records

For the old review history, I salvaged past e-mails and files to enter the records – somewhat annoying taking a long time... But finally, I can see my total review records like this. It should be better to record the current review in progress every time it is completed. 

The number of reviews has increased, where their frequency has also risen. 
It had been once every few months, but in recent years, it has become more common to review 3-4 cases per month.
This increase may be either as a result of the personal career development and the general increase in the number of publications.

Impact factor (IF) is often regarded as a standard evaluation index of academic journals, and the number of reviews for journals can be shown like this.

Impact factors of journals reviewed for

In my case, the mode is around IF = 2 to 3. This could be because I most often review for Geomorphology (Elsevier) as a board member.

Also, the trend can be compared with those in the field of earth and planetary science.
Researchers in this field seem to have many reviews conducted in journals with a bit higher IFs than me.

Another funny stat is the "weekly review punchcard" – you can see which day of the week you often submit your review report.

Weekly review punchcard

Though we can read manuscripts at any time, it is necessary to keep the time to summarize review results. The review reports, therefore, tend to be sent more frequently during the latter part of weeks where teaching classes and administrative meetings are relatively few, or on the weekend's evening (the midnight of Sunday – actually Monday).

Without doing anything, peer review is just over when it is returned – this should be, of course, a significant contribution to the scientific community. However, recording reviews would help researchers to motivate their review activities by the visualization of review records. 

By the way, we do not have, in fact, a standard lesson or training methods for reviewing so much. Some opportunities like seminars or workshops are being provided recently, but still not so many.
It is necessary to learn by themselves like an on the job training.
Preparation as a peer reviewer - PEPS Editors Blog (mostly in Japanese, but some references link to webpages in English)
Then, even by just looking at the peer review history of others, visualization of review records would help to know how to proceed with reviews.
It is not competition, of course, so the number or frequency of reviewing is not a big problem.
For the high-quality peer review, we can utilize this kind of system to enhance our motivation and to be evaluated in an appropriate manner. 

By the way, Publons has been very recently (last month) acquired by Clarivate Analytics
Also, for example, a major academic publisher like MDPI is collaborating with Publons on its peer review system.
It seems that the peer review evaluation services of Publons are becoming more popular.







Publonsの創設者にインタビュー:「科学加速化ミッション」 - Editage Insights 









IF=2-3 くらいの学術誌が突出して多いのですが、これはElsevier社のGeomorphologyの委員になっていることもあり、そこでよく査読を行っていることが影響していると思います。





査読者の心構え – PEPS Editors Blog

なお、ここで紹介したPublonsは、ごく最近、あのClarivate Analytics社に吸収されました。





PEPS大気水圏科学セクション編集委員の早坂 忠裕です。









PEPS科学セクション編集委員/東北大学 大学院理学研究科 早坂 忠裕


New Year Message from the Editor in Chief of Progress in Earth and Planetary Science

[Japanese text follows English. 日本語版は英語版の後に表示します。]

A happy new year to all

In 2016 PEPS received 60 manuscripts and published 37 papers. Since PEPS was only launched two and a half years ago we have not yet had time to finish our registrations with either of the main international research paper databases Scopus or Web of Science. In view of this I think we can regard the number of published papers as a modest sign of success. We have now completed our applications to both Scopus and Web of Science and these are currently being reviewed. I hope that I will be able to report sometime later this year that both applications have been accepted.

At the moment it takes on average 190 days from receiving a manuscript until it is published. The editorial team is focusing on “rapid review and publication” in order to reduce this time. The editorial office monitors the speed with which manuscripts proceed through the system and sends email notification to the relevant editors when there is any delayHowever the time to publication is largely dependent on the speed at which referees are able to review papers. I would therefore like to take this opportunity firstly to thank everyone who has or will review papers for PEPS for their hard work, and secondly to humbly ask that referees aim to complete their reviews as quickly as is reasonably possible.

We receive many manuscripts written by authors who are not native English speakers: for these manuscripts we have adopted a system of provisional acceptance. Under this system, manuscripts are sent out to referees and those by non native authors that are favorably reviewed are provisionally accepted and then sent to an English language proofreading company at the JpGU’s expense in order to correct any minor mistakes in expression or grammar. Although this all happens before the manuscript is formally accepted, in practice provisional and formal acceptance are nearly equivalent and we have so far published all of our provisionally accepted articles. The reaction to this process has generally been positive but we have had some complaints that manuscripts have been over edited, that the burden on authors is increased and that the system leads to delays in publication. There is some truth in these complaints. However it is our main goal that PEPS be regarded as a journal of the first rank: we are working hard to achieve this, and, I feel, have had much success in doing so. In order to maintain our status we have to consistently publish high quality papers where excellent scientific content is clearly presented and precisely explained. Unfortunately this requires, amongst other things, a certain level of English language competence. I apologize for the additional burden that this places on our non native authors, and hope that they will understand our position.

I have been aware for a while that Japanese institutions have been suffering from the rapid increase in online journal subscription fees. In the past few months there have been several international developments affecting this issue. At the end of last year negotiations between Elsevier and a group of German state funded universities and research institutions broke down with the result that from January 2017 researchers at these organizations will lose online access to many Elsevier journals. Similar problems have occurred in Taiwan and Peru (Nature News on 23 December 2016). Such problems serve to emphasize the important role that open access journals have to play in the field of academic publishing, and these developments further convince me that PEPS, as a high quality international open access journal covering all of the areas of Earth and planetary science, has an important role to play.

Recent years have seen the increasing popularity of letter journals, and indeed these offer a number of advantages: the compact presentation enables readers to quickly acquire an overview of material they are interested in whilst authors benefit from the relatively short time between submitting a paper and its publication. Despite these benefits, the importance of detailed exposition of new scientific ideas and results of course remains undiminished and this is why PEPS has chosen to focus on this area. We want our authors to explain their ideas in detail and we allow them up to 50,000 words per paper in order to do so (and, as I mentioned above, we are working to further reduce the time to publication of our papers).

The purpose of academic study is not only the creation of new knowledge. For it to be of any use such knowledge must be transmitted to the wider community, and we at PEPS are doing all we can to help with this. I would like to thank everyone for the assistance they have already given us and respectfully ask you all to continue to help, firstly by considering publishing new work in PEPS, and secondly by refereeing any articles we send you as quickly as is reasonably possible

Finally let me wish you all a happy, productive and prosperous 2017

Yasufumi Iryu
PEPS Editor in Chief

PEPS総編集長の井龍 康文です.2017年の年頭にあたり,皆様に御挨拶申し上げます.

2016年,PEPS37編の論文を出版しました.また,投稿を受け付けた原稿の数は60編でした.これは,PEPS創刊から28ヶ月ほどであること,Web of ScienceWOS)やSCOPUSという国際的な論文データベースに採録されていない現時点では,健闘している数値であると認識しています.なお,WOSおよびSCOPUSへの採録ですが,昨年,採録申請を行い,現在,審査を受けております.2017年に皆様に朗報をお届けできると期待しています.

現在,PEPSでは原稿受付から出版までに,平均で190日ほどを要しております.編集部では「Rapid review and publication」を心がけており,査読の遅れている原稿に対しては,編集事務局から編集委員の方々へ対応を求めるメールを送り,査読が迅速に進むようにしております.しかし,査読期間の長短は,査読者による査読の遅速にかかっています.査読者の御尽力に感謝するとともに,「Rapid review」達成のためにより迅速な査読を心がけていただくようお願いします.







井龍 康文