Reviewing

(last update: January 1st, 2024)

Here are some reviewing stats for nerds.

Method

The following stats are computed by accounting for all the reviews I worked on (either as a reviewer, subreviewer, or PC member) since 2020. They only include reviews for “full papers” (for journals, conferences, or workshops), and do not consider either (i) research grants, (ii) revisions/shepherding of papers, (iii) theses – all of which are also documents that I will gladly review, but which would skew the results.

To compute the numbers presented below, I devised a Jupyter Notebook which analyzes all the reviews I wrote (which I meticulously store on my own devices prior[a]As such, these numbers represent the review “at submission time”, and do not account for edits or changes that I may make afterward. to submitting them) and extracts some metrics.

If you are interested in knowing the venues I review for, you can look at my Service page.

Overview

First, a couple of figures that showcase my reviewing efforts.

Word Distribution per review (yearly)

Yearly Reviews

Observation. Starting from mid-2022, I changed the way I review papers. Specifically, I have begun “quoting” (verbatim) some of the statements written in the paper, and use such quotations as basis for my remarks. Despite being quite time-consuming, I found that such an approach can be very helpful – both for myself, but also (hopefully) for the authors, since it precisely shows the parts of the paper that are problematic (according to my own judgement). Of course, such an approach also leads to longer reviews.[b]I do not filter-out the quoted statements, since it would be impossible to clearly distinguish them from the rest of the content.

Detailed metrics

Below are some specific metrics per year.

YearReviewsLetters (tot)Words (tot)Avg. LettersAvg. Words (std.)
202025128 87120 7295 154829 (286)
202166345 15355 5785 229842 (472)
20221331 250 387208 4029 4011 566 (912)
20231141 369 281255 31112 0112 239 (808)
(all)3383 093 692540 0209 1521 597 (944)