Organising a Summer School: “How to be a peer reviewer”

The objective of this summer school is to provide academic-led information about the peer review process. There are limited opportunities for early career researchers to understand this essential process and sometimes getting insight, advice and/or training on how to conduct and respond to peer reviewing depends on informal networks and willing supervisors/line managers that not all individuals will have access to. The speakers include academic members of editorial boards, editors, publishing directors, and senior career consultants. The summer school is addressed to PhD students and postdoctoral researchers.

Participants should get an insight into the peer review process and learn how to be a constructive reviewer and how to respond to reviewers. Moreover, the summer school should be key in raising issues related to the peer review process such as authorship, plagiarism and ethics issues while highlighting biases in knowledge production and sex/gender considerations.

Early career researchers in this summer school will:

• Be able to understand what different review processes entail (e.g. article reviews, funding proposals);
• Learn how to be a constructive and effective reviewer;
• Learn about important aspects of reviewing, including biases in knowledge production and sex/gender considerations;
• Learn how to respond to reviewers
• Enhance their understanding of: the peer review process and how to get involved at an early career stage; benefits of becoming a peer reviewer (becoming a better writer; keeping updated with work in their field; feeling member of a community; networking; contributing to the wider community; step towards editorial boards –career development; small perks: discounts etc).

Organising a summer school open to PhD candidates and early career researchers will provide access to such information and training to all early career researchers irrespective of networks and personal contacts.

When the response and participation of participants is overwhelming it can lead to the recognition of the importance of offering training and insights into peer review process at departmental/institutional level with the view to it becoming embedded in the university training activities.

A pilot half day workshop on peer review process took place on the 29^th of May at the University of Warwick to gauge interest about this topic and get feedback about the content and structure of the subsequent summer school. PhD candidates and early career researchers expressed great interest and provided useful feedback for our summer school which informed the final programme. In this half day workshop, 2 academics (1 Editorial Board member and 1 member of a Research Council Peer Review College) and 1 Senior Editor (from Warwick’s academic journal) provided a brief insight into the peer review process. In the second half of the workshops, participants peer reviewed conference abstracts and engaged into a discussion about how to provide constructive feedback.

Based on this experience and feedback, we organised the PLOTINA summer school took place from September 10^th to 14^th, 2018, at the University of Warwick. The first two days (10^th, 11^th) were targeted at early career researchers from natural sciences and engineering and the 4th and 5th day (13^th, 14^th) were targeted at those from social sciences and arts and humanities. Each session was limited to 35 participants in order to enable active participation of all attendees. The morning sessions included talks from academics and representatives of publishing houses and scientific journals focused on the peer review process, from submission of a paper/funding proposal to the outcome of the peer review while the afternoon sessions were practical, hands-on workshops on how to conduct peer review, led by experienced editors and peer reviewers (see programme). There were also 2 dedicated sessions on considerations of the peer review process in relation to ethics, plagiarism, conflicts of interest and most importantly biases in the peer review process and considerations of sex and gender in research and publishing. The 3rd day (12^th) was comprised of a writing bootcamp and a CV workshop where both natural and social scientists had access to a quiet room with desks, lunch, refreshments and experienced facilitators to help them review papers, proposals, and CVs. Participants were invited to ask questions and raise any concerns and were given time for networking.

After the summer school, the material used by speakers was uploaded to the summer school’s website and participants were informed via email.

We chose to have a mix of internal (from the University of Warwick) and external speakers. Due to the rich academic expertise of Warwick staff, we contacted academics with long experience in editorial boards and evaluation of funding bids (from various departments including Chemistry, Life Sciences, Sociology,Economics). We also identified 9 external speakers who held senior positions in academic publishing (Elsevier, Nature; Springer Nature) and/or had extensive experience and recognition in peer review process, but also had been involved in initiatives of peer review which touched upon issues: biases, sex and gender or diversity (speakers from the European Association of Science Editors (EASE) and gender studies’academics).

See overview of the programme: https://warwick.ac.uk/fac/soc/pais/research/projects/plotina/summerschool/programme/

What is required for the organisation of such an event:

Before the event

• Decide and develop the content of the summer school, structuring the programme, drawing the budget
• Identify and invite speakers 4-5 months before the date of the event to ensure that they are available and they can commit to contribute to the programme
• Oversee the whole event (eligibility for budget costs, liaising with departments)
• Liaise with speakers for accommodation and travelling, provide expense claim form and be explicit about what costs will be covered
• Arrange the logstics of the event (food/coffee/lunch – providing some catering makes participants feel valued and more involved), laising with catering services
• Develop material (name badges, signs for event, folders with info on the programme and speakers)
• Develop relevant webpages for the aims of the event, speakers, programme and registration page;
• Promote and disseminate the event internally (within the institution) – see section Communication – if that is the target audience. Alternatively, address your external audience. Develop a poster and a blurb with the main objectives, structure and target group is important to attach in your communication.

A couple of days before/on the day of the event

• Send reminder email to speakers with the final programme, venue information and sending presentations/material in advance
• Send reminder email to participants about the final programme, venue information and registration
• Develop material (name badges, signs for event, folders with info on the programme and speakers for registration
• Get and print material for the workshops
• Ensure technical equipment and venue appropriateness in advance (go before the event to check that computer/projector/microphone are available and working.
• It is important to have 1-2 individuals available on the day(s) of registration

After the event

• Follow up with participants after the event; send them an evaluation form to provide feedback and also provide information about the availability of the slides and additional resources provided by the speakers and the organiser
• Follow up with speakers to ensure that they give consent for uploading their slides (they might want to amend them slightly)
• Write a thank you note to all speakers for all their help. For external speakers, remind them to send all their receipts for costs claimed
• Collect all costs and receipts to finalise the budget required

1. Participants may not think the discussion about biases in knowledge production and sex/gender considerations is relevant to them.
2. Getting highly esteemed speakers to contribute to this programme (they have very busy schedules).
3. People signed up to attend and then did not bother to come.

1. Be clear about the benefits of attending such an event and ensure that you get and respond to feedback from participants beforehand and during the programme.
2. Start planning well-ahead of the event and motivate potential speakers by the benefit to early career researchers.
3. Charging a small registration fee can encourage commitment and/or discourage casual sign-ups.

We circulated the invitation for this action through various newsletters (university-wide), including the Learning and Development Centre (LDC) and the Doctoral Training Centres at the University of Warwick. We contacted departmental administrators across various faculties and departments to promote the event to early career researchers. We also disseminated the call for participation to other universities within reasonable travelling distances, including the University of Birmingham, the University of Oxford, the University of Southampton, and Aston University, though we stopped advertising when the event was full.

Forty seven PhD students and postdoctoral researchers participated in the summer school. Speakers from a diverse array of backgrounds shared their experience and provided valuable insights into the peer review process, including sex/gender issues. Participants had the opportunity to enhance their knowledge on the peer review process, attending talks on the peer review process and funding bodies, how to respond to peer reviews, and how to write peer reviews. Participants also participated in a writing boot camp and attended a CV workshop and a peer review training course (delivered by Dr Julia Vilstrup Mouatt, head of Publons’ peer review training course, the Publons Academy). An evaluation form was delivered to participants.

The summer school was well received. We were overwhelmed with interest for the event (70 individuals registered) and we had to put participants on a waiting list (+20). The delegates who filled out the evaluation form (29 filled in the form) gave positive feedback. Among the topics they found particularly interesting and beneficial were the following: talks and discussion on how to conduct peer reviews, discussion about gender imbalance in the research context, and the practical sessions. Participants also mentioned that they benefited from the speakers’ variety of disciplines and backgrounds and commented positively on how well prepared the speakers were.

Participants highlighted that the event was very comprehensive and it covered everything about peer review. Some of them would have liked to participate in more practical sessions and hands on exercises  e.g. review longer articles or real articles submitted to the Warwick journal.

An evaluation form was designed on the summer school’s website (an evaluation form template is available and it can be used to develop an online form print and disseminate it at the end of the event). Participants were informed verbally about it at the end of the summer school and via email. They also received a reminder email a week later. The form was simple and brief including open ended questions to encourage participants to fill the form and expand on their feedback. The form was comprised of:

• Which topics or aspects of the summer school did you find most interesting or useful?
• What else do you feel should have been included? What could be omitted?
• Do you have any further comments/ suggestions?

Think about capacity and how many participants you can accommodate. While the talks can be open to a larger number of participants, when it comes to practical workshops, it is more difficult to run them with more than 35 participants (considering that they often do group work, 5 groups of 7 people can work well). We had an overwhelming response in registration (90 tried to register, we allowed for 70 and we had 20 people in the waiting list). However, there was a big dropout rate so we recommend allowing for 10-15% over-subscribing to mitigate this. For this reason, you might also want to consider adding the following statement in the registration form for running such a free event ‘By registering you are holding a place at this event. You need to notify the organisers as early as possible if you cannot attend so your place can be offered to another participant. Failure to do so might lead to a charge of x amount’. A small registration fee (e.g. to cover food costs) might be the best way to gain commitment.

Due to time constraints and hectic schedules, events like this could be run in 1.5 days tailored either to specific disciplines or broader disciplinary areas. Participants appreciated disciplinary expertise (at least broad disciplinary expertise) and would have valued a more ‘compact’ schedule to get the most of it. However, the half day pilot programme was a bit too compressed and participants did not get to try it out.

Develop the content of the programme and identify speakers 4-5 months before the day of the event.

Check whether you can capture (audio/video) the talks and get consent (from speakers and participants) to develop a resource that could be accessible to the wider community.

Budget for: food/coffee, travel/accommodation expenses for speakers, printing and folders, administrative assistance.

Researchers and research: gender equality and sex and gender perspective

Training/workshop

University of Warwick (WARWICK)
Higher education institution

Researchers and professors