Configuring review

 

Before submissions can be allocated to reviewers, you will need to configure the reviewing system to match your review process. There are 4 main steps to be completed before you can Allocate submissions to Reviewers:

1. Complete the Configure Review menu

2. Complete the Setup Marking Scheme menu

3. Complete the Configure Acceptance menu

4. Add Reviewers and either assign topics to them or invite them to pick their own topics

 

To get started, find your Setup Review card on your dashboard and click on the Configure Review Settings button.

 

 

Completing the Configure Review menu:

 

This is where you will add in your important dates and configure settings that impact the allocation.

Step 1: Important Dates

You can set the dates for when you intend on starting your review, ending your reviewing and notifying authors of your decisions. These dates can be thought of as reminders for your review timeline, but they do not trigger any actions. You will take action to formally open the review, end the review and notify authors. 

 

 

 

Step 2: Reviewing and Allocation

 

This step is very important. It is where you configure settings that will impact the allocation and reviewer experience.  

 

Submission Allocation Setup:

You will need to choose a number for the following options:

• number of reviewers per submission: Do you want 2 reviewers to look at all submissions, 3 reviewer, 4? 
• number of maximum reviews per reviewer: What is the maximum number of submission a single person should get? 

 

 

 

These numbers are very important for the allocation. Your allocation might have many empty slots if you do not carefully consider how many submissions and reviewers you have. For example:

• I have 30 reviewers. 
• I have 100 submissions and I want 2 reviewers to look at each. This means there will be 200 reviewer assignments to fill. 
• I want to ensure my 30 reviewers do have to review more than 5 submissions total. 
• 30 reviewers X 5 submissions maximum = 150
• I do not have enough reviewers to support these settings. I would need to find more reviewers or change the numbers. 

 

There are a few things that impact the allocation numbers including:

• Tracks and Review groups: If a reviewer can review in multiple tracks and/or review groups, the total maximum number of submissions is multiplied by the number of tracks/review groups. 
• Topics: It is important to ensure you have enough reviewers for each topic. If you have a single reviewer for topic 1 and 100 submissions with topic 1, the maximum number of submissions per reviewer would need to be 100 or unlimited; or you could find more reviewers to assist with topic 1. 

 

Panels Allocation Setup (panel sites only):

 

If your site has the panels feature enabled, you have 3 choices for the allocation of your panel contributions. The 3 panel allocation models are:

• Convenor review: panel organisers that are listed as convenors on the panel review all contributions inside of their panel
• Same reviewers model: reviewers are assigned to panel contributions if they share topics, but the system ensures the same reviewers get all of the panel contributions to ensure they have the complete picture
• Different reviewers model: reviewers are assigned to review panel contributions based on topic match only, meaning different people may be assigned different contributions.

 

Nepotism settings:

Please indicate if you wish to prevent or allow nepotism. If you prevent nepotism, the system will not assign a reviewer to a submission when:

• the reviewer and an author share the same email domain (e.g., @gmail.com)
• the reviewer and author share the exact same affiliation
• the reviewer and author are connected on another submission, meaning they are co-authors on a different submission

 

 

Blind Review Settings:

You can choose to hold a single or double blind review. Choose single blind if reviewers should see author names, and double blind if they should not. They system will never show authors the names of reviewers.  

 

 

Decline Assigned Review:

You can allow reviewers to a decline a review or not here. Allowing this enables reviewers to hit the Decline button on a specific review. They are asked to enter a reason in a text box. All admins on the site will be emailed, and any admin can log back in and assign the review to a different reviewer. 

 

 

That's it! After you click Done on this step, you will be taken to your dashboard. Feel free to take a break or jump right into the next review menu by clicking the Configure Marking Scheme Settings button from your Setup Review dashboard card.