Before you invite reviewers to the site to begin their reviews, you need to setup the marking scheme that they will use to grade submissions. Find the Setup Review card on your dashboard and select 'Configure Marking Scheme Settings'.
You can also navigate to your Reviews hub and find the menu item Setup Marking Scheme.
You will be brought directly to the first step in setting up the marking scheme.
Step 1) Scoring Categories
Here you can add or edit the categories listed to personalize it to your preferred marking scheme. You can add a category heading and a description beneath it. You can read more about scoring categories here.
Step 2) Scoring Grades
Once you have added the questions that the reviewers will be asked, you can provide guide answers to help them. For each of the questions you created, add in the appropriate text in the 1-5 boxes. You can change the name of the grade (in green below) and the explanation of each grade (in white).
Step 3) Category Weighting
Now, if you want a certain category's answers to weigh more in importance than another you can use this feature. On this page click on the 'Enable' button at the top of the page. Now all you have to do is click on the grey box and move the scroll bar to the right (if it's more important) or the left (if it's less important) to put the correct weighting on each category.
Step 4) Extra Settings
Once you're happy with the score weighting, there are some extra settings that you can configure to personalize your marking scheme even further. Just click on "Step 4: Extra Settings" and you will then be brought to this page:
As you can see, there are 4 settings that you can configure.
A) Will reviewers be asked to complete a summary about the submission? You can then decide if this is required or not required by click on the radio buttons beside the questions.
B) Is it required for reviewers to provide feedback for the authors? If this is Required, reviewers will see a text box where they must enter comments or upload a file with their comments. You can also make this Optional in case only some reviewers have feedback. You can then share these with authors after the review. If you turn this Off, the reviewers will not be asked for any feedback.
C) Are reviewers obligated to provide private comments to the conference chair? If this is Required, reviewers will see a text box where they must enter some text. This text will only be visible to the administrators of the conference. If you make this Optional, the reviewers can enter some comments if they feel the need. You can also switch this Off so reviewers don't have any space to give comments to you.
D) Would you like to show the individual scores from the reviewers to the authors? These scores will only be shown once the review is completed. Click on "Show Scores" or "Hide Scores" to set it up in accordance with your preferences.
Step 5) Preview
Lastly, to preview what the marking scheme will look like to reviewers, click on "Step 5; Preview" or click on "Done: Go To The Next Step" . You will then be brought to the preview page that will look something like this:
In the final step you can click on the green button 'Edit review', and you'll see what your reviewers will see here. You can click on a description of each category and their scoring options, an it will look something like on the picture below, depends how you configured your scoring categories and grades:
Then click on Done and ta-dah! You've successfully personalized your marking scheme.