In OpenWater there are two different methods to judge submissions. The first, and most common is the Evaluation Scorecard. The second is Ranking.

You can mix and match evaluation scorecard and ranking from round to round in your program if you wish. For example, you may hold a normal evaluation scoring in round 1 and then in round 2 you task your judges with ranking their favorites from 1st place to 5th place.

Below is a description of each approach.

Commonly known as a scorecard, scoring, or rubric, the judge is presented with the application on the left side of their screen and the evaluation criteria on the right. Judges are asked to score the submission and add comments if desired. We have different styles of scorecard questions which are outlined here.

With a scorecard, it is possible to have weighted questions, meaning that some questions are worth more towards the total score than others. The score for each question is added to the submission's total. So if there are five criteria and the maximum score per question is five points, than the maximum possible score a submission can get is 25 points. The results of judging will average the total scores that each judge gave that submission, and naturally the submissions with the highest averages will float to the top.

In addition to the above, we also support the style where you allow judges to score a submission form field right below the applicant's response, rather than having the questions on the right side of the screen.

Ranking is also known as a ballot. The concept behind ranking is any situation where you instruct judges to choose your top 5, or rank your preference from 1st place to last place, etc, etc. Submissions are stacked against each other instead of judged on an individual basis.

Within ranking, we support two modes -- order matters, or the order does not matter. For some programs, you need to choose 1st place, 2nd place, 3rd place, etc. But for others, there is no order and it is simply choosing your bag of favorites.