Scoreboard Help Thu Sep 28 19:29:21 EDT 2006 The scores of individual problems are described in the `scores' help file. This file describes how individual problem scores are put together into a scoreboard. The Scoreboard -------------- If you do not have a contest account provided by the contest judges, you will typically have been given a web URL at which a one page contest scoreboard is displayed. But in a contest account provided by the contest judges the scoreboard is best displayed by allocating a separate window to show the scoreboard and in that window running the command: scoreboard This command will show the contest scoreboard refreshing every 30 seconds. The command is best executed in its own window. To terminate type control-C. The scoreboard may have several pages, and in such a case this command cycles through all the pages, going to the next page every 30 seconds. scoreboard N This command just displays the N'th scoreboard page, refreshing every 30 seconds. scoreboard CONTEST-NAME scoreboard CONTEST-NAME N Sometimes one account can be used to enter many different contests. In this case you have to identify which contest you want to see the scoreboard of. Usually the `scoreboard' command by itself will list all the possible contests. The part of a scoreboard that shows team scores may be frozen during the last hour of a formal contest. This means the scoreboard only shows results as of the time when it was frozen, which is one hour before the contest ends. This is done to create suspense, so people will come to the post-contest meeting at which the final con- test results will be announced. Ranking Submitters ------- ---------- The scoreboard gives for each problem and submitter in- formation about the submissions made by the submitter for the problem. For each submitter the scoreboard typically gives the total number of problems correctly solved by the submitter, and a ranking score that ranks submitters who have the same number of solved problems. A submitter (contest account) may be a team or an in- dividual, depending on the type of contest. Here we use `submitter' to mean either `a team' or `an individual'. Some contests are completely unscored. For these the submitters are listed in alphabetical order, and for each problem and submitter an indication is given of whether the submitter has solved the problem, and if so, the date the problem was solved. On the other hand, if the contest is being scored, sub- mitters with more correctly solved problems appear earlier in the scoreboard. When a contest is being scored, there may be a way of ranking submitters who have the same number of solved problems. If there is no ranking for a scored contest, submitters are listed by alphabetical order within the group of submitters all of which have the same number of solved problems. There are two different methods of ranking that can be used. The classical way, used in all limited duration contests, is to measure the time required to solved each problem. This is called `timed' ranking. The other way, usable in untimed contests, is to use the kinds of feedback provided for incorrect submissions. Depending upon the submission type, either the judge's input or both the judge's input and output for the first failed test case can be provided for a submission. This is called `feedback' ranking. For either kind of ranking, each solution is assigned a numerical score, these scores are either added or aver- aged to produce a score for each submitter, and problems a submitter never solves do not count in any way toward scoring or ranking submitters. Below we discuss in separate sections timed, feedback, and unranked contests, so you may read just the appro- priate section. However the following is common to all kinds of contest. A scoreboard has an entry for each submitter and each problem. If there are no submissions for a problem by a submit- ter, `......' appears in the scoreboard for the submit- ter and problem. This also appears if there are no correct submissions and incorrect submissions are NOT being reported. But if incorrect submissions are being reported and there are some for a problem not yet cor- rectly submitted by a submitter, then `..../N' appears, where N is the number of incorrect submissions by the submitter for the problem. For each submitter in a ranked contest the number of solutions N and ranking score RRR of submitter are reported in the format `N/RRR'. For timed contests higher RRR is worse, but for feedback contests higher RRR is better. For scored but unranked contests, only N is reported, and there is no `/RRR'. In some contests, problems are scored first by an auto- judge, and then if they are not completely correct, the score is reviewed by a human judge. If you see a `*' prefixed or suffixed to any item of scoreboard informa- tion, this means that one or more submissions used to compute the item have been scored incorrect by the auto- judge, but are still awaiting review by the human judge. Formal Timed Contests ------ ----- -------- A formal contest is a contest of limited duration in which the problem descriptions are passed out at the start of the contest. Each solved problem is given a score that is the difference between the time the correct solution was submitted and the start time of the contest. In a formal contest a submitter's ranking score is the sum (NOT the average) of all the solution scores for the solutions submitted by the submitter. Lower time scores are better, higher time scores are worse. Also, if you think of problems as being solved consecutively, and a submitter solves, say, three problems, the first in time T1, the second in time T2, and the third in time T3, then the submitter's ranking score is 3*T1 + 2*T2 + T3, so it pays to solve the easiest problems first. Solution times are computed in seconds. Some contests apply a penalty of typically 20 minutes (1200 seconds) for each incorrect submission of a problem made before the first correct submission of the problem. Solution times may be represented on the scoreboard as `M:SSsN', with decimal numbers M, SS, and N, meaning that the solution time is M minutes and SS seconds, and there were N submissions, including N-1 incorrect sub- missions, before the first correct submission. The `s' means the time SS is in seconds. N may be omitted if incorrect submissions are not being reported on the scoreboard. `H:MMmN' instead means the time is H hours and MM min- utes, rounded down to the nearest minute. `D:HHhN' instead means the time is D days and HH hours, rounded down to the nearest hour. `DdN' means the time is D days, rounded down to the nearest day. The choice of time unit, s, m, h, or d, is made to give the elapsed time to the greatest accuracy within 6 display charac- ters, including the unit character. However the actual solution time used for computing a submitter's ranking score as described above is always in seconds. Informal Timed Contests -------- ----- -------- Informal contests derive timed scores without having a single contest start time or giving out paper problem descriptions. In an informal contest a submitter gets a problem named pppp by executing hpcm_get problems/pppp This obtains a description of the problem as a file in the submitter's problems/pppp directory. In general, the problem score for a timed contest for a submitter is the difference between the time the submit- ter submits a correct solution and the start time of the problem. There are two ways of scoring informal con- tests, depending on how problem start times are assign- ed. If the problem start times for a particular submitter are ALL the first time the submitter gets ANY problem in the contest, this time acts as the start time of the contest for the particular submitter, and the ranking score is the same as that of the formal contest above. In this case, an informal contest is just like a formal contest except each submitter (each team) can start at a different time. If a start time for a particular problem and submitter is the time the submitter hpcm_get's the problem, the ranking score is the AVERAGE of the problem scores, which is just the average time to solve any problem the submitter has solved. Again lower scores are better, higher scores are worse. In contests of this kind there is often a maximum problem score, i.e. a maximum problem solution time, so that any problem whose solution takes longer than this maximum is counted as if it took just this maximum amount of time to solve. Feedback Contests -------- -------- When a scored contest is not of limited duration and problems are not being timed, three different kinds of submission may be allowed, and different penalties are assigned for each kind of submission when that submis- sion is incorrect. The methods for making different kinds of submissions are as follows: Contest Provided Personal Account; Account Command: Send email with: make submit Subject: submit ppp.ext make in-submit Subject: submit ppp.ext in make inout-submit Subject: submit ppp.ext inout make solution-submit Subject: submit ppp.ext solution You use the `make' commands in an account provided to you by the contest managers, and you send email with the given `Subject:' fields from your own personal account that is not provided by the contest. You can also use the `make' commands in a personal account if you use the email UNIX tools: see the email_unix_tools help file. In what follows we refer to the different types of sub- mission as `submit', `in-submit', `inout-submit', and `solution-submit'. The penalties typically assigned for different submis- sion types in feedback scoring are: submit 10% in-submit 20% inout-submit 30% `Submit' returns just the submission score. `In-submit' also returns the judge's input for the first test case that was incorrect, when this is practical. `Inout- submit' returns both judge's input and judge's output for this test case. The score for each solution starts at 100.00, and is multiplied by 90% for each incorrect `submit', 80% for each incorrect `in-submit', and 70% for each incorrect `inout-submit'. Thus each penalty is applied to the re- maining score for a problem. Note that only incorrect submissions before the first correct submission count. A submitter's ranking score is the average (NOT the sum) of the solution scores for all problems solved by the submitter. `Solution-submit' does not affect scoring. It returns the judge's solution if the score is `Completely Correct'. After receiving a `Completely Correct' score for some other type of submission, you may resubmit using `solution-submit' to get the judge's solution without affecting your problem score. If incorrect submissions are being reported, then for each solution the solution score and the number of incorrect submissions are both reported using the format `IIiOOoXX:SS'. Here the upper case letter pairs denote numbers and the lower case letters `i' and `o' appear literally on the scoreboard. II is the number of in-submit' incorrect submissions, OO the number of `inout-submit' incorrect submissions, XX the number of `submit' incorrect submissions, and SS is the solution score rounded to the nearest integer. If there are no incorrect submissions, just `100' is reported instead. If incorrect submissions are NOT are being reported, then only SS is reported. Although SS is rounded to the nearest integer, in compu- ting the ranking score of a submitter, the solution score to many decimal places is used. Unranked Contests -------- -------- If a contest is unranked, there are no ranking scores, and the entry for a problem solution is just the date of the solution, followed by `/N' if N is the total number of submissions up to and including the first correct submission, provided that incorrect submissions are being reported. The scoreboard for an unranked contest may or may not report the number of correct submissions each submitter has. If it does, the submitters are sorted first by their numbers of correct submissions, and then alphabe- tically by submitter name, and the scoring is called `unranked scoring'. Otherwise submitters are all just sorted alphabetically, without reporting the number of correct submissions for each submitter, and the contest is said to be `unscored'. File: scoreboard Author: Bob Walton Date: See top of file. The authors have placed this file in the public domain; they make no warranty and accept no liability for this file. RCS Info (may not be true date or author): $Author: walton $ $Date: 2006/09/28 23:31:48 $ $RCSfile: scoreboard,v $ $Revision: 1.19 $