Scores Help Sat Dec 23 01:59:54 EST 2006 Here we describe the scores of individual problems. See the `scoreboard' help file for a description of how these individual scores are put together into a total score for each team or individual contestant. Results Come by Email: --------------------- You will get your results back by e-mail. Correct results will also be posted on the scoreboard (see below). Whatever you submit using `make submit' will be Cc'ed to your account. If you are using a personal account not provided by the contest, you may not be using `make submit', but may be simply sending email yourself to the autojudge, in which case it is your responsibility to Cc the mail to yourself if you wish. Asking Questions: ---------------- You may e-mail questions to the judges but the method to do this from a contest provided account is a little weird. First, edit your email into a file of your own choosing. If your first line has the form Subject: .... and the next line is completely empty (no spaces), then the first line will be the subject line of the message. Then hpcm_sendmail < filename to send the file as email to the judges. Note that the subject must NOT begin with the words `submit' or `get'. If you are using your own personal account not provided by the contest, just send ordinary mail to the auto- judge, taking care that it's subject does not begin with `submit' or `get'. A human judge will be notified and eventually respond. If you are using a personal account with the UNIX tools described in the `email_unix_tools' help file, you can also use hpcm_sendmail if you choose. If you ask for a clarification by e-mail, and a signifi- cant answer is given, that answer will be given by post- ing it on the scoreboard so everyone can see it. You will not receive email back in this case. What Scores are Possible ---- ------ --- -------- The possible scores for a particular submission are: Completely Correct Formatting Error Incomplete Output Cpu Time Limit Exceeded Output Size Limit Exceeded Program Crashed Incorrect Output The first score, `Completely Correct', means the sub- mission is correct. ALL the other scores mean the sub- mission is incorrect. Sometimes the word `accepted' is used instead of `correct' and the word `rejected' is used instead of `incorrect'. With this terminology, the score `Completely Correct' means the submission was accepted, and ALL the other scores listed above mean the submission was rejected. What Do the Scores Mean ---- -- --- ------ ---- Scoring is done by running the judge's input through your program to produce `your output', which is compared with the `judge's output', that is produced by running the judge's solution on the same input. With this in mind, the precise meanings of the possible submission scores are as follows: Completely Correct Your output was essentially the same as the judge's output. For some problems, `essentially the same' means exactly the same, character by character. For problems with floating point numeric output, numbers may not differ by more than a tolerance specified in the problem description. If the number of columns occupied by numbers and other output is specified by the problem descrip- tion, then numbers are required to end in the cor- rect column, even when they differ. If the number of decimal places in output numbers is specified, your numbers and the judge's numbers must have the same number of decimal places. If the presence or absence of exponents in output numbers is specified, your numbers must agree with the judge's numbers on having or not having an exponent. The absence of an exponent is implicit if an absolute number toler- ance is specified, such as 0.001; but specification of a relative number tolerance, such as 0.01%, implies that exponents are optional. If white spaces in the output are specified by the problem description, no extra spaces are allowed and no required spaces may be missing. Most problem descriptions specify the number of lines to be out- put, in which case no extra blank lines may be output and no required blank or empty lines may be missing. `Completely Correct' is the ONLY score that means your solution was accepted by the judges, and you have solved the problem. All other scores mean your solution has been rejected by the judges. Formatting Error Your output has the right words and the right num- bers (within any designated tolerance) in the right order, but there are unallowed differences in white- space, columnization, letter case, number of decimal places in numbers, or presence or absence of expo- nents in numbers. These differences can be as little as one extra blank line in your output! Although omitting a blank line is usually a Format- ting Error, if the line is used to separate test cases, omitting it may be an `Incorrect Output' error. For problems in which only the numbers really mat- ter, omitted, extra, or misspelled words may be considered to be just formatting errors. However, even in this case words like `Case' and `Data Set' that announce the beginning of a new test case in the output are considered to really matter, and if incorrect, will cause an `Incorrect Output' score and not a `Formatting Error' score. Incomplete Output Your output is not empty, and is either completely correct or has only formatting errors, but your out- put stops prematurely. Cpu Time Limit Exceeded The program was terminated by the system because it exceeded the CPU time limit imposed by the problem Makefile. If you open files instead of using the standard in- put and output you may get a Cpu Time Limit Exceeded score: see Common Mistakes in the solving help file. Output Size Limit Exceeded The program was terminated by the system because it exceeded the amount of output allowed by the problem Makefile. Program Crashed Your program did not produce any output, or your program sent output to the standard error stream, or your program terminated abnormally; but your program did not exceed either CPU time or output size limits. Exceeding memory limits will cause this score, because it is unfortunately difficult to distinguish memory limit exceedments from other kinds of program crashes (e.g., using unallocated memory). If you open files instead of using the standard in- put and output you may get a Program Crashed score: see Common Mistakes in the solving help file. Incorrect Output The program did not crash or terminate prematurely, and some word or number in your output does not match the corresponding word or number in the judge's output, even when letter case and whitespace are ignored and number differences within any toler- ance stated in the problem description are ignored. Omitting a blank line that is used to separate test cases may be an `Incorrect Output' error instead of a `Formatting Error'. In general, failure to output correctly lines that begin or separate test cases may be an `Incorrect Output' error, instead of a `Formatting Error'. The autojudge is generally very good at giving the right score, with the exception of certain cases where the autojudge scores `Incorrect Output' but a human judge would score `Formatting Error'. Misspelled, omitted, out-of-order, or extra words may be such cases. Although a human judge who is manually reviewing scores could change the score in such a case, she does not for most contests, because the extra effort to do so is not worth the minimal benefit to the contestants, who should find it fairly easy to catch such errors, and should not need the extra hint involved in changing the score. File: scores Author: Bob Walton Date: See top of file. The authors have placed this file in the public domain; they make no warranty and accept no liability for this file. RCS Info (may not be true date or author): $Author: walton $ $Date: 2006/12/23 07:03:53 $ $RCSfile: scores,v $ $Revision: 1.16 $