Results of the RERS 2012 Challenge
The RERS 2012 Challenge has ended during ISoLA 2012. On this page, we present the results of all five contestants. In order to provide a more fine-grained view on the results, we offer the possibility to customize the evaluation process, allowing visitors to investigate the specific strengths of the competing approaches.
Results
Rank | Name | Score | Total answers | Wrong answers | % correct |
1 | Twente | 25190 | 2437 | 52 | 97.9% |
2 | Paris | 13104 | 1486 | 10 | 99.3% |
3 | Vienna | 8433 | 938 | 36 | 96.2% |
4 | Passau | 5823 | 598 | 0 | 100.0% |
5 | Southampton | 5061 | 705 | 16 | 97.7% |
List of participants
- Passau
Dirk Beyer, Andreas Stahlbauer, Peter Häring
University of Passau, Germany - Twente
Jaco van de Pol
University of Twente, Netherlands - Vienna
Markus Schordan, Adrian Prantl
Technikum Wien, Austria / Lawrence Livermore National Laboratory, USA - Southampton
Jeremy Morse, Lucas Cordeiro, Denis Nicole, Bernd Fischer
University of Southampton, UK / Federal University of Amazonas, Brazil
Offline participation only - Paris
Pascal Cuoq, Virgile Prevosto, Boris Yakobowski
CEA LIST, France
Offline participation only
Customize evaluation
The following form allows you to customize the evaluation of the results, imposing other rules than the official ones. This is offered due to our idea of providing a profile of the tool landscape, providing a more fine-grained overview on the specific strengths and weaknesses of the contestant's tools.
The following values can be changed:
- Property types: Restrict evaluation to only reachability or behavioral (LTL) properties, or take both into account.
- Problem selection: Only include certain problems for the evaluation.
- Respect confidence: If checked, then the confidence values supplied by the contestants are taken into account. If not checked, than confidence 1 is assumed for all answers.
- Penalty factor: By default, wrong answers are penalized by subtracting twice the confidence value from the total score. Enter another number to change this factor.
- Double score for online problems: The score for the online problems 14 through 19 counts twice in the official evaluation. Uncheck to disable.