Results
The RERS Challenge 2020 has concluded with the formal announcement of the winners at the RERS event 2020 that was
held as a virtual event this year.
On this page we present the 6 teams that participated in this year's challenge and list the corresponding results.
In order to provide a more fine-grained view of the results, we also display ranking results per category for those teams who participated in the ranking.
For the competition however, only the results per track (see available tracks) were considered. For more details on the evaluation, please refer to the challenge rules.
List of participants
- Inria & ISTI-CNR
Frédéric Lang & Franco Mazzanti & Wendelin Serwe
Inria, Grenoble, France & ISTI-CNR, Pisa, Italy
Tools: CADP (developer), PMC (developer), KandISTI/FMC (developer) - LLNL
Markus Schordan
Lawrence Livermore National Laboratory, USA
Tools: CodeThorn/ROSE (developer) - LMU Munich
Karlheinz Friedberger
LMU Munich, Germany
Tools: CPAchecker (developer) - NUS & NITW & Huawei
Joxan Jaffar & Sangharatna Godboley & Rasool Maghareh
National University of Singapore, Singapore & National Institute of Technology Warangal, India & Huawei Heterogeneous Compiler Lab, Toronto, Canada
Tools: Tracer-X (developer), Frama-C - RWTH Aachen
Joshua Moerman & Philipp Berger
RWTH Aachen University, Germany
Tools: LearnLib, NuSMV, hybrid-ads, afl-fuzz, Frama-C
Links: description, repository - TU Delft
Tom Catshoek & Sicco Verwer
Delft University of Technology, Netherlands
Tools: AFL++, libFuzzer, NuSMV
Achievements
The following table shows the achievement certificates that were earned by participants of RERS 2020.
Category | Inria & ISTI-CNR | LLNL | LMU Munich | NUS & NITW & Huawei | RWTH Aachen | TU Delft |
Sequential Reachability: Plain | ||||||
Sequential Reachability: Arithmetic | ||||||
Sequential Reachability: Data Structures | ||||||
Sequential LTL: Plain | ||||||
Sequential LTL: Arithmetic | ||||||
Sequential LTL: Data Structures | ||||||
Parallel CTL: Small | ||||||
Parallel CTL: Medium | ||||||
Parallel CTL: Large |
Ranking evaluation
The following tables list the results of the RERS 2020 ranking evaluation. Drop-down lists below the tables allow to view ranking results for each category individually (if available). Note that these category-specific scores did not influence the ranking and are provided as additional information. Only the scores for each track were considered during the ranking. Note that a team's track score may be different from the sum of all corresponding track categories due to the non-linear penalty for wrong results.
For detailed information about the solved properties you can download a .zip archive that contains the (syntactically normalized) answers of those who participated in the ranking as well as the official solutions.
Sequential Reachability: Track ranking
Rank | Name | Score | Wrong answers |
1 | LLNL | 718 | 0 |
2 | NUS & NITW & Huawei | 714 | 0 |
3 | RWTH Aachen | 683 | 0 |
4 | LMU Munich | 312 | 0 |
5 | TU Delft | 248 | 0 |
Sequential LTL: Track ranking
Rank | Name | Score | Wrong answers |
1 | TU Delft | 123 | 0 |
2 | RWTH Aachen | 116 | 2 |
3 | LLNL | 63 | 0 |
Parallel CTL: Track ranking
Rank | Name | Score | Wrong answers |
1 | Inria & ISTI-CNR | 79 | 0 |
The following drop-down lists allow you to view the ranking evaluation for individual categories. Please click "view!" once selected.