Updates

  • 22/10/16 Solutions uploaded
  • 20/08/16 Parallel problems replaced
  • 07/08/16 Parallel problems released
  • 20/06/16 workshop date annouced
  • 13/06/16 Parallel release announced
  • 01/05/16 Problems up

Important Dates:

  • 09/10/16
       RERS workshop at ISoLA
  • 31/05/16
       Release Ranking Details
  • 01/05/16 - 15/09/16
       Challenge Phase
  • 01/09/16 - 15/09/16
       Solution Submission
  • 07/08/16 - 15/09/16
       Release Parallel Problem

Results

The RERS 2016 Challenge has ended with the formal announcement of the winners at ISoLA 2016. On this page we present the 6 teams that participated in the challengeand their results. In order to provide a more fine-grained view on the results, we offer the possibility to customize the evaluation process, allowing visitors to investigate the specific strengths of the competing approaches.

List of participants

  • Passau
    Karlheinz Friedberger
    University of Passau, Germany
    CPAchecker, Developer
  • Twente
    Jaco van de Pol, Vincent Bloemen
    University of Twente, Netherlands
    LTSmin, Developer
  • LLNL
    Markus Schordan, Marc Jasper
    Lawrence Livermore National Laboratory, USA
    CodeThorn, Developer
  • Radboud
    Rick Smetsers, Joshua Moerman, Mark Janssen and Sicco Verwer
    Radboud University Nijmegen, Netherlands
    Learnlib, AFL,
  • Freiburg
    Sergio Feo-Arenis, Michael Steinle, Marius Greitschus, Daniel Dietsch, Vincent Langenfeld, Claus Schätzle, Matthias Heizmann, Jochen Hoenicke, Tanja Schindler, Daniel Tischner, Christian Schilling, Betim Musa, Numair Mansur
    University of Freiburg, Germany
    Ultimate Automizer (Developer), Ultimate Brutalizer,Frama-C,
  • ISTI
    Franco Mazzanti
    ISTI-CNR, France
    KandISTI, Developer

Achievements

Category Freiburg ISTI LLNL Passau Radboud Twente
LTL:plain
LTL: arithmetic
LTL: data structures
Reachability:
plain
Reachability:
arithmetic
Reachability:
data structures

Ranking evaluation

The following selection allows you to view the ranking evaluation for each cateory. If detailed information about the solved properties is desired you can download a zip file containing the particpants normalized answers and the official solution file (detailed solutions containing counterexamples can be downloaded from the program download pages) here.

Reachability overall ranking

RankNameScoreWrong answers
1Freiburg3660
2Twente3340
3Radboud3280
3LLNL3280
5ISTI2291
6Passau2240