Scoring and Standings 2017/2018

The ERL Service Robots is composed of multiple Local Tournaments, held in different research labs across Europe, with certified test beds, and a few competitions as part of Major Tournaments, such as RoboCup. Teams participate in a minimum of 2 tournaments (Local and/or Major) per year and get scores based on their performances. A final end of year score is computed for each team, per Task Benchmark (TBM) and Functionality Benchmark (FBM), using the best two participations in tournaments, and teams are ranked based on their final score. More information can be found on ERL Service Robots scoring system document.

Prizes for the top teams (per TBM and per FBM where they have participated) will be awarded during the next European Robotics Forum (ERF) in Tampere, Finland, 13-15 March 2018.

The current scoring and standings for the ERL Service Robots Challenge (last update: 14/09/2017) are the ones below:

Task Benchmarks (TBM)

TBM1:  Getting to know my home

Sorted Based on Median

Team

A

P

Homer

6

0

Socrob

5

3

BARC

2

1

 

A - Achievements

P – Penalties

Light gold indicates the team has the minimum required (2) participations

 

TBM2:  Welcoming Visitors

Sorted Based on Median

Team

A

P

Hearts

5

0

BARC

4

0

 

TBM4: Visit my home (Navigation)

Sorted Based on Median

Team

A

P

Socrob

3

1

Homer

3

1

BARC

0

0

 

           

Functionality Benchmarks (FBM)

FBM1: Object Perception

Sorted Based on Median

Team

Class

Instance

Position

Time (s)

BARC

0.85

0.7

0.329

100.06

Socrob

0.8

0.6

0.569

47.3

Hearts

0.7

0.3

0.187

122.45

 

FBM2: Navigation

Sorted Based on Median

Team

Position

Orientation

collisions

Socrob

0.129

0.093

1

Homer

0.149

0.055

0

BARC

0.51

0.269

0

Hearts

0.712

1.57

1

 

FBM3: Speech Recognition

Sorted Based on Median

Team

FCR

AC

WER

SR

BARC

55.87

71.55

0.175

38.57

Socrob

44.27

77.86

0.13

51.43

Homer

43.17

81.88

0.08

67.63

Hearts

42.735

72.075

0.2

47.855