Global Average Precision

The metric for Kaggle Competition Google Landmark Recognition 2020

Global Average Precision Score

N predictions (label/confidence pairs) sorted in descending order by their confidence scores, then the Global Average Precision is computed as:

\[GAP = \frac{1}{M}\sum_{i=1}^{N}P(i)rel(i)\]

  • N is the total number of predictions returned by the system, across all queries

  • M is the total number of queries with at least one sample from the training set visible in it (note that some queries may not depict samples)

  • P(i) is the precision at rank i. (example: consider rank 3 - we have already made 3 predictions, and 2 of them are correct. Then P(3) will be 2/3)

  • rel(i) denotes the relevance of prediciton i: it’s 1 if the i-th prediction is correct, and 0 otherwise

evaluations.kaggle_2020.global_average_precision.global_average_precision_score(y_true: Dict[Any, Any], y_pred: Dict[Any, Tuple[Any, float]]) → float

Compute Global Average Precision score (GAP)

Parameters

  • y_true (Dict[Any, Any]) – Dictionary with query ids and true ids for query samples

  • y_pred (Dict[Any, Tuple[Any, float]]) – Dictionary with query ids and predictions (predicted id, confidence level)

Returns

GAP score

Return type

float

Examples

>>> from evaluations.kaggle_2020 import global_average_precision_score
>>> y_true = {
...         'id_001': 123,
...         'id_002': None,
...         'id_003': 999,
...         'id_004': 123,
...         'id_005': 999,
...         'id_006': 888,
...         'id_007': 666,
...         'id_008': 666,
...         'id_009': None,
...         'id_010': 666,
...     }
>>> y_pred = {
...         'id_001': (123, 0.15),
...         'id_002': (123, 0.10),
...         'id_003': (999, 0.30),
...         'id_005': (999, 0.40),
...         'id_007': (555, 0.60),
...         'id_008': (666, 0.70),
...         'id_010': (666, 0.99),
...     }
>>> global_average_precision_score(y_true, y_pred)
0.5479166666666666