Row-Wise Micro Averaged F1¶

The metric for Kaggle Competition Cornell Birdcall Identification

For each row we compute number of True Positives (TP), False Positives (FP) and False Negatives (FN) predictions.

For example, if true label is “bird1 bird2”, and we predict “bird2 bird3 bird4”, then we have:

TP = 1 (bird2)
FP = 2 (bird3, bird4)
FN = 1 (bird1)

Then we calculate F1 score by the next formula:

\[F1\ Score = \frac{2 * TP}{2*TP + FN + FP}\]

At the end we compute mean by all row scores.

Row-Wise Micro Averaged F1 Score¶

evaluations.kaggle_2020.row_wise_micro_averaged_f1.row_wise_micro_averaged_f1_score(y_true: List[str], y_pred: List[str]) → float¶

Compute row-wise micro averaged f1 score

Parameters

y_true (List[str]) – Target list of strings of a space separated birds names
y_pred (List[str]) – Predicted list of strings of a space separated birds names

Returns

Row-wise micro averaged F1 score

Return type

float

Examples

>>> from evaluations.kaggle_2020 import row_wise_micro_averaged_f1_score
>>> y_true = [
...         'amecro',
...         'amecro amerob',
...         'nocall',
...     ]
>>> y_pred = [
...         'amecro',
...         'amecro bird666',
...         'nocall',
...     ]
>>> row_wise_micro_averaged_f1_score(y_true, y_pred)
0.8333333333333333

Micro F1 Similarity¶

evaluations.kaggle_2020.row_wise_micro_averaged_f1.micro_f1_similarity(y_true: str, y_pred: str) → float¶

Compute micro f1 similarity for 1 row

Parameters

y_true (str) – True string of a space separated birds names
y_pred (str) – Predicted string of a space separated birds names

Returns

Micro F1 similarity

Return type

float

Examples

>>> from evaluations.kaggle_2020 import micro_f1_similarity
>>> y_true = 'amecro amerob'
>>> y_pred = 'amecro bird666'
>>> micro_f1_similarity(y_true, y_pred)
0.5