Explainability Annotations for Legal Judgment Prediction in Switzerland

Introduction

We recently presented a dataset for Legal Judgment Prediction (you try to predict the outcome of a case based on its facts) including 85K Swiss Federal Supreme Court decisions [2]. Although we achieved up to 80% Macro-F1 Score, the models still work as black boxes and are thus not interpretable. In this project, you will work together with a legal student, to annotate some decisions in the dataset for explainability. You will use the annotation tool Prodigy and write annotation guidelines for collecting high-quality annotations.