Advanced lab 2: Building a labelled parser

Objective

In this lab, you will extend the parser from the basic lab to support labelled parsing. This means that your parser should not only predict that there is a syntactic relation between two words but the type of this relation – for example, subject or object. You will validate your implementation by comparing the performance of your parser to those reported by Glavaš and Vulić (2021).

Instructions

1. Understand the task

   • Read the page on universal dependencies to get a rough understanding about syntactic relations.
   • Read Section 3 of Glavaš and Vulić (2021) to see how they compute relation scores.
   • You also need to understand how to compute the loss for the relation prediction task.

2. Modify your parser to support labelled parsing

   • Extend or adapt your implementation of the bi-affine layer to support the computation of relation scores.
   • Extend or adapt your implementation of the loss function.
   • Make only the minimal necessary modifications to your existing parser.

3. Validate your implementation

   • Attempt to replicate the results reported for BERT on the EWT, Table 1 in Glavaš and Vulić (2021).
   • You only need to replicate the results for the standard setup, not for the adapter setup.

4. Add your work to your portfolio

   • Include a short report summarising the changes you made and the results of your replication attempt.
   • Add your notebook and the report to your lab portfolio and present it at the oral exam.

Hints & considerations

• Computing the arc scores can be seen as a special case of computing the relation scores.
• The overall loss of the parser is the sum of the arc loss and the relation loss.

Deliverables

• parser.ipynb – a notebook containing your parser implementation
• report.md – short report summarising your work

Good luck! 🚀