Gender-fair post-editing of Machine Translation. Conducted by University of Graz and Vienna. In this challenge, you will find strategies for post-editing and improving biased MT outputs to achieve gender-fair translations between the languages English and German.
Manuel Lardelli (University of Graz), Dagmar Gromann (University of Vienna), Waltraud Kolb (University of Vienna) and Katharina Schuhmann (University of Vienna).
Machine translation (MT) systems have been identified as inherently gender-biased over the last years and several approaches to debias MT have been proposed, such as gender tagging and embedding debiasing. However, debiasing methods and discussions on bias in MT have predominantly focused on a binary conception of gender (male/female), disregarding queer and non-binary communities. Since language is an important vehicle for social reality, an inclusive society requires gender-fair language and MT should not facilitate inequality. In the first step, we propose to tackle this issue with a challenge on gender-fair post-editing of MT outputs, which in the future can provide inspiration for automated debiasing methods.
Post-editing has focused on reducing MT errors, stylistic improvements, and/or resolving terminological issues. The challenge is thus, to instead improve gender-biased MT outputs to achieve gender-fair translations between the languages English and German. For each source text, outputs of several commonly available, user-friendly MT systems will be provided for the convenience of the participating teams. However, if preferred, those outputs can be generated on the day of the challenge and implemented by the teams themselves to obtain the most recent MT translation of each system. The first step will consist of choosing one of the translation options considered. During the second step, participating teams will be asked to post-edit the provided translation with a particular focus on gender-fair language.
Different strategies to achieve gender-fair language for English and German will be introduced and exemplified. Participating teams are then asked to track, document and briefly explain their considerations for choosing a specific strategy. For instance, should it be “He is a chairman”, “She is a chairwoman”, “They are a chairperson” or “Ze is a chairperson” and why? These informally documented decisions will be submitted with the post-edited translations at the end of the challenge and compared across teams in an anonymized format, as part of a final challenge outcome shared with all participants.
A dataset of examples, as well as a handout with strategies to achieve gender-fair language in English and German will be a component of the challenge, including machine translated contents to be subjected to gender-fair post-editing and examples for gender-fair language.
Articles and websites:
Gender Bias in Machine Translation: https://arxiv.org/abs/2104.06001
On, ona, ono: Translating Gender Neutral Pronouns into Croatian: https://repozitorij.svkst.unist.hr/en/islandora/object/ffst%3A2887
Neural Machine Translation Doesn't Translate Gender Coreference Right Unless You Make It: https://arxiv.org/abs/2010.05332
Nibi-Space (German): https://nibi.space/geschlechtsneutrale_sprache
APA Style (English): https://apastyle.apa.org/style-grammar-guidelines/bias-free-language/gender
For this challenge, participants should have a good knowledge of English and ideally some knowledge of German. Additionally, participants should be interested in; if not passionate about, language, translation, and social equality.