SemEval-2020 Task 3: Graded Word Similarity in Context

<p>For this tasks we ask participants to build systems that try to predict the effect that context has in human perception of similarity of words.</p> <p>We have seen very interesting work that uses local context to predict <strong>discrete</strong> changes in meaning: the different senses of a polysemous word. However context also has more subtle, <strong>continuous</strong> (<strong>graded</strong>) effects on meaning, even for words not necessarily considered polysemous.</p> <p>In order to be able to look at these effects we are building several datasets where we ask annotators to score how similar a pair of words are after they have read a short paragraph (which contains the two words). Each pair is scored within two of these paragraphs, allowing us to look at changes in similarity ratings due to context.</p> <p>CodaLab was used to run this task, you can see the dedicated website and the results of the participants at: https://competitions.codalab.org/competitions/20905</p>