License: 
Creative Commons Attribution 3.0 Unported license (CC BY 3.0)
When quoting this document, please refer to the following
DOI: 10.4230/OASIcs.LDK.2019.10
URN: urn:nbn:de:0030-drops-103740
URL: http://dagstuhl.sunsite.rwth-aachen.de/volltexte/2019/10374/
Zayed, Omnia ;
McCrae, John P. ;
Buitelaar, Paul
Crowd-Sourcing A High-Quality Dataset for Metaphor Identification in Tweets
Abstract
Metaphor is one of the most important elements of human communication, especially in informal settings such as social media. There have been a number of datasets created for metaphor identification, however, this task has proven difficult due to the nebulous nature of metaphoricity. In this paper, we present a crowd-sourcing approach for the creation of a dataset for metaphor identification, that is able to rapidly achieve large coverage over the different usages of metaphor in a given corpus while maintaining high accuracy. We validate this methodology by creating a set of 2,500 manually annotated tweets in English, for which we achieve inter-annotator agreement scores over 0.8, which is higher than other reported results that did not limit the task. This methodology is based on the use of an existing classifier for metaphor in order to assist in the identification and the selection of the examples for annotation, in a way that reduces the cognitive load for annotators and enables quick and accurate annotation. We selected a corpus of both general language tweets and political tweets relating to Brexit and we compare the resulting corpus on these two domains. As a result of this work, we have published the first dataset of tweets annotated for metaphors, which we believe will be invaluable for the development, training and evaluation of approaches for metaphor identification in tweets.
BibTeX - Entry
@InProceedings{zayed_et_al:OASIcs:2019:10374,
author = {Omnia Zayed and John P. McCrae and Paul Buitelaar},
title = {{Crowd-Sourcing A High-Quality Dataset for Metaphor Identification in Tweets}},
booktitle = {2nd Conference on Language, Data and Knowledge (LDK 2019)},
pages = {10:1--10:17},
series = {OpenAccess Series in Informatics (OASIcs)},
ISBN = {978-3-95977-105-4},
ISSN = {2190-6807},
year = {2019},
volume = {70},
editor = {Maria Eskevich and Gerard de Melo and Christian F{\"a}th and John P. McCrae and Paul Buitelaar and Christian Chiarcos and Bettina Klimek and Milan Dojchinovski},
publisher = {Schloss Dagstuhl--Leibniz-Zentrum fuer Informatik},
address = {Dagstuhl, Germany},
URL = {http://drops.dagstuhl.de/opus/volltexte/2019/10374},
URN = {urn:nbn:de:0030-drops-103740},
doi = {10.4230/OASIcs.LDK.2019.10},
annote = {Keywords: metaphor, identification, tweets, dataset, annotation, crowd-sourcing}
}
|
Keywords: |
|
metaphor, identification, tweets, dataset, annotation, crowd-sourcing |
|
Collection: |
|
2nd Conference on Language, Data and Knowledge (LDK 2019) |
|
Issue Date: |
|
2019 |
|
Date of publication: |
|
16.05.2019 |
