License: Creative Commons Attribution 3.0 Unported license (CC BY 3.0)
When quoting this document, please refer to the following
DOI: 10.4230/OASIcs.SLATE.2016.4
URN: urn:nbn:de:0030-drops-60090
URL: http://dagstuhl.sunsite.rwth-aachen.de/volltexte/2016/6009/
Go to the corresponding OASIcs Volume Portal

Costa, Teresa ; Leal, José Paulo

Comparing and Benchmarking Semantic Measures Using SMComp

pdf-format:
OASIcs-SLATE-2016-4.pdf (0.5 MB)

Abstract

The goal of the semantic measures is to compare pairs of concepts, words, sentences or named entities. Their categorization depends on what they measure. If a measure only considers taxonomy relationships is a similarity measure; if it considers all type of relationships it is a relatedness measure.

The evaluation process of these measures usually relies on semantic gold standards. These datasets, with several pairs of words with a rating assigned by persons, are used to assess how well a semantic measure performs.

There are a few frameworks that provide tools to compute and analyze several well-known measures. This paper presents a novel tool - SMComp - a testbed designed for path-based semantic measures. At its current state, it is a domain-specific tool using three different versions of WordNet.

SMComp has two views: one to compute semantic measures of a pair of words and another to assess a semantic measure using a dataset. On the first view, it offers several measures described in the literature as well as the possibility of creating a new measure, by introducing Java code snippets on the GUI. The other view offers a large set of semantic benchmarks to use in the assessment process. It also offers the possibility of uploading a custom dataset to be used in the assessment.

BibTeX - Entry

@InProceedings{costa_et_al:OASIcs:2016:6009,
  author =	{Teresa Costa and Jos{\'e} Paulo Leal},
  title =	{{Comparing and Benchmarking Semantic Measures Using SMComp}},
  booktitle =	{5th Symposium on Languages, Applications and Technologies (SLATE'16)},
  pages =	{4:1--4:13},
  series =	{OpenAccess Series in Informatics (OASIcs)},
  ISBN =	{978-3-95977-006-4},
  ISSN =	{2190-6807},
  year =	{2016},
  volume =	{51},
  editor =	{Marjan Mernik and Jos{\'e} Paulo Leal and Hugo Gon{\c{c}}alo Oliveira},
  publisher =	{Schloss Dagstuhl--Leibniz-Zentrum fuer Informatik},
  address =	{Dagstuhl, Germany},
  URL =		{http://drops.dagstuhl.de/opus/volltexte/2016/6009},
  URN =		{urn:nbn:de:0030-drops-60090},
  doi =		{10.4230/OASIcs.SLATE.2016.4},
  annote =	{Keywords: Semantic similarity, semantic relatedness, testbed, web application}
}

Keywords: Semantic similarity, semantic relatedness, testbed, web application
Collection: 5th Symposium on Languages, Applications and Technologies (SLATE'16)
Issue Date: 2016
Date of publication: 21.06.2016

DROPS-Home | Fulltext Search | Imprint | Privacy Published by LZI