License: Creative Commons Attribution 3.0 Unported license (CC BY 3.0)
When quoting this document, please refer to the following
DOI: 10.4230/OASIcs.SLATE.2016.4
URN: urn:nbn:de:0030-drops-60090
URL: http://dagstuhl.sunsite.rwth-aachen.de/volltexte/2016/6009/
Costa, Teresa ;
Leal, José Paulo
Comparing and Benchmarking Semantic Measures Using SMComp
Abstract
The goal of the semantic measures is to compare pairs of concepts, words, sentences or named entities. Their categorization depends on what they measure. If a measure only considers taxonomy relationships is a similarity measure; if it considers all type of relationships it is a relatedness measure.
The evaluation process of these measures usually relies on semantic gold standards. These datasets, with several pairs of words with a rating assigned by persons, are used to assess how well a semantic measure performs.
There are a few frameworks that provide tools to compute and analyze several well-known measures. This paper presents a novel tool - SMComp - a testbed designed for path-based semantic measures. At its current state, it is a domain-specific tool using three different versions of WordNet.
SMComp has two views: one to compute semantic measures of a pair of words and another to assess a semantic measure using a dataset. On the first view, it offers several measures described in the literature as well as the possibility of creating a new measure, by introducing Java code snippets on the GUI. The other view offers a large set of semantic benchmarks to use in the assessment process. It also offers the possibility of uploading a custom dataset to be used in the assessment.
BibTeX - Entry
@InProceedings{costa_et_al:OASIcs:2016:6009,
author = {Teresa Costa and Jos{\'e} Paulo Leal},
title = {{Comparing and Benchmarking Semantic Measures Using SMComp}},
booktitle = {5th Symposium on Languages, Applications and Technologies (SLATE'16)},
pages = {4:1--4:13},
series = {OpenAccess Series in Informatics (OASIcs)},
ISBN = {978-3-95977-006-4},
ISSN = {2190-6807},
year = {2016},
volume = {51},
editor = {Marjan Mernik and Jos{\'e} Paulo Leal and Hugo Gon{\c{c}}alo Oliveira},
publisher = {Schloss Dagstuhl--Leibniz-Zentrum fuer Informatik},
address = {Dagstuhl, Germany},
URL = {http://drops.dagstuhl.de/opus/volltexte/2016/6009},
URN = {urn:nbn:de:0030-drops-60090},
doi = {10.4230/OASIcs.SLATE.2016.4},
annote = {Keywords: Semantic similarity, semantic relatedness, testbed, web application}
}
Keywords: |
|
Semantic similarity, semantic relatedness, testbed, web application |
Collection: |
|
5th Symposium on Languages, Applications and Technologies (SLATE'16) |
Issue Date: |
|
2016 |
Date of publication: |
|
21.06.2016 |