License: Creative Commons Attribution 3.0 Unported license (CC BY 3.0)
When quoting this document, please refer to the following
DOI: 10.4230/DagRep.3.10.92
URN: urn:nbn:de:0030-drops-44335
URL: http://dagstuhl.sunsite.rwth-aachen.de/volltexte/2014/4433/
Go back to Dagstuhl Reports

Agosti, Maristella ; Fuhr, Norbert ; Toms, Elaine ; Vakkari, Pertti
Weitere Beteiligte (Hrsg. etc.): Maristella Agosti and Norbert Fuhr and Elaine Toms and Pertti Vakkari

Evaluation Methodologies in Information Retrieval (Dagstuhl Seminar 13441)

pdf-format:
dagrep_v003_i010_p092_s13441.pdf (0.8 MB)

Abstract

This report documents the program and the outcome of Dagstuhl Seminar 13441 "Evaluation Methodologies in Information Retrieval", which brought together 42 participants from 11 countries. The seminar was motivated by the fact that today's information retrieval (IR) applications can hardly be evaluated based on the classic test collection paradigm, thus there is a need for new evaluation approaches. The event started with five introductory talks on evaluation frameworks, user modeling for evaluation, evaluation criteria, measures, evaluation methodology, and new trends in IR evaluation. The seminar participants then formed working groups addressing specific aspects of IR evaluation, such as reliability and validity, task-based IR, learning as search outcome, searching for fun, IR and social media, graph search, domain-specific IR, interaction measures and models, and searcher-aware information access systems.

BibTeX - Entry

@Article{agosti_et_al:DR:2014:4433,
  author =	{Maristella Agosti and Norbert Fuhr and Elaine Toms and Pertti Vakkari},
  title =	{{Evaluation Methodologies in Information Retrieval (Dagstuhl Seminar 13441)}},
  pages =	{92--126},
  journal =	{Dagstuhl Reports},
  ISSN =	{2192-5283},
  year =	{2014},
  volume =	{3},
  number =	{10},
  editor =	{Maristella Agosti and Norbert Fuhr and Elaine Toms and Pertti Vakkari},
  publisher =	{Schloss Dagstuhl--Leibniz-Zentrum fuer Informatik},
  address =	{Dagstuhl, Germany},
  URL =		{http://drops.dagstuhl.de/opus/volltexte/2014/4433},
  URN =		{urn:nbn:de:0030-drops-44335},
  doi =		{10.4230/DagRep.3.10.92},
  annote =	{Keywords: evaluation, testbeds, user studies, measures}
}

Keywords: evaluation, testbeds, user studies, measures
Collection: Dagstuhl Reports, Volume 3, Issue 10
Issue Date: 2014
Date of publication: 14.02.2014

DROPS-Home | Fulltext Search | Imprint | Privacy Published by LZI