License: 
Creative Commons Attribution 4.0 International license (CC BY 4.0)
When quoting this document, please refer to the following
DOI: 10.4230/DagSemProc.04121.5
URN: urn:nbn:de:0030-drops-4609
URL: http://dagstuhl.sunsite.rwth-aachen.de/volltexte/2006/460/
|
Go to the corresponding Portal |
Gerhard, Michael
Evaluating Embodied Conversational Agents in Collaborative Virtual Environments
Abstract
There are currently no evaluation methods
specific to ECAs in CVEs and traditional
evaluation methods are limited in their
applicability and consequently unlikely to
address the full range of aspects now inherent in
such systems. We argue that a combination of
controlled experimentation, quasi-experiments,
review-based evaluation and heuristic expert
reviews is needed. To operationalise these
traditional evaluation methods the concept of
presence was deployed, and it was argued that
presence as a cognitive variable can be measured
and that such a measure can be a key indicator of
the usability of ECAs in CVEs. Presence measures
can be administered within controlled experiments
and quasi-experiments to test certain aspects of
the system. Such experiments might turn out
particularly useful as a means of selecting
between two or more design options and it is
argued that issues concerning ECAs in CVEs can be
meaningfully evaluated by comparing subjects’
experience of presence. Further, although
implementation issues were not the primary
concern of this study, the strength and
shortcomings of the prototype agent were
evaluated as secondary variables within that
experiment. A set of criteria developed for the
evaluation of the strengths and shortcomings of
the current prototype agent are partly based on
Nielsen’s general usability guidelines and partly
on a set of heuristics proposed for non-embodied conversational systems.
BibTeX - Entry
@InProceedings{gerhard:DagSemProc.04121.5,
author = {Gerhard, Michael},
title = {{Evaluating Embodied Conversational Agents in Collaborative Virtual Environments}},
booktitle = {Evaluating Embodied Conversational Agents},
series = {Dagstuhl Seminar Proceedings (DagSemProc)},
ISSN = {1862-4405},
year = {2006},
volume = {4121},
editor = {Zsofia Ruttkay and Elisabeth Andr\'{e} and W. Lewis Johnson and Catherine Pelachaud},
publisher = {Schloss Dagstuhl -- Leibniz-Zentrum f{\"u}r Informatik},
address = {Dagstuhl, Germany},
URL = {https://drops.dagstuhl.de/opus/volltexte/2006/460},
URN = {urn:nbn:de:0030-drops-4609},
doi = {10.4230/DagSemProc.04121.5},
annote = {Keywords: Embodied Conversational Agents,Collaborative Virtual Environments, Presence}
}
|
Keywords: |
|
Embodied Conversational Agents,Collaborative Virtual Environments, Presence |
|
Collection: |
|
04121 - Evaluating Embodied Conversational Agents |
|
Issue Date: |
|
2006 |
|
Date of publication: |
|
14.03.2006 |
