License: Creative Commons Attribution 3.0 Unported license (CC BY 3.0)
When quoting this document, please refer to the following
DOI: 10.4230/OASIcs.ICCSW.2013.89
URN: urn:nbn:de:0030-drops-42765
URL: http://dagstuhl.sunsite.rwth-aachen.de/volltexte/2013/4276/
Go to the corresponding OASIcs Volume Portal

de Penning, Leo ; D'Avila Garcez, Artur ; Meyer, John-Jules C.

Dreaming Machines: On multimodal fusion and information retrieval using neural-symbolic cognitive agents

pdf-format:
15.pdf (0.7 MB)

Abstract

Deep Boltzmann Machines (DBM) have been used as a computational cognitive model in various AI-related research and applications, notably in computational vision and multimodal fusion. Being regarded as a biological plausible model of the human brain, the DBM is also becoming a popular instrument to investigate various cortical processes in neuroscience. In this paper, we describe how a multimodal DBM is implemented as part of a Neural-Symbolic Cognitive Agent (NSCA) for real-time multimodal fusion and inference of streaming audio and video data. We describe how this agent can be used to simulate certain neurological mechanisms related to hallucinations and dreaming and how these mechanisms are beneficial to the integrity of the DBM. Finally, we will explain how the NSCA is used to extract multimodal information from the DBM and provide a compact and practical iconographic temporal logic formula for complex relations between visual and auditory patterns.

BibTeX - Entry

@InProceedings{depenning_et_al:OASIcs:2013:4276,
  author =	{Leo de Penning and Artur D'Avila Garcez and John-Jules C. Meyer},
  title =	{{Dreaming Machines: On multimodal fusion and information retrieval using neural-symbolic cognitive agents}},
  booktitle =	{2013 Imperial College Computing Student Workshop},
  pages =	{89--94},
  series =	{OpenAccess Series in Informatics (OASIcs)},
  ISBN =	{978-3-939897-63-7},
  ISSN =	{2190-6807},
  year =	{2013},
  volume =	{35},
  editor =	{Andrew V. Jones and Nicholas Ng},
  publisher =	{Schloss Dagstuhl--Leibniz-Zentrum fuer Informatik},
  address =	{Dagstuhl, Germany},
  URL =		{http://drops.dagstuhl.de/opus/volltexte/2013/4276},
  URN =		{urn:nbn:de:0030-drops-42765},
  doi =		{10.4230/OASIcs.ICCSW.2013.89},
  annote =	{Keywords: Multimodal fusion, Deep Boltzmann Machine, Neural-Symbolic Cognitive Agent, Dreaming, Hallucinations}
}

Keywords: Multimodal fusion, Deep Boltzmann Machine, Neural-Symbolic Cognitive Agent, Dreaming, Hallucinations
Collection: 2013 Imperial College Computing Student Workshop
Issue Date: 2013
Date of publication: 14.10.2013

DROPS-Home | Fulltext Search | Imprint | Privacy Published by LZI