License: 
Creative Commons Attribution 3.0 Unported license (CC BY 3.0)
When quoting this document, please refer to the following
DOI: 10.4230/LIPIcs.DISC.2017.33
URN: urn:nbn:de:0030-drops-79863
URL: http://dagstuhl.sunsite.rwth-aachen.de/volltexte/2017/7986/
Lynch, Nancy ;
Musco, Cameron ;
Parter, Merav
Neuro-RAM Unit with Applications to Similarity Testing and Compression in Spiking Neural Networks
Abstract
We study distributed algorithms implemented in a simplified biologically inspired model for stochastic spiking neural networks. We focus on tradeoffs between computation time and network complexity, along with the role of noise and randomness in efficient neural computation.
It is widely accepted that neural spike responses, and neural computation in general, is inherently stochastic. In recent work, we explored how this stochasticity could be leveraged to solve the 'winner-take-all' leader election task. Here, we focus on using randomness in neural algorithms for similarity testing and compression. In the most basic setting, given two n-length patterns of firing neurons, we wish to distinguish if the patterns are equal or epsilon-far from equal.
Randomization allows us to solve this task with a very compact network, using O((sqrt(n) log n)/epsilon) auxiliary neurons, which is sublinear in the input size. At the heart of our solution is the design of a t-round neural random access memory, or indexing network, which we call a neuro-RAM. This module can be implemented with O(n/t) auxiliary neurons and is useful in many applications beyond similarity testing - e.g., we discuss its application to compression via random projection.
Using a VC dimension-based argument, we show that the tradeoff between runtime and network size in our neuro-RAM is nearly optimal. To the best of our knowledge, we are the first to apply these techniques to stochastic spiking networks. Our result has several implications - since our neuro-RAM can be implemented with deterministic threshold gates, it demonstrates that, in contrast to similarity testing, randomness does not provide significant computational advantages for this problem. It also establishes a separation between our networks, which spike with a sigmoidal probability function, and well-studied deterministic sigmoidal networks, whose gates output real number values, and which can implement a neuro-RAM much more efficiently.
BibTeX - Entry
@InProceedings{lynch_et_al:LIPIcs:2017:7986,
author = {Nancy Lynch and Cameron Musco and Merav Parter},
title = {{Neuro-RAM Unit with Applications to Similarity Testing and Compression in Spiking Neural Networks}},
booktitle = {31st International Symposium on Distributed Computing (DISC 2017)},
pages = {33:1--33:16},
series = {Leibniz International Proceedings in Informatics (LIPIcs)},
ISBN = {978-3-95977-053-8},
ISSN = {1868-8969},
year = {2017},
volume = {91},
editor = {Andr{\'e}a W. Richa},
publisher = {Schloss Dagstuhl--Leibniz-Zentrum fuer Informatik},
address = {Dagstuhl, Germany},
URL = {http://drops.dagstuhl.de/opus/volltexte/2017/7986},
URN = {urn:nbn:de:0030-drops-79863},
doi = {10.4230/LIPIcs.DISC.2017.33},
annote = {Keywords: spiking neural networks, biological distributed algorithms, circuit design}
}
|
Keywords: |
|
spiking neural networks, biological distributed algorithms, circuit design |
|
Collection: |
|
31st International Symposium on Distributed Computing (DISC 2017) |
|
Issue Date: |
|
2017 |
|
Date of publication: |
|
12.10.2017 |
