License: Creative Commons Attribution 3.0 Unported license (CC BY 3.0)
When quoting this document, please refer to the following
DOI: 10.4230/LIPIcs.ITCS.2017.15
URN: urn:nbn:de:0030-drops-81952
URL: http://dagstuhl.sunsite.rwth-aachen.de/volltexte/2017/8195/
Go to the corresponding LIPIcs Volume Portal

Lynch, Nancy ; Musco, Cameron ; Parter, Merav

Computational Tradeoffs in Biological Neural Networks: Self-Stabilizing Winner-Take-All Networks

pdf-format:
LIPIcs-ITCS-2017-15.pdf (1 MB)

Abstract

We initiate a line of investigation into biological neural networks from an algorithmic perspective. We develop a simplified but biologically plausible model for distributed computation in stochastic spiking neural networks and study tradeoffs between computation time and network complexity in this model. Our aim is to abstract real neural networks in a way that, while not capturing all interesting features, preserves high-level behavior and allows us to make biologically relevant conclusions.

In this paper, we focus on the important 'winner-take-all' (WTA) problem, which is analogous to a neural leader election unit: a network consisting of $n$ input neurons and n corresponding output neurons must converge to a state in which a single output corresponding to a firing input (the 'winner') fires, while all other outputs remain silent. Neural circuits for WTA rely on inhibitory neurons, which suppress the activity of competing outputs and drive the network towards a converged state with a single firing winner. We attempt to understand how the number of inhibitors used affects network convergence time.

We show that it is possible to significantly outperform naive WTA constructions through a more refined use of inhibition, solving the problem in O(\theta) rounds in expectation with just O(\log^{1/\theta} n) inhibitors for any \theta. An alternative construction gives convergence in O(\log^{1/\theta} n) rounds with O(\theta) inhibitors. We complement these upper bounds with our main technical contribution, a nearly matching lower bound for networks using \ge \log \log n inhibitors. Our lower bound uses familiar indistinguishability and locality arguments from distributed computing theory applied to the neural setting. It lets us derive a number of interesting conclusions about the structure of any network solving WTA with good probability, and the use of randomness and inhibition within such a network.

BibTeX - Entry

@InProceedings{lynch_et_al:LIPIcs:2017:8195,
  author =	{Nancy Lynch and Cameron Musco and Merav Parter},
  title =	{{Computational Tradeoffs in Biological Neural Networks: Self-Stabilizing Winner-Take-All Networks}},
  booktitle =	{8th Innovations in Theoretical Computer Science Conference (ITCS 2017)},
  pages =	{15:1--15:44},
  series =	{Leibniz International Proceedings in Informatics (LIPIcs)},
  ISBN =	{978-3-95977-029-3},
  ISSN =	{1868-8969},
  year =	{2017},
  volume =	{67},
  editor =	{Christos H. Papadimitriou},
  publisher =	{Schloss Dagstuhl--Leibniz-Zentrum fuer Informatik},
  address =	{Dagstuhl, Germany},
  URL =		{http://drops.dagstuhl.de/opus/volltexte/2017/8195},
  URN =		{urn:nbn:de:0030-drops-81952},
  doi =		{10.4230/LIPIcs.ITCS.2017.15},
  annote =	{Keywords: biological distributed algorithms, neural networks, distributed lower bounds, winner-take-all networks}
}

Keywords: biological distributed algorithms, neural networks, distributed lower bounds, winner-take-all networks
Collection: 8th Innovations in Theoretical Computer Science Conference (ITCS 2017)
Issue Date: 2017
Date of publication: 28.11.2017

DROPS-Home | Fulltext Search | Imprint | Privacy Published by LZI