License: Creative Commons Attribution 3.0 Unported license (CC BY 3.0)
When quoting this document, please refer to the following
DOI: 10.4230/LIPIcs.CSL.2020.18
URN: urn:nbn:de:0030-drops-116611
URL: http://dagstuhl.sunsite.rwth-aachen.de/volltexte/2020/11661/
Go to the corresponding LIPIcs Volume Portal

Cockett, Robin ; Cruttwell, Geoffrey ; Gallagher, Jonathan ; Lemay, Jean-Simon Pacaud ; MacAdam, Benjamin ; Plotkin, Gordon ; Pronk, Dorette

Reverse Derivative Categories

pdf-format:
LIPIcs-CSL-2020-18.pdf (0.5 MB)

Abstract

The reverse derivative is a fundamental operation in machine learning and automatic differentiation [Martín Abadi et al., 2015; Griewank, 2012]. This paper gives a direct axiomatization of a category with a reverse derivative operation, in a similar style to that given by [Blute et al., 2009] for a forward derivative. Intriguingly, a category with a reverse derivative also has a forward derivative, but the converse is not true. In fact, we show explicitly what a forward derivative is missing: a reverse derivative is equivalent to a forward derivative with a dagger structure on its subcategory of linear maps. Furthermore, we show that these linear maps form an additively enriched category with dagger biproducts.

BibTeX - Entry

@InProceedings{cockett_et_al:LIPIcs:2020:11661,
  author =	{Robin Cockett and Geoffrey Cruttwell and Jonathan Gallagher and Jean-Simon Pacaud Lemay and Benjamin MacAdam and Gordon Plotkin and Dorette Pronk},
  title =	{{Reverse Derivative Categories}},
  booktitle =	{28th EACSL Annual Conference on Computer Science Logic (CSL 2020)},
  pages =	{18:1--18:16},
  series =	{Leibniz International Proceedings in Informatics (LIPIcs)},
  ISBN =	{978-3-95977-132-0},
  ISSN =	{1868-8969},
  year =	{2020},
  volume =	{152},
  editor =	{Maribel Fern{\'a}ndez and Anca Muscholl},
  publisher =	{Schloss Dagstuhl--Leibniz-Zentrum fuer Informatik},
  address =	{Dagstuhl, Germany},
  URL =		{https://drops.dagstuhl.de/opus/volltexte/2020/11661},
  URN =		{urn:nbn:de:0030-drops-116611},
  doi =		{10.4230/LIPIcs.CSL.2020.18},
  annote =	{Keywords: Reverse Derivatives, Cartesian Reverse Differential Categories, Categorical Semantics, Cartesian Differential Categories, Dagger Categories, Automati}
}

Keywords: Reverse Derivatives, Cartesian Reverse Differential Categories, Categorical Semantics, Cartesian Differential Categories, Dagger Categories, Automati
Collection: 28th EACSL Annual Conference on Computer Science Logic (CSL 2020)
Issue Date: 2020
Date of publication: 06.01.2020

DROPS-Home | Fulltext Search | Imprint | Privacy Published by LZI