License: 
Creative Commons Attribution 3.0 Unported license (CC BY 3.0)
When quoting this document, please refer to the following
DOI: 10.4230/LIPIcs.ITCS.2017.30
URN: urn:nbn:de:0030-drops-81722
URL: http://dagstuhl.sunsite.rwth-aachen.de/volltexte/2017/8172/
Servedio, Rocco A. ;
Tan, Li-Yang
What Circuit Classes Can Be Learned with Non-Trivial Savings?
Abstract
Despite decades of intensive research, efficient - or even sub-exponential time - distribution-free PAC learning algorithms are not known for many important Boolean function classes. In this work we suggest a new perspective on these learning problems, inspired by a surge of recent research in complexity theory, in which the goal is to determine whether and how much of a savings over a naive 2^n runtime can be achieved.
We establish a range of exploratory results towards this end. In more detail,
(1) We first observe that a simple approach building on known uniform-distribution learning results gives non-trivial distribution-free learning algorithms for several well-studied classes including AC0, arbitrary functions of a few linear threshold functions (LTFs), and AC0 augmented with mod_p gates.
(2) Next we present an approach, based on the method of random restrictions from circuit complexity, which can be used to obtain several distribution-free learning algorithms that do not appear to be achievable by approach (1) above. The results achieved in this way include learning algorithms with non-trivial savings for LTF-of-AC0 circuits and improved savings for learning parity-of-AC0 circuits.
(3) Finally, our third contribution is a generic technique for converting lower bounds proved using Neciporuk's method to learning algorithms with non-trivial savings. This technique, which is the most involved of our three approaches, yields distribution-free learning algorithms for a range of classes where previously even non-trivial uniform-distribution learning algorithms were not known; these classes include full-basis formulas, branching programs, span programs, etc. up to some fixed polynomial size.
BibTeX - Entry
@InProceedings{servedio_et_al:LIPIcs:2017:8172,
author = {Rocco A. Servedio and Li-Yang Tan},
title = {{What Circuit Classes Can Be Learned with Non-Trivial Savingsl}},
booktitle = {8th Innovations in Theoretical Computer Science Conference (ITCS 2017)},
pages = {30:1--30:21},
series = {Leibniz International Proceedings in Informatics (LIPIcs)},
ISBN = {978-3-95977-029-3},
ISSN = {1868-8969},
year = {2017},
volume = {67},
editor = {Christos H. Papadimitriou},
publisher = {Schloss Dagstuhl--Leibniz-Zentrum fuer Informatik},
address = {Dagstuhl, Germany},
URL = {http://drops.dagstuhl.de/opus/volltexte/2017/8172},
URN = {urn:nbn:de:0030-drops-81722},
doi = {10.4230/LIPIcs.ITCS.2017.30},
annote = {Keywords: computational learning theory, circuit complexity, non-trivial savings}
}
|
Keywords: |
|
computational learning theory, circuit complexity, non-trivial savings |
|
Collection: |
|
8th Innovations in Theoretical Computer Science Conference (ITCS 2017) |
|
Issue Date: |
|
2017 |
|
Date of publication: |
|
28.11.2017 |
