License: Creative Commons Attribution 3.0 Unported license (CC BY 3.0)
When quoting this document, please refer to the following
DOI: 10.4230/LIPIcs.APPROX-RANDOM.2017.27
URN: urn:nbn:de:0030-drops-75761
URL: http://dagstuhl.sunsite.rwth-aachen.de/volltexte/2017/7576/
Go to the corresponding LIPIcs Volume Portal

Avron, Haim ; Clarkson, Kenneth L. ; Woodruff, David P.

Sharper Bounds for Regularized Data Fitting

pdf-format:
LIPIcs-APPROX-RANDOM-2017-27.pdf (0.6 MB)

Abstract

We study matrix sketching methods for regularized variants of linear regression, low rank approximation, and canonical correlation analysis. Our main focus is on sketching techniques which preserve the objective function value for regularized problems, which is an area that has remained largely unexplored. We study regularization both in a fairly broad setting, and in the specific context of the popular and widely used technique of ridge regularization; for the latter, as applied to each of these problems, we show algorithmic resource bounds in which the statistical dimension appears in places where in previous bounds the rank would appear. The statistical dimension is always smaller than the rank, and decreases as the amount of regularization increases. In particular we show this for the ridge low-rank approximation problem as well as regularized low-rank approximation problems in a much more general setting, where the regularizing function satisfies some very general conditions (chiefly, invariance under orthogonal transformations).

BibTeX - Entry

@InProceedings{avron_et_al:LIPIcs:2017:7576,
  author =	{Haim Avron and Kenneth L. Clarkson and David P. Woodruff},
  title =	{{Sharper Bounds for Regularized Data Fitting}},
  booktitle =	{Approximation, Randomization, and Combinatorial Optimization. Algorithms and Techniques (APPROX/RANDOM 2017)},
  pages =	{27:1--27:22},
  series =	{Leibniz International Proceedings in Informatics (LIPIcs)},
  ISBN =	{978-3-95977-044-6},
  ISSN =	{1868-8969},
  year =	{2017},
  volume =	{81},
  editor =	{Klaus Jansen and Jos{\'e} D. P. Rolim and David Williamson and Santosh S. Vempala},
  publisher =	{Schloss Dagstuhl--Leibniz-Zentrum fuer Informatik},
  address =	{Dagstuhl, Germany},
  URL =		{http://drops.dagstuhl.de/opus/volltexte/2017/7576},
  URN =		{urn:nbn:de:0030-drops-75761},
  doi =		{10.4230/LIPIcs.APPROX-RANDOM.2017.27},
  annote =	{Keywords: Matrices, Regression, Low-rank approximation, Regularization, Canonical Correlation Analysis}
}

Keywords: Matrices, Regression, Low-rank approximation, Regularization, Canonical Correlation Analysis
Collection: Approximation, Randomization, and Combinatorial Optimization. Algorithms and Techniques (APPROX/RANDOM 2017)
Issue Date: 2017
Date of publication: 11.08.2017

DROPS-Home | Fulltext Search | Imprint | Privacy Published by LZI