On Parameter Selection for First-Order Methods: A Matrix Analysis Approach

10 Pages Posted: 16 Feb 2022

See all articles by Eder Baron-Prada

Eder Baron-Prada

affiliation not provided to SSRN

Salman Alsubaihi

affiliation not provided to SSRN

Khaled Alshehri

King Fahd University of Petroleum & Minerals (KFUPM)

Fahad Albalawi

Taif University

Abstract

First-order convex optimization algorithms are popular due to their computational attractiveness and applicability to a wide range of domains such as machine learning and control. Despite the substantial progress being made over the last few decades, some open questions related to their convergence speed remain unaddressed. In this manuscript, we answer one of them with two main contributions: First, we provide an extension of the current framework, where we analyze the speed of convergence of the algorithm using the contractive theory and linear algebra. Second, we find explicit values of the tuning parameters that are guarantee to be stable for gradient $L$- Lipchitz functions. Beyond optimization, we also discuss potential applications for Nash equilibrium computation in non-cooperative games. Finally, our proposed framework finds tighter bounds for already existing first-order methods with momentum.

Keywords: Convex OptimizationLinear SystemsGame TheoryOptimal Parameter selection

Suggested Citation

Baron-Prada, Eder and Alsubaihi, Salman and Alshehri, Khaled and Albalawi, Fahad, On Parameter Selection for First-Order Methods: A Matrix Analysis Approach. Available at SSRN: https://ssrn.com/abstract=4003402 or http://dx.doi.org/10.2139/ssrn.4003402

Eder Baron-Prada (Contact Author)

affiliation not provided to SSRN ( email )

No Address Available

Salman Alsubaihi

affiliation not provided to SSRN ( email )

No Address Available

Khaled Alshehri

King Fahd University of Petroleum & Minerals (KFUPM) ( email )

Dhahran, 31261
Saudi Arabia

Fahad Albalawi

Taif University ( email )

Airport Rd
Al Huwaya
Ta'if
Saudi Arabia

Do you have negative results from your research you’d like to share?

Paper statistics

Downloads
111
Abstract Views
303
Rank
444,918
PlumX Metrics