Iteratively Re-weighted Schemes for Non-smooth Optimization

Abstract

Iteratively Re-weighted Least Squares (IRLS) has long been used to solve both convex optimization problems, including l_1 regression and compressed sensing, as well as non-convex optimization problems, including l_p regression for (0 < p < 1). The thesis is organized as follows. Following the introduction in Chapter 1, in Chapter 2 we give a robust phase retrieval counterpart to the seminal paper by Candes and Tao on compressed sensing (l_1 regression) [Decoding by linear programming. IEEE Transactions on Information Theory, 51(12):42034215, 2005]. Chapter 3 answers a question raised in [Iteratively re-weighted least squares minimization for sparse recovery, Communications on Pure and Applied Mathematics, 63(2010) 1–38]. In particular, we find examples where IRLS algorithm in the paper provably fails and provide a remedy. In Chapter 4 we show that under the assumption that the entries of A are i.i.d. standard normal, locally linear convergence rate is achieved for IRLS algorithm, when applied to robust phase retrieval problem min_x |||Ax| − b||_1. Furthermore, we provide several other IRLS variants which can be applied to phase retrieval problems. Both the noiseless case and the case with sparse noise are considered. In Chapter 5 we talk about the application of IRLS in general cases.