Seminar for Statistics

Search

ETH-FDS seminar series

More information about ETH Foundations of Data Science can be found here

×

Modal title

Modal content

Please subscribe here if you would you like to be notified about these presentations via e-mail. Moreover you can subscribe to the iCal/ics Calender.

Spring Semester 2023

Date / Time Speaker Title Location
2 March 2023
15:00-16:00 		Felix Krahmer
TU München
Event Details

ETH-FDS seminar

Title Robust low-rank matrix completion with adversarial noise
Speaker, Affiliation Felix Krahmer, TU München
Date, Time 2 March 2023, 15:00-16:00
Location HG E 1.1
Abstract The problem of recovering a high-dimensional low-rank matrix from a limited set of random measurements has enjoyed various applications and gained a detailed theoretical foundation over the last 15 years. An instance of particular interest is the matrix completion problem where the measurements are entry observations. The first rirgorous recovery guarantees for this problem were derived for the nuclear norm minimization approach, a convex proxy for the NP-hard problem of constrained rank minimization. For matrices whose entries are ”spread out” well enough, this convex problem admits a unique solution which corresponds to the ground truth. In the presence of random measurement noise, the reconstruction performance is also well-studied, but the performance for adversarial noise remains less understood. While some error bounds have been derived for both convex and nonconvex approaches, these bounds exhibit a gap to information-theoretic lower bounds and provable performance for Gaussian measurements. However, a recent analysis of the problem suggests that under small-scale adversarsial noise, the reconstruction error can be significantly amplified. In this talk, we investigate this amplification quantitatively and provide new reconstruction bounds for both small and large noise levels that suggest a quadratic dependence between the reconstruction error and the noise level. This is joint work with Julia Kostin (TUM/ETH) and Dominik Stöger (KU Eichstätt-Ingolstadt).
Robust low-rank matrix completion with adversarial noiseread_more
HG E 1.1
18 April 2023
14:15-15:05 		Courtney Paquette
McGill University, Canada
Event Details

ETH-FDS seminar

Title DACO-​FDS: Stochastic Algorithms in the Large: Batch Size Saturation, Stepsize Criticality, Generalization Performance, and Exact Dynamics (Part I)
Speaker, Affiliation Courtney Paquette, McGill University, Canada
Date, Time 18 April 2023, 14:15-15:05
Location HG G 19.1
Abstract Random matrices frequently appear in many different fields — physics, computer science, applied and pure mathematics. Oftentimes the random matrix of interest will have non-​trivial structure — entries that are dependent and have potentially different means and variances (e.g. sparse Wigner matrices, matrices corresponding to adjacencies of random graphs, sample covariance matrices). However, current understanding of such complex random matrices remains lacking. In this talk, I will discuss recent results concerning the spectrum of sums of independent random matrices with a.s. bounded operator norms. In particular, I will demonstrate that under some fairly general conditions, such sums will exhibit the following universality phenomenon — their spectrum will lie close to that of a Gaussian random matrix with the same mean and covariance. No prior background in random matrix theory is required — basic knowledge of probability and linear algebra are sufficient. (joint with Ramon van Handel) Pre-​print link: https://web.math.princeton.edu/~rvan/tuniv220113.pdf
DACO-​FDS: Stochastic Algorithms in the Large: Batch Size Saturation, Stepsize Criticality, Generalization Performance, and Exact Dynamics (Part I)read_more
HG G 19.1
18 April 2023
15:10-16:00 		Elliot Paquette
McGill University, Canada
Event Details

ETH-FDS seminar

Title DACO-​FDS: Stochastic Algorithms in the Large: Batch Size Saturation, Stepsize Criticality, Generalization Performance, and Exact Dynamics (Part II)
Speaker, Affiliation Elliot Paquette, McGill University, Canada
Date, Time 18 April 2023, 15:10-16:00
Location HG G 19.1
Abstract In this talk, we will present a framework for analyzing dynamics of stochastic optimization algorithms (e.g., stochastic gradient descent (SGD) and momentum (SGD+M)) when both the number of samples and dimensions are large. For the analysis, we will introduce a stochastic differential equation, called homogenized SGD. We show that homogenized SGD is the high-​dimensional equivalent of SGD -- for any quadratic statistic (e.g., population risk with quadratic loss), the statistic under the iterates of SGD converges to the statistic under homogenized SGD when the number of samples n and number of features d are polynomially related. By analyzing homogenized SGD, we provide exact non-​asymptotic high-​dimensional expressions for the training dynamics and generalization performance of SGD in terms of a solution of a Volterra integral equation. The analysis is formulated for data matrices and target vectors that satisfy a family of resolvent conditions, which can roughly be viewed as a weak form of delocalization of sample-​side singular vectors of the data. By analyzing these limiting dynamics, we can provide insights into learning rate, momentum parameter, and batch size selection. For instance, we identify a stability measurement, the implicit conditioning ratio (ICR), which regulates the ability of SGD+M to accelerate the algorithm. When the batch size exceeds this ICR, SGD+M converges linearly at a rate of $O(1/ \kappa)$, matching optimal full-​batch momentum (in particular performing as well as a full-​batch but with a fraction of the size). For batch sizes smaller than the ICR, in contrast, SGD+M has rates that scale like a multiple of the single batch SGD rate. We give explicit choices for the learning rate and momentum parameter in terms of the Hessian spectra that achieve this performance. Finally we show this model matches performances on real data sets.
DACO-​FDS: Stochastic Algorithms in the Large: Batch Size Saturation, Stepsize Criticality, Generalization Performance, and Exact Dynamics (Part II)read_more
HG G 19.1
11 May 2023
16:15-17:15 		Stephan Mandt
University of California
Event Details

ETH-FDS seminar

Title Deep Latent Variable Models for Compression and Natural Science
Speaker, Affiliation Stephan Mandt , University of California
Date, Time 11 May 2023, 16:15-17:15
Location HG D 1.2
Abstract Latent variable models have been an integral part of probabilistic machine learning, ranging from simple mixture models to variational autoencoders to powerful diffusion probabilistic models at the center of recent media attention. Perhaps less well-appreciated is the intimate connection between latent variable models and data compression, and the potential of these models for advancing natural science. This talk will explore these topics. I will begin by showcasing connections between variational methods and the theory and practice of neural data compression. On the applied side, variational methods lead to machine-learned compressors of data such as images and videos and offer principled techniques for enhancing their compression performance, as well as reducing their decoding complexity. On the theory side, variational methods also provide scalable bounds on the fundamental compressibility of real-world data, such as images and particle physics data. Lastly, I will also delve into climate science projects, where a combination of deep latent variable modeling and vector quantization enables assessing distribution shifts induced by varying climate models and the effects of global warming. Short Bio: Stephan Mandt is an Associate Professor of Computer Science and Statistics at the University of California, Irvine. From 2016 until 2018, he was a Senior Researcher and Head of the statistical machine learning group at Disney Research in Pittsburgh and Los Angeles. He held previous postdoctoral positions at Columbia University and Princeton University. Stephan holds a Ph.D. in Theoretical Physics from the University of Cologne in Germany, where he received the National Merit Scholarship. He received the NSF CAREER Award, a Kavli Fellowship of the U.S. National Academy of Sciences, the German Research Foundation's Mercator Fellowship, and the UCI ICS Mid-Career Excellence in Research Award. He is a member of the ELLIS Society and a former visiting researcher at Google Brain. Stephan will serve as Program Chair of the AISTATS 2024 conference, currently serves as an Action Editor for JMLR and TMLR, and frequently serves as Area Chair for NeurIPS, ICML, AAAI, and ICLR.
Deep Latent Variable Models for Compression and Natural Scienceread_more
HG D 1.2
1 June 2023
16:15-17:15 		Yurii Nesterov
UCLouvain
Event Details

ETH-FDS seminar

Title New perspectives for higher-order methods in Convex Optimization
Speaker, Affiliation Yurii Nesterov, UCLouvain
Date, Time 1 June 2023, 16:15-17:15
Location HG D 1.2
Abstract In the recent years, the most important developments in Optimization were related to clarification of abilities of the higher-order methods. These schemes have potentially much higher rate of convergence as compared to the lower-order methods. However, the possibility of their implementation in the form of practically efficient algorithms was questionable during decades. In this talk, we discuss different possibilities for advancing in this direction, which avoid all standard fears on tensor methods (memory requirements, complexity of computing the tensor components, etc.). Moreover, in this way we get the new second-order methods with memory, which converge provably faster than the conventional upper limits provided by the Complexity Theory.
New perspectives for higher-order methods in Convex Optimizationread_more
HG D 1.2

Archive: SS 24 AS 23 SS 23 AS 22 SS 22 AS 21 SS 21 AS 20 SS 20 AS 19 SS 19

JavaScript has been disabled in your browser