Statistics research seminar

×

Modal title

Modal content

Autumn Semester 2012

Date / Time Speaker Title Location
* 2 October 2012
15:15-16:15
Shaowei Lin
University of California Berkeley
Event Details

Research Seminar in Statistics

Title Understanding the curse of singularities in machine learning
Speaker, Affiliation Shaowei Lin, University of California Berkeley
Date, Time 2 October 2012, 15:15-16:15
Location HG G 19.1
Abstract Many parameter estimation and integral approximation problems in machine learning suffer, not from the curse of dimensionality as commonly believed, but from the curse of singularities. A common way of overcoming such problems is regularization using sparse penalties. Recent developments in the learning theory of singular models might be the key to understanding this phenomenon. In this talk, we give a brief introduction to Sumio Watanabe's Singular Learning Theory, as outlined in his book "Algebraic Geometry and Statistical Learning Theory". We will learn how geometry and resolution of singularities help us approximate integrals efficiently.
Understanding the curse of singularities in machine learningread_more
HG G 19.1
19 October 2012
15:15-16:30
Gabor Lugosi
Universitat Pompeu Fabra, Barcelona
Event Details

Research Seminar in Statistics

Title Detection of correlations in high dimension
Speaker, Affiliation Gabor Lugosi, Universitat Pompeu Fabra, Barcelona
Date, Time 19 October 2012, 15:15-16:30
Location HG G 19.1
Abstract We consider the problem of finding information in high-dimensional noisy data. Our goal is to understand the possibilities and limitations of such correlation detection problems. The mathematical analysis reveals some interesting phase transitions. We also discuss an interesting connection with random geometric graphs. (The talk is mostly based on joint work with Ery Arias-Castro and Sebasiten Bubeck.)
Detection of correlations in high dimensionread_more
HG G 19.1
9 November 2012
15:15-16:30
Garvesh Raskutti
UC California, Berkeley, USA
Event Details

Research Seminar in Statistics

Title Early stopping of gradient descent for non-parametric regression: An optimal data-dependent stopping rule
Speaker, Affiliation Garvesh Raskutti, UC California, Berkeley, USA
Date, Time 9 November 2012, 15:15-16:30
Location HG G 19.1
Abstract The phenomenon of overfitting is ubiquitous throughout statistics, and is particularly problematic in non-parametric problems. As a result, regularization or shrinkage estimators are used. The early stopping strategy is to run an iterative algorithm for a fixed but finite number of iterations. Early stopping of iterative algorithms is known to achieve regularization since it implicitly shrinks the solution of the un-regularized objective towards the starting point of the algorithm. When using early stopping as a strategy for regularization,a critical issue is determining when to stop. In this talk, I present analysis for an iterative update corresponding to gradient descent applied to the non-parametric least-squares loss in an appropriately chosen co-ordinate system. In particular, for our iterative update, I present a computable data-dependent stopping rule developed by me and my former advisors. Our stopping rule achieves minimax optimal rates in mean-squared error for Sobolev space or finite-rank Reproducing kernel Hilbert space (RKHS). Importantly, our stopping rule does not require data-intensive methods such as cross-validation or hold-out data and has optimal mean-squared error performance. This work is joint with my former advisors, Martin Wainwright and Bin Yu.
Early stopping of gradient descent for non-parametric regression: An optimal data-dependent stopping ruleread_more
HG G 19.1
16 November 2012
15:15-16:30
Ya'acov Ritov
Hebrew University, Jerusalem
Event Details

Research Seminar in Statistics

Title Some empirical Bayes results.
Speaker, Affiliation Ya'acov Ritov, Hebrew University, Jerusalem
Date, Time 16 November 2012, 15:15-16:30
Location HG G 19.1
Abstract In this talk we consider some empirical Bayes results. Mainly we consider a new approach to the very classical Poisson model, to the use of proxies (AKA covariates).
Some empirical Bayes results.read_more
HG G 19.1
30 November 2012
15:15-16:15
Sebastian Reich
Universität Potsdam
Event Details

Research Seminar in Statistics

Title Bayesian inference and sequential filtering: An optimal coupling of measures perspective
Speaker, Affiliation Sebastian Reich, Universität Potsdam
Date, Time 30 November 2012, 15:15-16:15
Location HG G 19.1
Abstract Sequential filtering relies on the propagation of uncertainty under a given model dynamics within a Monte Carlo (MC) setting combined with an assimilation of observations using Bayes' theorem. The recursive application of Bayes' theorem within a dynamic MC framework poses major computational challenges. The popular class of sequential Monte Carlo methods (SMCMs) relies on a proposal step and an importance resampling step. However, SMCMs are subject to the curse of dimensionality and alternative methods are needed for filtering in high-dimension. The ensemble Kalman filter (EnKF) has emerged as a promising alternative to SMCMs but is also known to lead to asymptotically inconsistent results. Following an introduction to sequential filtering, I will discuss a McKean approach to Bayesian inference and its implementation using optimal couplings. Applying this approach to the sequential filtering leads to new perspectives on both SMCMs and EnKFs as well as to novel filter algorithms.
Bayesian inference and sequential filtering: An optimal coupling of measures perspectiveread_more
HG G 19.1
* 30 November 2012
16:15-17:00
Aad van der Vaart
University of Leiden
Event Details

Research Seminar in Statistics

Title Nonparametric Credible Sets
Speaker, Affiliation Aad van der Vaart, University of Leiden
Date, Time 30 November 2012, 16:15-17:00
Location HG G 19.1
Abstract Bayesian nonparametric procedures for function estimation (densities, regression functions, drift functions, etc.) have been shown to perform well, if some care is taken in the choice of the prior. Many nonparametric priors do not "wash out" as the number of data points increases, unlike for finite-dimensional parameters, but by introducing hyperparameters they can give reconstructions that adapts to the properties of large classes of true underlying functions, similar to the best non-Bayesian procedures for function estimation. Besides a reconstruction a posterior distribution also gives a sense of remaining uncertainty about the true parameter, through its spread. In practice "credible sets, which are central sets of prescribed posterior probability, are often treated as if they are confidence sets. We present some results that show that this practice can be justified, but also results that show that it can be extremely misleading. The situation is particularly delicate if the prior is adapted through hyperparameters (by either empirical of hierarchical Bayes). General, non-Bayesian, difficulties with nonparametric confidence sets play an important role in the resulting difficulties. Although the message of the results is thought to be general, our talk will be limited to the special case of prior distributions furnished by Gaussian processes.
Nonparametric Credible Setsread_more
HG G 19.1
7 December 2012
15:15-16:30
Michael Wolf
Universität Zürich
Event Details

Research Seminar in Statistics

Title Nonlinear Shrinkage Estimation of Large-Dimensional Covariance Matrices
Speaker, Affiliation Michael Wolf, Universität Zürich
Date, Time 7 December 2012, 15:15-16:30
Location HG G 19.1
Abstract Many statistical applications require an estimate of a covariance matrix and/or its inverse. When the matrix dimension is large compared to the sample size, which happens frequently, the sample covariance matrix is known to perform poorly and may suffer from ill-conditioning. There already exists an extensive literature concerning improved estimators in such situations. In the absence of further knowledge about the structure of the true covariance matrix, the most successful approach so far, arguably, has been shrinkage estimation. Shrinking the sample covariance matrix to a multiple of the identity, by taking a weighted average of the two, turns out to be equivalent to linearly shrinking the sample eigenvalues to their grand mean, while retaining the sample eigenvectors. Our paper extends this approach by considering nonlinear transformations of the sample eigenvalues. We show how to construct an estimator that is asymptotically equivalent to an oracle estimator suggested in previous work. As demonstrated in extensive Monte Carlo simulations, the resulting bona fide estimator can result in sizeable improvements over the sample covariance matrix and also over linear shrinkage.
Nonlinear Shrinkage Estimation of Large-Dimensional Covariance Matricesread_more
HG G 19.1
* 10 December 2012
15:00-16:00
Andreas Buja
The Wharton School, University of Pennsylvania
Event Details

Research Seminar in Statistics

Title Valid Post-Selection Inference
Speaker, Affiliation Andreas Buja, The Wharton School, University of Pennsylvania
Date, Time 10 December 2012, 15:00-16:00
Location HG G 19.1
Abstract It is common practice in statistical data analysis to perform data-driven variable selection and derive statistical inference from the resulting model. Such inference enjoys none of the guarantees that classical statistical theory provides for tests and confidence intervals when the model has been chosen a priori. We propose to produce valid "post-selection inference" by reducing the problem to one of simultaneous inference and hence suitably widening conventional confidence and retention intervals. Simultaneity is required for all linear functions that arise as coefficient estimates in all submodels. By purchasing "simultaneity insurance" for all possible submodels, the resulting post-selection inference is rendered universally valid under all possible model selection procedures. This inference is therefore generally conservative for particular selection procedures, but it is always less conservative than full Scheffe protection. Importantly it does NOT depend on the truth of the selected submodel, and hence it produces valid inference even in wrong models. We describe the structure of the simultaneous inference problem and give some asymptotic results. JOINT WITH: Richard Berk, Larry Brown, Kai Zhang, Linda Zhao
Valid Post-Selection Inferenceread_more
HG G 19.1
* 14 December 2012
14:15-15:30
Mohamed Hebiri
Université Paris-Est Marne-la-Vallée
Event Details

Research Seminar in Statistics

Title Learning heteroscedastic models via SOCP under group sparsity
Speaker, Affiliation Mohamed Hebiri, Université Paris-Est Marne-la-Vallée
Date, Time 14 December 2012, 14:15-15:30
Location HG G 19.1
Abstract Sparse estimation methods based on `1 relaxation, such as the Lasso and the Dantzig selector, are powerful tools for estimating high dimensional linear models. However, in order to properly tune these methods, the variance of the noise is often required. This constitutes a major obstacle for practical applications of these methods in various frameworks – such as time series, random fields, inverse problems – for which noise is rarely homoscedastic or with a level that is hard to know in advance. In this paper, we propose a new approach to the joint estimation of the conditional mean and the conditional variance in a high-dimensional (auto-)regression setting. An attractive feature of our proposed estimator is that it is computable by solving a second-order cone program (SOCP). We present numerical results assessing the performance of the proposed procedure both on simulations and on real data. We also establish non-asymptotic risk bounds which are nearly as strong as those for original `1-penalized estimators. This work is joint with Arnak Dalalyan, Katia Meziani and Joseph Salmon
Learning heteroscedastic models via SOCP under group sparsityread_more
HG G 19.1

Notes: events marked with an asterisk (*) indicate that the time and/or location are different from the usual time and/or location and if you want you can subscribe to the iCal/ics Calender.

Organizers: Peter Bühlmann, Leonhard Held, Hans Rudolf Künsch, Marloes Maathuis, Sara van de Geer, Michael Wolf

JavaScript has been disabled in your browser