Statistics research seminar

Modal title

Modal content

Date / Time

Speaker

Title

Location

9 September 2022
14:15-15:15

Bin Yu
UC Berkeley

Event Details

Title	Complexity, simplicity, and decision trees
Speaker, Affiliation	Bin Yu, UC Berkeley
Date, Time	9 September 2022, 14:15-15:15
Location	HG G 19.1
Abstract	Occam's razor is a general principle for science to pursue the simplest explanation or model when the empirical support evidence is the same for the explanations or models under consideration. To quantify simplicy, a complexity measure is necessary and many such measures have been used in the literature including uniform stability. Both complexity and stability are central to interpretable machine learning. In this talk, we first give an overview of interpretable machine learning and then delve into our recent work on decisions trees, which are especially useful interpretable methods in high-stake applications such as medicine and public policy. In particular, we show that decision trees are sub-optimal for additive regression models. To improve upon decision trees, we introduce a new method called Fast Interpretable Greedy-Tree Sums (FIGS) that fits additive trees while controlling the total number of splits. The state-of-the-art performance of FIGS will be illustrated through case studies for clinical decision rules.

Complexity, simplicity, and decision treesread_more

30 September 2022
15:15-16:15

Alexander Henzi
ETH, Seminar for Statistics

Event Details

Title	Isotonic distributional regression
Speaker, Affiliation	Alexander Henzi, ETH, Seminar for Statistics
Date, Time	30 September 2022, 15:15-16:15
Location	HG G 19.1
Abstract	Statistical predictions should provide a quantification of forecast uncertainty. Ideally, this uncertainty quantification is in the form of a probability distribution for the outcome of interest conditional on the available information. Isotonic distributional regression (IDR) is a nonparametric method that allows to derive probabilistic forecasts from a training data set of point predictions and observations, solely under the assumption of stochastic monotonicity. IDR does not require parameter tuning, and it has interesting properties when analyzed under the paradigm of maximizing sharpness subject to calibration. The method can serve as a natural benchmark for postprocessing forecasts both from statistical models and external sources, which is illustrated through applications in weather forecasting and medicine.

Isotonic distributional regressionread_more

21 October 2022
15:15-16:15

Mona Azadkia
ETH Zürich

Event Details

Title	Linear regression with unmatched data: a deconvolution perspective
Speaker, Affiliation	Mona Azadkia, ETH Zürich
Date, Time	21 October 2022, 15:15-16:15
Location	HG G 19.1
Abstract	Consider the regression problem where the response Y∈ ℝ and the covariate X ∈ ℝ^d for d≥1 are \textit{unmatched}. Under this scenario, we do not have access to pairs of observations from the distribution of (X,Y), but instead, we have separate datasets {Yi}_ni=1 and {Xj}_mj=1, possibly collected from different sources. We study this problem assuming that the regression function is linear and the noise distribution is known or can be estimated. We introduce an estimator of the regression vector based on deconvolution and demonstrate its consistency and asymptotic normality under an identifiability assumption. In the general case, we show that our estimator (DLSE: Deconvolution Least Squared Estimator) is consistent in terms of an extended ℓ2 norm. Using this observation, we devise a method for semi-supervised learning, i.e., when we have access to a small sample of matched pairs (Xk,Yk). Several applications with synthetic and real datasets are considered to illustrate the theory.

Linear regression with unmatched data: a deconvolution perspectiveread_more

1 December 2022
16:00-17:00

Pfister Niklas
University of Copenhagen

Event Details

Title	Distribution Generalization and Identifiability in IV Models
Speaker, Affiliation	Pfister Niklas, University of Copenhagen
Date, Time	1 December 2022, 16:00-17:00
Location	HG G 19.1
Abstract	Causal models can provide good predictions even under distributional shifts. This observation has led to the development of various methods that use causal learning to improve the generalization performance of predictive models. In this talk, we consider this type of approach for instrumental variable (IV) models. IV allows us to identify a causal function between covariates X and a response Y, even in the presence of unobserved confounding. In many practical prediction settings the causal function is however not fully identifiable. We consider two approaches for dealing with this under-identified setting: (1) By adding a sparsity constraint and (2) by introducing the invariant most predictive (IMP) model, which deals with the under-identifiability by selecting the most predictive model among all feasible IV solutions. Furthermore, we analyze to which types of distributional shifts these models generalize.

Distribution Generalization and Identifiability in IV Modelsread_more

16 December 2022
15:15-16:15

Weijie Su
Wharton, University of Pennsylvania

Event Details

Title	Some Geometric Patterns of Real-World Deep Neural Networks
Speaker, Affiliation	Weijie Su, Wharton, University of Pennsylvania
Date, Time	16 December 2022, 15:15-16:15
Location	HG G 19.1
Abstract	In this talk, we will investigate the emergence of geometric patterns in well-trained deep learning models by making use of the layer-peeled model and the law of equi-separation. The former is a nonconvex optimization program that models the last-layer features and weights. We use the model to shed light on the neural collapse phenomenon of Papyan, Han, and Donoho, and to predict a hitherto-unknown phenomenon that we term minority collapse in imbalanced training. This is based on joint work with Cong Fang, Hangfeng He, and Qi Long (arXiv:2101.12699). In the second part, we study how real-world deep neural networks process data in the interior layers. Our finding is a simple and quantitative law that governs how deep neural networks separate data according to class membership throughout all layers for classification. This law shows that each layer improves data separation at a constant geometric rate, and its emergence is observed in an authoritative collection of network architectures and datasets during training. This law offers practical guidelines for designing architectures, improving model robustness and out-of-sample performance, as well as interpreting the predictions. This is based on joint work with Hangfeng He (arXiv:2210.17020).

Some Geometric Patterns of Real-World Deep Neural Networksread_more

Note: if you want you can subscribe to the iCal/ics Calender.

Archive: SS 24 AS 23 SS 23 AS 22 SS 22 AS 21 SS 20 AS 19 SS 19 AS 18 SS 18 AS 17 SS 17 AS 16 SS 16 AS 15 SS 15 AS 14 SS 14 AS 13 SS 13 AS 12 SS 12 AS 11 SS 11 AS 10 SS 10 AS 09