Seminar overview

This talk discusses a general framework for smoothing parameter estimation for models with regular likelihoods constructed in terms of unknown smooth functions of covariates. Gaussian random effects and parametric terms may also be present. The approach allows smoothing parameter uncertainty to be quantified, suggesting a fix for a well known problem with AIC for such models. The smooth functions are represented by reduced rank spline like smoothers, with associated quadratic penalties measuring function smoothness. Model estimation is by penalized likelihood maximization, where the smoothing parameters controlling the extent of penalization are estimated by Laplace approximate marginal likelihood. The methods cover, for example, generalized additive models for non-exponential family responses (for example beta, ordered categorical, scaled t distribution, negative binomial and Tweedie distributions), generalized additive models for location scale and shape (for example two stage zero inflation models, and Gaussian location-scale models), Cox proportional hazards models and multivariate additive models. The framework reduces the implementation of new model classes to the coding of some standard derivatives of the log likelihood.

Abstract: In various applications, one is given the advice or predictions of several classifiers of unknown reliability, over multiple questions or queries. This scenario is different from the standard supervised setting where classifier accuracy can be assessed from available labeled training or validation data, and raises several questions: given only the predictions of several classifiers of unknown accuracies, over a large set of unlabeled test data, is it possible to a) reliably rank them, and b) construct a meta-classifier more accurate than any individual classifier in the ensemble? In this talk we'll show that under various independence assumptions between classifier errors, this high dimensional data hides simple low dimensional structures. In particular, we'll present simple spectral methods to address the above questions, and derive new unsupervised spectral meta-learners. We'll prove these methods are asymptotically consistent when the model assumptions hold, and also present their empirical success on a variety of unsupervised learning problems.

I will discuss trend filtering, a newly proposed tool of Steidl et al. (2006), Kim et al. (2009) for nonparametric regression. The trend filtering estimate is defined as the minimizer of a penalized least squares criterion, in which the penalty term sums the absolute kth order discrete derivatives over the input points. I will give an overview of some interesting connections between these estimates and adaptive spline estimation, in particular, a connection to locally adaptive regression splines of Mammen and van de Geer (1997). If time permits, I will discuss some extensions of trend filtering, namely, to high-dimensional data and (separately) to graph-based data. I will also discuss some of the challenges I see in each of these settings. This represents joint work with Veeranjaneyulu Sadhanala, Yu-Xiang Wang, James Sharpnack, and Alex Smola.

A common approach to statistical learning on big data is to randomly split it among m machines and calculate the parameter of interest by averaging their m individual estimates. Focusing on empirical risk minimization, or equivalently M-estimation, we study the statistical error incurred by this strategy. We consider two asymptotic settings: one where the number of samples per machine n->inf but the number of parameters p is fixed, and a second high-dimensional regime where both p,n-> inf with p/n-> kappa. Most previous works provided only moment bounds on the error incurred by splitting the data in the fixed p setting. In contrast, we present for both regimes asymptotically exact distributions for this estimation error. In the fixed-p setting, under suitable assumptions, we thus prove that to leading order, averaging is as accurate as the centralized solution. In the high-dimensional setting, we show a qualitatively different behavior: data splitting does incur a first order accuracy loss, which we quantify precisely. In addition, our asymptotic distributions allow the construction of confidence intervals and hypothesis testing on the estimated parameters. Our main conclusion is that in both regimes, averaging parallelized estimates is an attractive way to speedup computations and save on memory, while incurring a quantifiable and typically moderate excess error.

In den vergangenen 20 Jahren seit der Deregulierung des Schweizer Motorfahrzeug-Versicherungsmarkt sind die Tarifstrukturen immer komplexer geworden. Es werden unterschiedliche Kriterien in verschiedenen Ausprägungen und Stärke bei dem einzelnen Versicherer verwendet. Die statistischen Methoden auf der anderen Seite sind weitgehend dieselben, die Verallgemeinerten Linearen Modelle haben sich im Versicherungsmarkt durchgesetzt. Mit Telematik-Daten kommt eine neue Herausforderung auf die Versicherungsindustrie zu. Einerseits bringt der Ansatz eine individuelle Beurteilung des Fahrverhaltens in Kombination mit der Solidarität innerhalb einer Risikogruppe mit sich. Andererseits sind das Datenvolumen und damit auch die verwendeten statistischen Methoden grundlegend anders als bei klassischer Risikobeurteilung. In der Präsentation werden anhand der Telematik Daten der AXA Winterthur die Herausforderungen bei der Aufbereitung und Plausibilisierung der Rohdaten und deren Darstellung gezeigt, sowie die Entwicklung von Algorithmen zur Zusammenführung der Daten mit Karteninformationen diskutiert. Anhand deskriptiver Analysen werden Muster, Gemeinsamkeiten oder Unterschiede gesucht, welche zur Bildung von Hypothesen dienen. In ersten Schadenmodellen werden diese untersucht.

Abstract: In this work in progress we treat a functional mixed effects model in the setting of spectral analysis of subject-replicated time series data. We assume that the time series subjects share a common population spectral curve (functional fixed effect), additional to some random subject-specific deviation around this curve (functional random effects), which models the variability within the population. In contrast to existing work we allow this variability to be non-diagonal, i.e. there may exist explicit corre- lation between the different subjects in the population. To estimate the common population curve we project the subject-curves onto an appropriate orthonormal basis (such as a wavelet basis) and continue working in the coefficient domain instead of the functional domain. In a sampled data model, with discretely observed noisy subject-curves, the model in the co- efficient domain reduces to a finite-dimensional linear mixed model. This allows us, for estimation and prediction of the fixed and random effect coefficients, to apply both traditional linear mixed model meth- ods and, if necessary by the spatially variable nature of the spectral curves, work with some appropriate non-linear thresholding approach. We derive some theoretical properties of our methodology highlighting the influence of the correlation in the subject population. To illustrate the proposed functional mixed model, we show some examples using simulated time series data, and an analysis of empirical subject-replicated EEG data. We conclude with some possible extensions, among which we allow situations where the data show po- tential breakpoints in its second order (spectral) structure over time. The presented work is joint with Joris Chau (ISBA, UCL).

More information: https://www.math.ethz.ch/sfs/news-and-events/research-seminar.html?s=fs16call_made

Real world networks often exhibit a community structure, in the sense that the vertices of the network are partitioned into groups such that the concentration of linkages is high among vertices in the same group and low otherwise. This moti- vates us to introduce a class of Gaussian graphical models whose network structure is determined by a stochastic block model. The stochastic block model is a random graph in which vertices are partitioned into communities and the existence of a link between two vertices is determined by a Bernoulli trial with a probability that de- pends on the communities the vertices belong to. A natural question that arises in this framework is how to detect communities from a random sample of observations. We introduce a community-detection algorithm called Blockbuster, which consists of applying spectral clustering to the sample covariance matrix, that is, it applies k-means clustering to the eigenvectors corresponding to its largest eigenvalues. We study the properties of the procedure and show that Blockbuster consistently de- tects communities when the network dimension and the sample size are large. The methodology is used to study real activity clustering in the United States and Eu- rope. Keywords: Partial Correlation Networks, Random Graphs, Community Detection, Spec- tral Clustering, Graphical Models JEL: C3, C33, C55

Bayesian hierarchical models are formulated to handle complex data dependencies but also to allow the user to include prior or expert knowledge at different model stages. However, the challenge in understanding and controlling all model components, as well as the lack of easy-to-use software may hinder practitioners to use Bayesian hierarchical models. In this talk we present two case studies where we try to address the aforementioned obstacles. The first case study concerns Bayesian disease mapping. We propose to re-think the common Besag-York-Mollie model to facilitate prior definitions and to obtain a better model understanding. The second case study concerns bivariate meta-analysis of diagnostic test studies, where the inclusion of prior knowledge has proven to be beneficial as data are often sparse. To intuitively incorporate available prior knowledge in the last hierarchy level, we show how to use penalised complexity priors. The methodology is implemented in the R-package meta4diag which provides an interactive graphical user interface offering full functionality without requiring any R programming.

We propose to use the Realized Beta GARCH model of Hansen et al. (2014) to exploit the potential of using high-frequency data in commodity markets for the modeling of correlations. The model produces accurate forecasts of pairwise correlations between commodities which can be used to construct a composite covariance matrix. We eval- uate the attractiveness of this matrix in a portfolio context and compare it to models more commonly used in the industry. We demonstrate significant economic gains in a realistic setting including short selling constraints and transaction costs. Keywords: Commodities, financialization, futures market. JEL Classification: C53, C58, G12, G13, G15, G17, G32.

Sports forecasting models – beyond their interest to bettors – are important resources for sports analysts and coaches. Like the best athletes, the best forecasting models should be rigorously tested and judged by how well their performance holds up against top competitors. Although a number of models have been proposed for predicting match outcomes in professional tennis, their comparative performance is largely unknown. In this talk I will present results comparing the performance of 11 published forecasting models for predicting the outcomes of 2395 singles matches during the 2014 season of the Association of Tennis Professionals Tour. I’ll discuss the implications of these findings for current application of forecasting in tennis and how the advent of computer vision data will shape how prediction is used in future applications.

More information: https://www.math.ethz.ch/sfs/news-and-events/seminar-applied-statistics.htmlcall_made

We introduce a class of random graphs on the reals R defined by the exchangeability of their vertices. A straightforward adaptation of a result by Kallenberg yields a representation theorem: every such random graph is characterized by three (potentially random) components: a nonnegative real I, an integrable function S : R+ to R+, and a symmetric measurable function W: R+^2 to [0,1] that satisfies several weak integrability conditions. We call the triple (I,S,W) a graphex, in analogy to graphons, which characterize the (dense) exchangeable graphs on N. I will present some results about the structure and consistent estimation of these random graphs. Joint work with Victor Veitch