# Working meeting "Uncertainty quantification and machine learning"

### March, 10th 2020

### Seminar organized by the GdR MASCOT-NUM

**Organizers:** Sébastien Da Veiga, Bertrand Iooss, Anthony Nouy, Guillaume Perrin, Victor Picheny

It will take place on:

March 10, 2020, at Amphithéâtre Hermite, Institut Henri Poincaré, Paris.

## Presentation of the workshop

Recent advances which emerged in parallel in the machine learning and UQ communities show that they could both benefit even further from joint research efforts and practice sharing. In particular, the trending issues in machine learning related to nonlinear approximation theory, uncertainty prediction and explainability of black-box models have deep links with some methodologies explored in the UQ community. The goal of this workshop is to investigate these links and gather a diverse audience from both research areas to facilitate future collaborations between them.

## Agenda

9h15 - Welcome - Introduction

### Theme 1: Nonlinear Approximation - Deep learning

Chair: Victor Picheny

9h30 - Anthony Nouy (Centrale Nantes): Learning with deep tensor networks - nouy20.pdf

10h15 - Mark van der Wilk (Imperial College London): Gaussian processes for uncertainty quantification in deep models - vanderwilk20.pdf

11h00 - Break

### Theme 2: Explainability and interpretability

Chair: Bertrand Iooss

11h30 - Max Halford (Université Toulouse): Global explanation of machine learning with sensitivity analysis - halford20.pdf

12h15 - Lunch break (on your own)

14h30 - Christophe Labreuche (Thalès): Interpretability methods in AI and a comparison with sensitivity analysis - labreuche20.pdf

### Theme 3: Prediction's uncertainty

Chair: Guillaume Perrin

15h15 - Nicolas Brosse (Thalès): Uncertainties for classification tasks in Deep Neural Networks: a last layer approach - brosse20.pdf

16h00 - Break

16h15 - Sébastien Da Veiga (Safran Tech): Sampling posteriors in high dimension: potential industrial applications with UQ - daveiga20.pdf

17h00 - End

## Abstracts

- Anthony Nouy (Ecole Centrale Nantes) : Learning with deep tensor networks

- Mark van der Wilk (Imperial College London) : Gaussian processes for uncertainty quantification in deep models

- Max Halford (Université Toulouse): Global explanation of machine learning with sensitivity analysis

- Christophe Labreuche (Thalès): Interpretability methods in AI and a comparison with sensitivity analysis

However, we can draw interesting connections between them.

Prior to that, we will start by describing some challenges in a large class of interpretability methods called Feature Attribution.

Feature Attribution aims at allocating the level of influence of each feature to the output of the AI model. The Shapley value of one of the leading concepts for feature attribution.

Its benefit is its axiomatic justification in Cooperative Game Theory.

It has been adapted to different fields of AI. In Machine Learning, the difficulty is to take into account the dependencies among featutes.

In Decision Aiding, the features are often organized in a hierarchical way and the standard Shapley value is not suitable.

We will show interesting connections between Feature Attribution methods and sensitivity analysis. Under some assumptions, the Sobol indices correspond to a concept which is a variant of the Shapley values.

- Nicolas Brosse (Thalès): Uncertainties for classification tasks in Deep Neural Networks: a last layer approach

We propose a family of algorithms which split the classification task into two stages: representation learning and uncertainty estimation. We compare four specific instances, where uncertainty estimation is performed via either an ensemble of Stochastic Gradient Descent or Stochastic Gradient Langevin Dynamics snapshots, an ensemble of bootstrapped logistic regressions, or via a number of Monte Carlo Dropout passes.

We evaluate their performance in terms of selective classification (risk-coverage), and their ability to detect out-of-distribution samples. Our experiments suggest there is limited value in adding multiple uncertainty layers to deep classifiers, and we observe that these simple methods outperform a vanilla point-estimate Stochastic Gradient Descent in some complex benchmarks like ImageNet.

- Sébastien Da Veiga (Safran Tech): Sampling posteriors in high dimension: potential industrial applications with UQ

In this talk we investigate their potential in the context of fully Bayesian Gaussian process regression.

We will focus on high-dimensional problems with sparsity-inducing priors (e.g. Acosso models).

In addition, we will show how the stochastic gradient point of view can handle a large number of design points, either by subsampling or with random Fourier features.