Proper Scoring Rules | Bohan Chen's Personal Webpage

Invited Talk at SIAM MDS26: Learning Filtering Distributions Using Strictly Proper Scoring Rules

Tue, 14 Jul 2026 00:00:00 +0000

Table of Contents

I am pleased to share that I will give an invited talk at the in Salt Lake City this November.

My talk, “Learning Filtering Distributions Using Strictly Proper Scoring Rules,” will be part of the minisymposium

Structure-Preserving Data Assimilation and Learning for Complex Dynamical Systems — Part I of III

organized by Tongtong Li (University of Maryland, Baltimore County) and Lizuo Liu (Dartmouth College). I am grateful to the organizers for the invitation and look forward to the discussion.

Talk at a Glance

Talk: Learning Filtering Distributions Using Strictly Proper Scoring Rules
Minisymposium: Structure-Preserving Data Assimilation and Learning for Complex Dynamical Systems — Part I of III
Conference: SIAM Conference on Mathematics of Data Science (MDS26)
Conference dates: November 16–20, 2026
Venue: Salt Palace Convention Center, Salt Lake City, Utah, U.S.
Session date, time, and room: To be announced in the official program

The detailed session assignment has not yet appeared on the . I will update this post when SIAM publishes the schedule.

About the Talk

Bayesian filtering seeks the evolving conditional distribution of a hidden dynamical state given partial and noisy observations. This distribution is the natural object for uncertainty quantification, especially when nonlinear dynamics or observations lead to non-Gaussian and multimodal posteriors. Yet the true filtering distribution is generally intractable and therefore unavailable as a supervised learning target.

In this talk, I will present the proper scoring ensemble filter (PSEF), which learns an ensemble analysis map using simulated state–observation trajectories. The central idea is to train with a strictly proper scoring rule—the energy score in our implementation—so that the expected objective is uniquely optimized by the full filtering distribution rather than only its conditional mean.

Our analysis map is a permutation-invariant transformer that takes a forecast ensemble and the latest observation as input and returns an analysis ensemble. Under a realizability assumption, we show that the population objective recovers the Bayesian filtering distribution. Numerical experiments demonstrate accurate filtering for nonlinear and non-Gaussian systems, including multimodal posteriors that are missed by classical Gaussian filters and learning methods based on mean-squared error.

This is joint work with Eviatar Bach, Ricardo Baptista, Jochen Bröcker, and Andrew Stuart.

Paper:
Code:
Publication page and BibTeX:
Research announcement:

Minisymposium Theme

The three-part minisymposium brings together researchers working on structure-preserving data assimilation, inverse problems, uncertainty quantification, feature-aware learning, and scientific machine learning. Its focus is on methods that incorporate knowledge of the underlying dynamics—such as physical constraints, coherent structures, and informative representations—to improve the stability, robustness, interpretability, and predictive performance of learning and inference in nonlinear and multiscale systems.

I look forward to presenting this work and to seeing many of you in Salt Lake City!

Learning Probabilistic Filters with Strictly Proper Scoring Rules

Thu, 25 Jun 2026 00:00:00 +0000

The proper scoring ensemble filter (PSEF) learns an ensemble analysis operator that targets the complete Bayesian filtering distribution rather than only its conditional mean. The operator takes a forecast ensemble and a new observation as input and returns an analysis ensemble. A permutation-invariant transformer ensures that the result respects the exchangeability of ensemble members and can be evaluated at different ensemble sizes.

The key training device is a strictly proper scoring rule, implemented with the energy score. Its expected value is uniquely minimized by the true filtering distribution, yet its empirical form requires only simulated state–observation trajectories—not direct access to the generally intractable filtering distribution. Under a realizability assumption, the population objective recovers the Bayesian filter, and a mean-field time-averaging argument connects the practical single-trajectory objective to this population target.

Experiments cover a linear–Gaussian diagnostic, the strongly non-Gaussian doubling-angle model, Lorenz-63, and Lorenz-96. Energy-score training captures multimodal distributions missed by Gaussian filters and mean-based learning objectives. The results also reveal a useful architectural distinction: an EnKF correction provides an effective inductive bias in close-to-Gaussian problems, while a flexible end-to-end update is better suited to strongly non-Gaussian posterior geometry.

The manuscript has been submitted to the Journal of Machine Learning Research.

Learning the Whole Filtering Distribution with Proper Scoring Rules

Thu, 25 Jun 2026 00:00:00 +0000

Table of Contents

Our new paper, “Learning Probabilistic Filters with Strictly Proper Scoring Rules,” is now available on arXiv and has been submitted to the Journal of Machine Learning Research (JMLR).

Paper: ·
Code:
Publication page and BibTeX:
Original announcement:

Learning a Distribution Without Seeing It

Bayesian filtering aims to infer the evolving distribution of a hidden state from partial and noisy observations. That full distribution—not only its mean—is what represents uncertainty, multimodality, and the range of plausible system states.

This creates a basic learning problem: the true filtering distribution is generally intractable, so it is not available as a supervised target. A simulator can readily produce state trajectories and their corresponding noisy observations, but it does not hand us the conditional distribution we would like a filter to learn.

Our solution is to change the training objective. A strictly proper scoring rule evaluates a predictive distribution against a realized state, and its expected score is uniquely optimized by the true data-generating distribution. We use the energy score, which lets us train from simulated state–observation trajectories while still targeting the entire Bayesian filtering distribution.

The Proper Scoring Ensemble Filter

We introduce the proper scoring ensemble filter (PSEF). Its learned analysis map takes a forecast ensemble and the latest observation, then outputs an updated analysis ensemble. The map is implemented with a permutation-invariant transformer, respecting the fact that an ensemble represents an unordered empirical distribution. The same parameterization can also operate across different ensemble sizes, with lightweight fine-tuning where needed.

The paper studies two ways to parameterize the update:

Correction terms: retain the EnKF update as an inductive bias and learn how to correct it.
End to end: learn a more flexible analysis map without constraining it to an EnKF-style correction.

The distinction matters. In close-to-Gaussian settings, the EnKF structure is useful and the correction-based approach performs best. When the observation creates a strongly non-Gaussian or multimodal posterior, the end-to-end map has the flexibility needed to represent its geometry.

Why the Loss Function Matters

The header figure gives a controlled example on the doubling-angle model. The true posterior is bimodal. Training the same learning architecture with the energy score captures both modes across ensemble sizes, including ensembles much smaller than those used by the classical baselines. A normalized mean-squared objective and Kalman-type filters instead miss the distributional structure, because matching a conditional mean is not enough to learn uncertainty.

The theory mirrors this empirical difference. Under a realizability assumption, the population proper-scoring objective is minimized by the true Bayesian filtering distribution. We also connect the finite-ensemble, single-trajectory loss used in training to the population objective through a mean-field consistency and time-averaging argument.

Non-Gaussian Filtering in Lorenz Systems

Lorenz-63 with a partial square observation. The end-to-end, energy-score-trained PSEF most closely follows the particle-filter reference, including its cross-shaped and bimodal structures.

We evaluate PSEF on a linear–Gaussian diagnostic, the doubling-angle model, Lorenz-63, and Lorenz-96. In Lorenz-63, squaring the observed coordinate introduces sign ambiguity and produces complex non-Gaussian filtering distributions. The end-to-end PSEF trained with the energy score reproduces these structures more faithfully than mean-based learned filters and classical Gaussian ensemble filters.

Lorenz-96 results across identity, square, and arctangent observations. Proper-score-trained filters remain among the strongest methods, while the best architecture depends on the posterior geometry.

The Lorenz-96 results reinforce that there is no single architectural bias that is optimal for every problem. Correction terms are effective when the filtering distribution remains approximately unimodal, whereas the end-to-end energy-score model is strongest when the observation induces multimodality. Across both cases, probabilistic training is what allows the learned ensemble to target distributional accuracy rather than only state-estimation error.

Collaboration and Reproducibility

I led this project, including the implementation and numerical experiments, in collaboration with Eviatar Bach, Ricardo Baptista, Jochen Bröcker, and Andrew Stuart. The contains the code and experiment scripts.

Figures on this page were extracted from the paper, which is available under a CC BY 4.0 license.