Michael C. Burkhart

Hello

… and welcome to my website.

About Me

I earned my Ph.D. in 2019 from Brown University's Division of Applied Mathematics. For my dissertation, I derived a novel approach to Bayesian filtering, the Discriminative Kalman Filter, motivated by and developed with my advisor M. Harrison and collaborators D. Brandman and L. Hochberg. We validated and successfully implemented this filter as part of the BrainGate Clinical Trial that enables participants with quadriplegia to communicate and interact with their environments in real time using mental imagery alone.
I then spent three years working as a machine learning scientist at Adobe in California. In 2021, I joined Cambridge University as a research associate to develop machine learning-based approaches for the early diagnosis of neurodegenerative disease.

Filtering

Suppose there is some underlying process Z_1:t=Z₁,…,Z_t about which we are very interested, but that we cannot observe. Instead, we are sequentially presented with observations or measurements X_1:t=X₁,…,X_t where each X_i depends only on the current latent state Z_i. We visualize this process with the following graph:

Filtering is the process by which we use the observations X₁,…,X_t to form our best guess for the current latent state Z_t.

Dynamic State Space Models

Under the Bayesian approach to filtering, X_1:t , Z_1:t are endowed with a joint probability distribution. The graphical model encodes the process to generate X_1:t , Z_1:t as:

This model is variously known as a dynamic state-space or hidden Markov model. It provides a visual description of how to generate a sample x_1:t , z_1:t from the random variables X_1:t , Z_1:t. We start with z₁ drawn from its marginal distribution p(z₁). We then generate an observation x₁ using the distribution p(x₁|z₁). At each subsequent time step t, we draw z_t from the distribution p(z_t|z_t-1) and x_t from the distribution p(x_t|z_t). These two conditional distributions are very important and characterize the generative process up to the specification of Z₁. The first, p(z_t|z_t-1), relates the state at time t to the state at time t-1 and is often called the state or prediction model. The second, p(x_t|z_t ), relates the current observation to the current state and is called the measurement or observation model.

Bayesian Filtering

The Bayesian solution to the filtering problem returns the conditional distribution of Z_t given that X₁ ,…, X_t have been observed to be x₁ ,…, x_t. We refer to this distribution p(z_t|x_1:t) as the posterior distribution, or simply the posterior. The Chapman–Kolmogorov recursion

relates the posterior at time t to the one at time t-1. Bayesian filtering solves or approximates the above recursion. Common approaches include Kalman filtering, variational methods, quadrature methods, and Monte Carlo-based particle filtering.

Kalman Filter

The Kalman filter specifies both the state model and measurement model as linear, Gaussian.

Here, η_d(· ; μ ,Σ) denotes the d-dimensional Gaussian density with mean μ and covariance Σ. In this way, the posterior is Gaussian and quickly computable. NASA used a variant of this filter to orient the Apollo Lunar Module and land the first humans on the moon.

Our Approach

We apply Bayes' rule to the measurement model and make the Gaussian approximation p(z_t|x_t ) ≈ η_d(z_t ; f(x_t ), Q(x_t )) where the functions f and Q can be learned from data.

This approach allows for a nonlinear relationship between the measurements and latent states and has been found to perform particularly well when the X_t are much higher dimensional than the Z_t, as is often the case with neural decoding.

Discriminative Kalman Filter

The Discriminative Kalman Filter adopts the Kalman state model, p(z_t|z_t-1) = η_d(z_t ; Az_t-1, Γ), with initialization p(z₀) = η_d(z₀; 0, S) where S satisfies S=ASA'+Γ (so that the latent process is stationary) and uses the measurement model introduced above. Given these specifications, it follows that we may recursively approximate the posterior as Gaussian. Namely, if

p(z_t-1|x_1:t-1) ≈ η_d(z_t-1 ; μ_t-1, Σ_t-1),

then given a new observation X_t=x_t, we have that

p(z_t|x_1:t ) ≈ η_d(z_t ; μ_t , Σ_t )

where

M_t = AΣ_t-1A' + Γ,

Σ_t = (M_t^-1 + Q(x_t )^-1 - S^-1)^-1,

μ_t = Σ_t(M_t^-1Aμ_t-1 + Q(x_t )^-1f(x_t)).

This approximation is functionally exact when Q(x_t )^-1 - S^-1 is positive-definite; otherwise we let

Σ_t = (M_t^-1 + Q(x_t )^-1)^-1.

In this way, the Discriminative Kalman Filter maintains fast, closed-form updates while allowing for a nonlinear relationship between the latent states and observations. When supervised training data is available, off-the-shelf nonlinear/nonparameteric regression tools can readily be used to learn the discriminatively specified observation model. In related work, we demonstrate how this framework can also be leveraged to ameliorate non-stationarities, or changes to the relationship between the latent states and observations, and increase the robustness of estimates.