Inferring single-trial neural population dynamics using sequential auto-encoders

Nature Methods 15:805–815, doi 10.1038/s41592-018-0109-9. Author manuscript, PMC6380887 (NIHMS1500948). Read by the main session for research direction R3, 4 October 2026. Provenance: papers/amadeus/pandarinath2018_lfads.provenance.json.

What was read

• Read in full: the PMC author-manuscript HTML, converted to text: abstract, introduction, results, discussion, figure captions, Methods tables 1–2, the full Online Methods (model equations, loss, GRU, AR(1) prior, stitching, hyperparameters, posterior averaging, related work, synthetic datasets, human and monkey datasets, analysis by figure), and the references.
• Not read: the supplementary files (supplement 1–3, Supp. Figs 1–12 and Supp. Tables, datasets 1–3, software, the three videos) and the figures as images. The synthetic-data results summarised as "LFADS outperforms GPFA, PfLDS and vLGP" sit in the unread supplement. Here they rest on the main text's statement.

What they did

• LFADS (Latent Factor Analysis via Dynamical Systems) treats each trial of spiking activity as generated by a nonlinear dynamical system, ẋ = F(x, u). It is a sequential variational autoencoder:
  • a bidirectional encoder RNN compresses each trial to an initial state g0;
  • a generator RNN (GRU) with fixed weights evolves g0 into low-dimensional factors, then Poisson rates for every neuron;
  • optionally, a controller RNN infers time-varying inputs u(t), regularised to be simpler than the dynamics.
  • Training maximises the evidence lower bound (Poisson likelihood minus KL terms). No behavioural or condition labels are given.
• Data:
  • monkey M1/PMd (Utah arrays, "Maze" task: 202 neurons, 2,296 trials, 108 conditions);
  • two people with paralysis in the BrainGate trial (T5, spinal cord injury; T7, ALS), with intracortical arrays;
  • monkey P: 44 sessions with 24-channel probes over 162 days;
  • "Cursor Jump" perturbation trials.

Main results (verified)

• Single-trial denoising predicts behaviour far better. Hand-velocity decoding gave R² 0.90 from LFADS rates, against 0.66 from smoothed spikes, 0.69 from GPFA and 0.34 from binning. LFADS with only 25–50 of the 202 neurons beat the other methods using all 202.
• Held-out neurons are predicted better than with GPFA (p < 1e-8 at every population size).
• Known rotational dynamics appear on single trials in monkey and human motor cortex: 2,296 monkey trials and 114 trials from T5.
• Dynamics generalise to unseen conditions. Generators trained without a target-angle bin modelled its trials with the same rotations. The initial jPCA position, held-in against held-out, correlated at r = 0.97 and 0.77.
• Stitching across months.
  • One shared encoder and generator, with a read-in and read-out matrix per session, fit 44 sessions over 162 days with entirely different neurons in each.
  • Condition-averaged factor trajectories were consistent across sessions.
  • The stitched model beat the 44 single-session models in kinematic decoding by a mean ΔR² of +0.22, with one decoder for all sessions. Reaction-time correlation improved by +0.15.
• Inferred inputs mark external events. In Cursor Jump, inferred inputs differed by target at target onset and by perturbation direction at the jump, and clustered by perturbation type on single trials.
• Fast oscillations in the inferred rates matched local field potentials (15–40 Hz), but only when inputs were allowed. Those fast dynamics are not reproduced by an autonomous system from the initial state alone.

Limits

• Trial-structured data. Short (800–1,200 ms), aligned trials of stereotyped motor tasks. Continuous, free-running activity is not modelled here.
• The authors' cautions:
  • "We advise against making inferences about properties of the biological network by studying the structure of the generator." It is an abstract dynamics model, not a mechanism.
  • Inferred inputs may be model mismatch or noise. They are acausal (the bidirectional encoder sees the whole trial), and their shape is not physiological.
  • Reconstruction cost is "not an ideal metric": with as many inputs as data dimensions, the model could copy the data and learn nothing.
• In practice, the AR(1) input prior collapsed to white noise (τ ≈ 0).
• Inconsistencies in the manuscript:
  • Maze trial length is given as 800 ms, 900 ms (±450 ms) and ±400 ms in different places;
  • the methods cite stitching as "Fig. 5" (it is Fig. 4) and swap the LFP and Cursor Jump figure numbers.
  • None affects the main claims.
• Invasive intracortical arrays in a few monkeys and two clinical-trial participants.

What it means for Amadeus (inference)

• This is the user's "constant, not the cause" in working form. LFADS learns the governing dynamics F of a population and explicitly disclaims mechanism: an abstract model that "captures the computations being performed" without modelling biological components. That is P3 (function, not mechanism), proven for local circuits in tasks.
• A month of recording needs stitching. Electrodes drift and populations change across days. LFADS shows that separately recorded, non-overlapping populations across 5 months can be fused into one consistent dynamical model per subject, and that the fusion beats per-day models. A model of one brain over a month would be built this way: a shared per-person core, per-session adapters.
• Spontaneous thought is the open problem. The authors name it as a target: internal dynamics matter most where computations "have no clear, observable external behavioral correlates on a moment-by-moment basis, such as integration of evidence … or attentional regulation". Their method was shown only on aligned motor trials. Free-running daydreaming has no trials to align, so inferred inputs are the natural place where "something new entered" shows up. Their caveats (acausal, possibly mismatch) mean such inputs need behavioural anchors (experience sampling, R2) before they are read as thoughts.
• An evaluation warning that transfers. A generative model can reconstruct its data perfectly while learning nothing, if its input channel is wide enough. Amadeus's slot faces the same trap: a slot that can store raw records verbatim reconstructs them without modelling the person. Judge by held-out prediction (held-out neurons, conditions, sessions), as LFADS does, not by reconstruction. This matches P5's learning-equivalence stance.