AIs trained on biological data are advancing quickly. I lead a research program which applies mechanistic intepretability to them to test and improve fundamental mechinterp methods, conduct biosecurity audit, and extract novel biological knowledge from them.
Such models are trained on large collections of DNA sequences, proteins, gene-expression profiles, cell images, tissue samples, spatial measurements, and other biological data. They are usually trained to predict missing data in a self-supervised manner.
Sometimes people come and ask what could be good research questions to work on within this agenda. Here I aim to answer this question.
The central hope behind biological mechanistic interpretability is that bio models contain more than useful input-output mappings. Their internal representations may contain compressed maps of proteins, pathways, cell states, regulatory systems, developmental trajectories, and evolutionary structure. Mechanistic interpretability could let us read those “map inside the machine”, determine how the models use them, and extract knowledge that survives independently of the original model.
Overall, for almost any question, there are at least 6 levels of what reseach can result in, in the order of increasing coolness:
Some biological property can be decoded from the model.
An internal feature has a coherent biological interpretation.
The feature adds information beyond simple baselines and known confounders.
Intervening on it causally changes the model’s computation.
The result transfers across data, models, or species, or becomes a standalone tool.
It predicts a previously unknown biological fact that survives prospective experimental validation.
Most of the difficult work begins between levels three and five.
The problems below range from relatively direct replication projects to long-term bets. Some ask how to interpret existing models. Others ask how to train models whose internals are worth interpreting in the first place.
I. What would count as a mechanistic result?
1. Build a shared framework of “interpretability”
Different papers currently use words such as “interpretable,” “causal,” and “mechanistic” for very different levels of evidence. The field needs a standard framework stating whether a result is merely decodable, statistically predictive, causally important inside the model, biologically causal, transferable, or experimentally validated. Overall, bio mechinterp is lagging behind LLM mechinterp (not that LLM mechinterp is advancing very quickly), and one can do advancement just by applying most recent tools from LLM mechinterp or by bringing more rigor.
2. Define “biologically meaningful” more strictly
A feature may be enriched for a known gene pathway because the pathway contains highly expressed genes, well-annotated genes, or genes that occur frequently in the training data. Biological interpretation should be tested against matched all kinds of of controls: annotation frequency, expression level, gene degree, alternative explanations.
3. Find the right level of explanation
Should we explain a biological model using individual genes, protein motifs, sparse features, pathways, low-dimensional manifolds, attention heads, or entire circuits? The most useful level may vary by task. We need criteria for choosing a level that remains stable under retraining and intervention.
4. Attach calibrated uncertainty to interpretations
We need better uncertainty estimates for feature names, circuit edges, pathway assignments, and proposed mechanisms. “Unannotated” should also be separated into several possibilities: possibly novel biology, incomplete databases, mixed features, technical artifacts, and failed interpretation.
5. Measure the scientific return on interpretability
For a fixed amount of compute, researcher time, and laboratory money, does interpretability generate better hypotheses than differential expression, classical network inference, black-box attribution, or ordinary biological screening? This comparison will determine whether the field becomes a useful scientific instrument or an elaborate way to produce attractive diagrams.
II. Where does biological knowledge live inside a model?
Systematically compare embeddings, attention, feed-forward blocks, residual streams, normalization layers, and output heads under the same tasks and controls (and for many models). This would tell us where each kind of biological information is stored and where it is actually transformed.
7. Explain layer-by-layer transformations
What changes as a gene, protein, or cell representation moves through the network? Early layers may retain local relationships, while deeper layers may compress them into broad categories. We need explicit computational accounts of these transformations rather than a sequence of layer-wise gene set enrichment plots.
8. Recover context-dependent gene representations
The same gene behaves differently in a stem cell, tumour cell, neuron, and macrophage. How does its internal representation change across contexts? Can we identify which dimensions remain stable and which encode tissue, disease, developmental stage, or environmental response?
9. Measure how distributed each concept is
Some biological concepts may correspond to one sparse feature. Others may require hundreds of features or a curved region of representation space. We need measures of distributedness that distinguish actually distributed computation from poor feature decomposition.
10. Separate stored knowledge from used knowledge
A model may contain a decodable representation of a pathway without using it for the current prediction. Causal interventions, mediation analysis, and task-conditioned circuit tracing should determine which stored biological facts participate in particular computations.
11. Track what fine-tuning changes
Fine-tuning may add new biological representations, repurpose existing ones, or merely alter the output head. Mapping feature and circuit changes before and after fine-tuning could reveal how disease classifiers, perturbation predictors, and generative models acquire specialised capabilities.
12. Watch biological knowledge emerge during training
Train small biological models while saving many intermediate checkpoints. When do pathways, cell identities, regulatory relationships, evolutionary structure, or dangerous capabilities first appear? Do they emerge gradually, suddenly, or in a predictable order?
13. Relate scale to extractable knowledge
Single-cell reconstruction models show useful scaling when data are abundant and little benefit from model growth when data are scarce. An open question is whether interpretable biological knowledge follows the same scaling laws as predictive loss, and whether larger models contain qualitatively new mechanisms or simply denser versions of old ones.
III. Gene regulatory network inference
A gene regulatory network is a graph of how regulators influence target genes. Attention matrices initially looked like convenient ready-made networks, but systematic tests found that attention often tracks co-expression or gene-level importance rather than unique causal edges. Residual geometry appears richer, yet real benchmarks which rely on wet lab gene perturbations are very noisy, and method rankings can change when the tissue, reference database, or candidate edge set changes.
14. Locate regulatory signal outside attention
Search systematically through residual streams, sparse features, feed-forward blocks, output gradients, and combinations of components. The main question is whether any internal substrate contains regulatory information beyond expression level, co-expression, and regulator popularity. In particular, my experiments with tensor networks demonstrated that quite a lot of regulatory information is contained in their FFN layers.
15. Recover causal direction
A model may place two related genes close together without representing which gene influences the other. Can internal interventions distinguish regulator-to-target relationships from target-to-regulator relationships, shared causes, and symmetric functional similarity?
16. Separate direct from indirect regulation
If regulator A affects gene C through intermediate gene B, a model may encode all three pairwise relationships. We need methods that recover the shortest plausible causal chain and distinguish direct molecular interactions from downstream consequences.
17. Infer activation and repression separately
Regulatory edges have signs. A transcription factor may activate one target and repress another, depending on context. Can a model’s internal computation reliably recover the direction of expression change after intervention rather than merely identifying related genes?
18. Build cell-state-specific networks
A single universal network is usually biologically unrealistic. Regulatory relationships change across tissues, cell types, developmental stages, disease states, and environmental conditions. Interpretability methods should recover conditional networks and explain which internal features switch particular edges on or off.
19. Recover network rewiring under perturbation
When a gene is knocked down, a drug is added, or a cell becomes diseased, the network itself may change. Can model internals show which connections disappear, which compensatory pathways activate, and which new dependencies emerge?
20. Add time to network extraction
Most single-cell datasets are static snapshots, like images. Regulatory mechanisms unfold over minutes, hours, and developmental timescales. Models trained on time-course data could be interpreted to recover delays, transient interactions, and the ordering of regulatory events.
21. Find feedback loops
Feedback is central to homeostasis, differentiation, oscillation, and disease. Standard pairwise scoring can miss feedback or confuse it with correlation. Can circuit tracing identify closed regulatory loops and determine whether they are stabilising, amplifying, or oscillatory?
22. Model combinatorial regulation
Genes are often controlled by combinations of transcription factors rather than one regulator at a time. We need methods for detecting AND-like, OR-like, threshold, cooperative, antagonistic, and context-dependent combinations inside model circuits.
23. Train and interpret perturbation-native models
Models trained mainly on observational expression snapshots have little incentive to learn causal direction. Train foundation models directly on large perturbation datasets, then ask whether their internals contain better regulatory networks than those of masked-expression models.
24. Fuse transcriptomic and regulatory modalities
Combine gene expression with chromatin accessibility, transcription-factor binding, protein abundance, methylation, spatial context, and perturbation outcomes. Interpretability could reveal which modality contributes evidence for each proposed edge in the gene regulatory network and where different modalities disagree.
25. Handle unmeasured variables
Many apparent gene-to-gene relationships are driven by missing proteins, metabolites, environmental signals, or hidden cell states. Can models represent latent causes explicitly? Can interpretability detect when a proposed regulatory edge is really a proxy for an unobserved variable?
26. Make benchmark rankings stable
Network-inference rankings can reverse when researchers change the candidate gene set, tissue, or reference database. We need to build evaluation suites that report performance across a distribution of reasonable protocols and expose methods whose apparent superiority depends on one convenient setup.
27. Use active learning to select perturbations
Given a partially inferred network, which next gene knockout, combination, dose, or time point would reduce uncertainty most? Interpretability-derived networks could become active experiment-design systems.
28. Export executable regulatory models
A static edge list is limited. Can we extract a compact dynamical system that predicts how gene expression evolves after interventions? Such a model might use equations, logical rules, or a small neural circuit and remain usable without the original foundation model.
IV. Causal circuits and internal computation
My circuit-tracing work has begun to map directed dependencies between sparse features in single-cell models. These maps show biological coherence, strong inhibitory structure, substantial redundancy, but only a weak bridge from model-internal causal edges to real regulatory causality.
29. Map complete circuits rather than selected examples
Researchers often choose a few features that already have clear annotations. Exhaustive tracing can reveal different global structure, including unannotated hubs. We need scalable methods that map all active components without requiring millions of prohibitively expensive interventions.
30. Translate feature circuits into biological variables
A directed graph between sparse features remains a graph between artificial objects. Develop principled ways to map those nodes and edges onto genes, pathways, molecular processes, cell states, and experimentally measurable quantities.
31. Characterise higher-order redundancy
Ablating one feature may have little effect because several other features perform overlapping functions. Pairwise tests are insufficient when three, ten, or a hundred components compensate for one another. We need efficient methods for finding minimal causal sets.
32. Detect synergy as well as redundancy
Some computations may appear only when several features act together. Develop tests that identify superadditive interactions without enumerating every possible feature combination.
33. Validate that interventions remain on-manifold
Zeroing or amplifying an activation can create hidden states that the model never encountered during training. Compare different interventions, including patching from real examples, projection onto valid activation regions, and constrained optimization, to determine which causal conclusions survive.
34. Explain inhibitory dominance
Many biological circuit maps contain many inhibitory feature-to-feature effects. Does this reflect biological suppression, competition between representations, normalization, sparse coding artifacts, or the model’s way of removing irrelevant information?
35. Recover context-conditioned circuits
The circuit used for a gene in one cell type may differ sharply from the circuit used in another. Build methods that return conditional circuits and identify stable cores, context-specific branches, and switches between alternative computations.
36. Measure circuit completeness and faithfulness
A proposed circuit should reproduce the model’s behaviour when run independently or when substituted into the original model. Develop tests for how much performance, calibration, and robustness the circuit preserves across inputs.
37. Edit circuits with local side effects
Can we remove a technical artifact, disease-associated shortcut, or unsafe capability while preserving unrelated functions? Circuit editing should report both the intended change and a broad audit of collateral damage.
38. Find universal biological circuits
Do independently trained models converge on equivalent circuits for translation, secretion, immune activation, differentiation, or DNA repair? Cross-model convergence would provide evidence that a circuit reflects durable structure in the data-generating process.
V. Sparse features and representation geometry
Sparse autoencoders have exposed interpretable features in protein, single-cell, microscopy, pathology, and physiological models. Their outputs are still method-dependent: two autoencoders trained on the same activations can recover different feature sets, and different dictionary sizes can split or merge concepts.
39. Make sparse features stable across random seeds
Measure which features recur across independent training runs and which are arbitrary decompositions. Develop alignment methods, consensus dictionaries, or objectives that reward stable recovery without forcing trully ambiguous concepts into artificial agreement.
40. Understand feature splitting and merging
A broad immune-response feature at one dictionary size may become several cytokine, interferon, and stress features at another. We need a hierarchical account of how features divide, combine, and relate across levels of resolution.
41. Align features across models
Create methods for matching functionally equivalent features between different architectures, model sizes, training datasets, and modalities. A shared feature may be rotated, split, distributed, or implemented through a different circuit, so simple vector similarity will often fail.
42. Recover rare biological concepts
Rare cell states, transient developmental events, unusual pathogens, and uncommon diseases may contribute little to an average reconstruction objective. Train specialised dictionaries or sampling schemes that recover rare features without hallucinating structure from a handful of examples.
43. Interpret negative and absence-based features
A feature may represent the absence of a pathway, loss of differentiation, missing structural elements, or suppression of a response. These concepts are harder to annotate because biological databases mainly record positive associations.
44. Learn nonlinear and multiscale features
Standard sparse autoencoders represent activations as linear combinations of features. Biology contains spatial patterns, sequence motifs, interactions, trajectories, and nested hierarchies. Develop feature models that preserve this richer structure while remaining causally testable. I tested bilinear autoencoders, without much success so far.
45. Connect sparse features to global geometry
Sparse features describe recurring directions, while manifold methods describe the shape of whole representation spaces. Determine when these views agree, when they capture different information, and whether sparse features can serve as coordinates on biological manifolds.
46. Make topological findings protocol-robust
Clusters, branches, loops, and curved manifolds can change with preprocessing, distance metric, sample density, and dimensionality reduction. A topological claim should survive plausible analysis choices, matched null models, and external datasets.
47. Separate sequence semantics from cellular semantics
A single-cell model may appear to know gene function because its gene embeddings inherit information that a protein-sequence model could already supply. Every claim about cell-derived knowledge should compare against sequence-only, expression-only, and annotation-only baselines.
48. Interpret genuinely unknown features
Large language models can generate plausible names for almost any gene list. Build annotation pipelines that show evidence, counterevidence, nearest known concepts, causal behaviour, and experimental tests, while allowing the label “currently unknown.”
VI. Extracting standalone biological algorithms
In one my study, I report a compact algorithm representing blood cell development extracted from scGPT through frozen attention operators. The extracted algorithm runs 34.5 times faster than a standard multilayer probe, uses roughly 1,000 times fewer trainable parameters, and retains interpretable cell lineage factors. This is a promising demonstration, and independent replication is now very valuable.
49. Independently replicate the hematopoietic result
Apply the same pipeline to independently processed datasets, alternative checkpoints, other single cell foundation models, and strict donor-held-out evaluations. Replication would establish whether the method is general or unusually well matched to one model and developmental system.
50. Extract algorithms for more bioinformatics tasks
Search for attention heads, convolutional filters, state-space operators, graph-message-passing rules, or small feature circuits that can be copied out of a model with little or no retraining. Direct export gives especially strong evidence that the computation was already present.
52. Translate neural computations into equations or rules
Can an internal circuit be expressed as a scoring function, differential equation, logical program, graph algorithm, or short piece of conventional code? Symbolic extraction would make biological assumptions easier to inspect and modify.
53. Eliminate target-dataset retraining
A compelling extracted algorithm should work on new data using fixed parameters or minimal calibration. Otherwise, performance may come from the newly trained adaptor rather than from knowledge recovered from the foundation model.
54. Test extreme distribution shifts
Evaluate extracted algorithms across donors, laboratories, platforms, tissues, diseases, species, and evolutionary distances. The most interesting result would be an algorithm that transfers further than the original model’s usual downstream pipelines.
55. Compare extraction against purpose-built methods
For each task, compare predictive performance, calibration, speed, memory, data requirements, interpretability, and maintenance cost. A compact extracted algorithm should earn its place against mature specialist bioinformatics tools.
56. Compose extracted algorithms
Can separate exported modules for cell identity, developmental state, signalling, and perturbation response be joined into a larger transparent system? Composition may reveal whether the original model contains reusable biological subroutines.
57. Preserve uncertainty and safety during extraction
Compact tools can make capabilities cheaper and easier to distribute. Algorithm extraction should include a review of whether compression amplifies access to potentially dangerous biological capabilities.
VII. Novel biological discovery and laboratory validation
Interpretable features have already been proposed for filling missing protein annotations, identifying model-driving biomarkers, refining networks, mapping cell states, and guiding experimental design. The decisive next step is prospective validation in experiments selected before the result is known.
58. Discover biomarkers that survive model removal
Use interpretability to identify a compact set of measurable variables that are associated with some condition (like disease or age), then build a conventional statistical model directly from raw biological data. Validation should occur in an external cohort with no dependency on the original foundation model.
59. Discover previously unrecognised cell states
Find internal features or geometric regions that identify reproducible cell populations absent from existing taxonomies. Require coherent molecular programmes, recurrence across donors, and confirmation through independent measurements or functional experiments.
60. Find hidden transition states
Developmental and disease processes may pass through brief intermediate states that standard clustering misses. Representation geometry could identify bottlenecks, branches, or sparse regions and suggest markers for experimentally isolating those cells.
61. Investigate unannotated circuit hubs
Exhaustive circuit mapping can reveal highly connected features with no clear database annotation. These hubs are promising discovery targets, although they may also encode technical properties of datasets. Characterise them through targeted perturbations and independent datasets.
62. Map conserved and divergent mechanisms
Compare equivalent internal circuits across tissues and species. Stable cores may reveal conserved biological programmes, while specific points of divergence may identify tissue-specific or lineage-specific regulatory innovations.
63. Prioritise drug targets through selective circuit control
Identify perturbations that suppress disease-associated features while preserving healthy-state features. Of course, experimental validation should test both efficacy and specificity, since “strongly changes the model” is a rather adventurous drug-development criterion.
64. Discover side-effect and toxicity programmes
Trace how candidate interventions activate stress, immune, metabolic, or cell-death circuits. This could expose likely liabilities before expensive animal studies, provided the predicted programmes transfer to real experimental systems.
65. Design experiments from representation geometry
Use uncertainty, curvature, sparse regions, and disagreements between models to choose new samples or interventions. Compare this strategy against random selection, expert selection, diversity sampling, and ordinary active learning.
66. Automate discovery without automating self-deception
Agentic systems could search thousands of feature, pathway, and circuit hypotheses. They also create a powerful multiple-testing machine. Automated pipelines need held-out data, explicit null models, correction for repeated search, preregistered stopping rules, and independent reviewer agents.
VIII. Training better models for mechanistic biology
Interpretability sometimes reveals that some models never learned the desired mechanism. In that case, increasingly elaborate microscopes will not manufacture a missing object. Model objectives, data, and architectures may need to change.
67. Pretrain directly on interventions
Include gene knockouts, gene activation, drug treatments, environmental changes, and combinations during pretraining. Then test whether causal direction and regulatory effects become easier to recover from internal representations.
68. Use counterfactual training objectives
Ask models to predict how the same biological system would behave under alternative interventions. Counterfactual objectives may encourage representations of mechanisms that remain stable across changing conditions.
69. Train on time and lineage information
Add time courses, lineage tracing, developmental ancestry, and repeated measurements. These data may help models distinguish causes, consequences, transient responses, and stable cell identities.
70. Build multimodal models
Jointly model DNA, chromatin, RNA, protein abundance, metabolites, morphology, spatial position, and perturbations. Interpretability should reveal which modalities contribute unique information and where the model learns cross-modal causal links. I am training such models now, so you can join!
71. Add causal inductive biases
Explore directed graph modules, explicit intervention variables, structural causal models, dynamical systems, and asymmetric message passing. Compare these designs against ordinary transformers under equal data and compute.
72. Make models interpretable by construction
Use modular pathways, sparse bottlenecks, hierarchical representations, explicit gene sets, and constrained interactions during training. The key question is how much interpretability can be gained without forcing the model’s performance to deteriorate.
73. Compare architecture families mechanistically
Transformers, graph neural networks, state-space models, diffusion models, variational autoencoders, and neural differential equations may learn different biological abstractions. Evaluate both predictive performance and the recoverability of causal mechanisms.
74. Remove technical variation during training
Develop objectives that suppress batch, laboratory, platform, and donor artifacts while preserving real biological heterogeneity. Interpretability can then verify whether deconfounding succeeded or merely hid the artifact in a less obvious subspace.
75. Derive scaling laws for causal knowledge
Measure how regulatory direction, perturbation transfer, circuit coherence, algorithm extractability, and discovery rates change with model size and data diversity. Predictive loss may improve smoothly while causal knowledge remains flat, or vice versa.
76. Use synthetic causal curricula
Generate artificial cells, gene networks, proteins, or evolutionary systems with exactly known mechanisms. Train models from simple to complex systems and test whether interpretable representations track the known causal structure.
IX. Replication, transfer, and benchmark design
Replication has special importance here because model training, feature decomposition, biological preprocessing, and reference databases all introduce degrees of freedom. Network-method rankings have already shown meaningful sensitivity to evaluation choices.
77. Replicate across random seeds
Repeat model training, feature extraction, circuit tracing, and downstream evaluation across seeds. Report which conclusions remain stable even when individual features rotate, split, or disappear.
78. Replicate across checkpoints and model sizes
A mechanism found in one checkpoint may be transient or scale-specific. Track equivalent features and circuits across small, medium, and large versions of the same model family, like for example for Geneformer.
79. Replicate across architectures
Test whether findings from a transformer survive in graph, state-space, convolutional, or variational models trained on comparable data. Convergence across architectures is stronger evidence than recurrence within one model family.
80. Replicate across datasets, laboratories, and platforms
Run the same analysis on data from different sequencing technologies, sample-processing protocols, hospitals, and laboratories.
81. Replicate across donors, tissues, diseases, and species
State the boundary conditions of every mechanism. Some circuits may be universal, some human-specific, some tissue-specific, and some artifacts of one immortalised cell line.
82. Replicate across intervention types
Compare gene knockout, partial knockdown, overexpression, chemical inhibition, dosage variation, and environmental changes.
83. Standardise leakage-resistant evaluation
Biological data contain near-duplicates, related donors, homologous sequences, shared studies, and annotation leakage. Benchmarks should split data according to the scientific generalisation claim, including donor, laboratory, tissue, family, or evolutionary-distance splits.
84. Publish negative replications and versioned atlases
Feature atlases should record model versions, data versions, autoencoder settings, annotation databases, and confidence. Failed replications should be searchable alongside successes.
X. Beyond ordinary single-cell gene expression
I focus mostly on single cell models, mostly because I believe there is much more knowledge to extract from them. But mechanistic interpretability is already spreading across proteins, microscopy, pathology, spatial biology, and physiological signals. Each modality offers different ground truth and different opportunities for scientific discovery.
85. Recover protein mechanisms from sequence models
Move from features correlated with known annotations to circuits that compute binding, catalysis, structural stability, localization, or allosteric effects. Validate proposed mechanisms using mutational scans and biochemical assays.
86. Interpret protein-generation decisions
Which internal features cause a generative protein model to favour a fold, active site, binding interface, or taxonomic family? Can steering alter one property while preserving stability and unrelated functions?
87. Decode genomic regulatory grammar
Genome models may learn promoter structure, enhancers, splice signals, chromatin boundaries, long-range interactions, and evolutionary constraints. Identify the internal algorithms that combine motifs over thousands or millions of bases.
88. Interpret RNA foundation models
RNA models combine sequence, secondary structure, modification, localization, expression, and interaction information. Open problems include recovering folding rules, binding programmes, splicing mechanisms, degradation signals, and context-dependent regulatory elements.
89. Map spatial tissue circuits
Spatial models can represent which cells influence their neighbours and how tissue architecture constrains signalling. Interpret them to recover local communication rules, niches, boundaries, and disease-associated neighbourhoods.
90. Extract morphological programmes from microscopy models
Sparse dictionary methods can already recover cell types and perturbation-related morphology. The next step is to identify causal visual programmes, connect them to molecular pathways, and design experiments around previously unnamed morphological states.
91. Interpret structural and interaction models
Study how protein-structure and molecular-interaction models represent geometry, physical constraints, confidence, alternative conformations, and binding. This may reveal computational approximations that can be exported into faster structural algorithms.
92. Trace cross-modal circuits in multi-omics models
When a model predicts gene expression from chromatin or proteins from RNA, which internal pathways carry the information? Cross-modal circuit tracing could expose real biological coupling as well as shortcut mappings.
93. Extract signalling, metabolic, and whole-cell computations
Extend interpretability to models of signalling networks, metabolic flux, cellular simulations, and virtual cells. The long-term target is a comprehensive hierarchy connecting molecular events to cell behaviour.
XI. Biosecurity and safety auditing
My take is that you can’t do security audit of bio models without mechinterp. Genome language models have now been used to generate complete viable bacteriophage genomes, and a separate work found that fine-tuning could partially restore virus-related capabilities in a model whose original training data excluded certain viral sequences.
94. Define risk-relevant internal capabilities
Specify which internal abilities matter for biosecurity: for example, they may relate to understanding host range, immune interaction, functional sequence constraints, molecular evasion, or whole-genome coherence.
95. Locate risk-relevant features and circuits
Can interpretability identify internal representations associated with hazardous biological functions before researchers rely on broad output evaluations? Such features would need causal validation and careful access controls.
96. Detect pre-capability indicators
A model may develop partial components of a capability before succeeding on an end-to-end benchmark. Track precursor features, reusable subcircuits, and sudden changes in representation geometry across training checkpoints.
97. Test the robustness of training-data exclusion
When sensitive categories are removed from pretraining, determine which related abstractions remain. Models may reconstruct missing capabilities from neighbouring biological domains, evolutionary regularities, or later fine-tuning.
98. Monitor capability restoration during fine-tuning
Interpretability could show which dormant representations become reactivated or recombined during adaptation. This may support earlier detection than waiting for a model to succeed on a complete misuse-relevant task.
99. Develop mechanistically verified unlearning
After removing sensitive data or capabilities, test whether the relevant circuits have actually changed. Output suppression alone may leave recoverable internal knowledge intact or move it into a different representation.
100. Red-team interpretability-based safeguards
Assume that an adversary knows the auditing method. Test whether fine-tuning, representation rotation, feature splitting, model merging, or alternative prompting can bypass feature-level monitors.
101. Use interpretability to improve sequence screening
Internal representations may help identify functional similarity even when raw sequence similarity is low. Evaluate whether interpretable features can complement existing screening systems without producing unacceptable false-positive rates.
XII. Biology as a testbed for general mechanistic interpretability
Biological models offer unusually rich ground truth: known pathways, protein structures, evolutionary relationships, perturbation experiments, lineage information, and measurable phenotypes. They are also often smaller than frontier language models. This makes them useful model (but also real) organisms for mechanistic interpretability itself, provided biological databases are not mistaken for perfect ground truth.
102. Create “model organisms” for mechanistic interpretability
Select a small set of biological models, datasets, and tasks for repeated, exhaustive study. The equivalent of E. coli or Drosophila for interpretability would let methods accumulate on shared systems rather than restarting from zero in every paper.
103. Build synthetic biology tasks with exact ground truth
Generate data from known regulatory networks, evolutionary processes, spatial systems, and biochemical simulators. Then test whether interpretability methods recover the exact mechanisms used to generate both the data and the model’s predictions.
Run attention analysis, probing, gradients, sparse autoencoders, transcoders, causal tracing, causal abstraction, geometry, and circuit discovery on identical models. Evaluate accuracy against shared causal and biological targets.
105. Test methods under biological redundancy
Biological systems contain duplicated genes, parallel pathways, compensation, feedback, and many-to-one mappings. These properties create a demanding benchmark for whether interpretability can recover distributed and degenerate computations.
106. Exhaustively analyse very small biological models
Train small transformers or graph models on controlled biological tasks and map every relevant neuron, feature, edge, and intervention. Complete explanations on small systems can test assumptions before scaling methods to billion-parameter models.
107. Release shared activation and intervention datasets
Store model activations, sparse features, patching results, perturbation outcomes, and annotation mappings in standard formats. This would let researchers compare methods without repeatedly paying the full inference cost.
108. Build agentic research loops with independent quality gates
Automate hypothesis generation, analysis, replication, and report production while separating executor, critic, statistician, and biological-review roles. Hold-out datasets and preregistered tests should remain inaccessible to the hypothesis-generating agent until evaluation.
109. Just apply modern mechinterp techniques to bio models and see if the results are better
Manifolds, j-space, natural language autoencoders, compositional methods whatever—it is (usually) easier to test it on bio models as compared to LLMs.
If you start working on it, please get in touch with me to coordinate!
Open Problems in Mechanistic Interpretability of Biological Foundation Models
AIs trained on biological data are advancing quickly. I lead a research program which applies mechanistic intepretability to them to test and improve fundamental mechinterp methods, conduct biosecurity audit, and extract novel biological knowledge from them.
Such models are trained on large collections of DNA sequences, proteins, gene-expression profiles, cell images, tissue samples, spatial measurements, and other biological data. They are usually trained to predict missing data in a self-supervised manner.
Sometimes people come and ask what could be good research questions to work on within this agenda. Here I aim to answer this question.
The central hope behind biological mechanistic interpretability is that bio models contain more than useful input-output mappings. Their internal representations may contain compressed maps of proteins, pathways, cell states, regulatory systems, developmental trajectories, and evolutionary structure. Mechanistic interpretability could let us read those “map inside the machine”, determine how the models use them, and extract knowledge that survives independently of the original model.
Overall, for almost any question, there are at least 6 levels of what reseach can result in, in the order of increasing coolness:
Some biological property can be decoded from the model.
An internal feature has a coherent biological interpretation.
The feature adds information beyond simple baselines and known confounders.
Intervening on it causally changes the model’s computation.
The result transfers across data, models, or species, or becomes a standalone tool.
It predicts a previously unknown biological fact that survives prospective experimental validation.
Most of the difficult work begins between levels three and five.
The problems below range from relatively direct replication projects to long-term bets. Some ask how to interpret existing models. Others ask how to train models whose internals are worth interpreting in the first place.
I. What would count as a mechanistic result?
1. Build a shared framework of “interpretability”
Different papers currently use words such as “interpretable,” “causal,” and “mechanistic” for very different levels of evidence. The field needs a standard framework stating whether a result is merely decodable, statistically predictive, causally important inside the model, biologically causal, transferable, or experimentally validated. Overall, bio mechinterp is lagging behind LLM mechinterp (not that LLM mechinterp is advancing very quickly), and one can do advancement just by applying most recent tools from LLM mechinterp or by bringing more rigor.
2. Define “biologically meaningful” more strictly
A feature may be enriched for a known gene pathway because the pathway contains highly expressed genes, well-annotated genes, or genes that occur frequently in the training data. Biological interpretation should be tested against matched all kinds of of controls: annotation frequency, expression level, gene degree, alternative explanations.
3. Find the right level of explanation
Should we explain a biological model using individual genes, protein motifs, sparse features, pathways, low-dimensional manifolds, attention heads, or entire circuits? The most useful level may vary by task. We need criteria for choosing a level that remains stable under retraining and intervention.
4. Attach calibrated uncertainty to interpretations
We need better uncertainty estimates for feature names, circuit edges, pathway assignments, and proposed mechanisms. “Unannotated” should also be separated into several possibilities: possibly novel biology, incomplete databases, mixed features, technical artifacts, and failed interpretation.
5. Measure the scientific return on interpretability
For a fixed amount of compute, researcher time, and laboratory money, does interpretability generate better hypotheses than differential expression, classical network inference, black-box attribution, or ordinary biological screening? This comparison will determine whether the field becomes a useful scientific instrument or an elaborate way to produce attractive diagrams.
II. Where does biological knowledge live inside a model?
Transformers contain several plausible computational substrates: input embeddings, attention layers, the main hidden state flowing through the network, feed-forward blocks, and output heads. Current work suggests that different substrates can contain rather different forms of biological structure. Attention may reflect co-expression, while residual-stream geometry contains some information on gene regulations and sparse features recapitulate cellular biology.
6. Produce a complete substrate map
Systematically compare embeddings, attention, feed-forward blocks, residual streams, normalization layers, and output heads under the same tasks and controls (and for many models). This would tell us where each kind of biological information is stored and where it is actually transformed.
7. Explain layer-by-layer transformations
What changes as a gene, protein, or cell representation moves through the network? Early layers may retain local relationships, while deeper layers may compress them into broad categories. We need explicit computational accounts of these transformations rather than a sequence of layer-wise gene set enrichment plots.
8. Recover context-dependent gene representations
The same gene behaves differently in a stem cell, tumour cell, neuron, and macrophage. How does its internal representation change across contexts? Can we identify which dimensions remain stable and which encode tissue, disease, developmental stage, or environmental response?
9. Measure how distributed each concept is
Some biological concepts may correspond to one sparse feature. Others may require hundreds of features or a curved region of representation space. We need measures of distributedness that distinguish actually distributed computation from poor feature decomposition.
10. Separate stored knowledge from used knowledge
A model may contain a decodable representation of a pathway without using it for the current prediction. Causal interventions, mediation analysis, and task-conditioned circuit tracing should determine which stored biological facts participate in particular computations.
11. Track what fine-tuning changes
Fine-tuning may add new biological representations, repurpose existing ones, or merely alter the output head. Mapping feature and circuit changes before and after fine-tuning could reveal how disease classifiers, perturbation predictors, and generative models acquire specialised capabilities.
12. Watch biological knowledge emerge during training
Train small biological models while saving many intermediate checkpoints. When do pathways, cell identities, regulatory relationships, evolutionary structure, or dangerous capabilities first appear? Do they emerge gradually, suddenly, or in a predictable order?
13. Relate scale to extractable knowledge
Single-cell reconstruction models show useful scaling when data are abundant and little benefit from model growth when data are scarce. An open question is whether interpretable biological knowledge follows the same scaling laws as predictive loss, and whether larger models contain qualitatively new mechanisms or simply denser versions of old ones.
III. Gene regulatory network inference
A gene regulatory network is a graph of how regulators influence target genes. Attention matrices initially looked like convenient ready-made networks, but systematic tests found that attention often tracks co-expression or gene-level importance rather than unique causal edges. Residual geometry appears richer, yet real benchmarks which rely on wet lab gene perturbations are very noisy, and method rankings can change when the tissue, reference database, or candidate edge set changes.
14. Locate regulatory signal outside attention
Search systematically through residual streams, sparse features, feed-forward blocks, output gradients, and combinations of components. The main question is whether any internal substrate contains regulatory information beyond expression level, co-expression, and regulator popularity. In particular, my experiments with tensor networks demonstrated that quite a lot of regulatory information is contained in their FFN layers.
15. Recover causal direction
A model may place two related genes close together without representing which gene influences the other. Can internal interventions distinguish regulator-to-target relationships from target-to-regulator relationships, shared causes, and symmetric functional similarity?
16. Separate direct from indirect regulation
If regulator A affects gene C through intermediate gene B, a model may encode all three pairwise relationships. We need methods that recover the shortest plausible causal chain and distinguish direct molecular interactions from downstream consequences.
17. Infer activation and repression separately
Regulatory edges have signs. A transcription factor may activate one target and repress another, depending on context. Can a model’s internal computation reliably recover the direction of expression change after intervention rather than merely identifying related genes?
18. Build cell-state-specific networks
A single universal network is usually biologically unrealistic. Regulatory relationships change across tissues, cell types, developmental stages, disease states, and environmental conditions. Interpretability methods should recover conditional networks and explain which internal features switch particular edges on or off.
19. Recover network rewiring under perturbation
When a gene is knocked down, a drug is added, or a cell becomes diseased, the network itself may change. Can model internals show which connections disappear, which compensatory pathways activate, and which new dependencies emerge?
20. Add time to network extraction
Most single-cell datasets are static snapshots, like images. Regulatory mechanisms unfold over minutes, hours, and developmental timescales. Models trained on time-course data could be interpreted to recover delays, transient interactions, and the ordering of regulatory events.
21. Find feedback loops
Feedback is central to homeostasis, differentiation, oscillation, and disease. Standard pairwise scoring can miss feedback or confuse it with correlation. Can circuit tracing identify closed regulatory loops and determine whether they are stabilising, amplifying, or oscillatory?
22. Model combinatorial regulation
Genes are often controlled by combinations of transcription factors rather than one regulator at a time. We need methods for detecting AND-like, OR-like, threshold, cooperative, antagonistic, and context-dependent combinations inside model circuits.
23. Train and interpret perturbation-native models
Models trained mainly on observational expression snapshots have little incentive to learn causal direction. Train foundation models directly on large perturbation datasets, then ask whether their internals contain better regulatory networks than those of masked-expression models.
24. Fuse transcriptomic and regulatory modalities
Combine gene expression with chromatin accessibility, transcription-factor binding, protein abundance, methylation, spatial context, and perturbation outcomes. Interpretability could reveal which modality contributes evidence for each proposed edge in the gene regulatory network and where different modalities disagree.
25. Handle unmeasured variables
Many apparent gene-to-gene relationships are driven by missing proteins, metabolites, environmental signals, or hidden cell states. Can models represent latent causes explicitly? Can interpretability detect when a proposed regulatory edge is really a proxy for an unobserved variable?
26. Make benchmark rankings stable
Network-inference rankings can reverse when researchers change the candidate gene set, tissue, or reference database. We need to build evaluation suites that report performance across a distribution of reasonable protocols and expose methods whose apparent superiority depends on one convenient setup.
27. Use active learning to select perturbations
Given a partially inferred network, which next gene knockout, combination, dose, or time point would reduce uncertainty most? Interpretability-derived networks could become active experiment-design systems.
28. Export executable regulatory models
A static edge list is limited. Can we extract a compact dynamical system that predicts how gene expression evolves after interventions? Such a model might use equations, logical rules, or a small neural circuit and remain usable without the original foundation model.
IV. Causal circuits and internal computation
My circuit-tracing work has begun to map directed dependencies between sparse features in single-cell models. These maps show biological coherence, strong inhibitory structure, substantial redundancy, but only a weak bridge from model-internal causal edges to real regulatory causality.
29. Map complete circuits rather than selected examples
Researchers often choose a few features that already have clear annotations. Exhaustive tracing can reveal different global structure, including unannotated hubs. We need scalable methods that map all active components without requiring millions of prohibitively expensive interventions.
30. Translate feature circuits into biological variables
A directed graph between sparse features remains a graph between artificial objects. Develop principled ways to map those nodes and edges onto genes, pathways, molecular processes, cell states, and experimentally measurable quantities.
31. Characterise higher-order redundancy
Ablating one feature may have little effect because several other features perform overlapping functions. Pairwise tests are insufficient when three, ten, or a hundred components compensate for one another. We need efficient methods for finding minimal causal sets.
32. Detect synergy as well as redundancy
Some computations may appear only when several features act together. Develop tests that identify superadditive interactions without enumerating every possible feature combination.
33. Validate that interventions remain on-manifold
Zeroing or amplifying an activation can create hidden states that the model never encountered during training. Compare different interventions, including patching from real examples, projection onto valid activation regions, and constrained optimization, to determine which causal conclusions survive.
34. Explain inhibitory dominance
Many biological circuit maps contain many inhibitory feature-to-feature effects. Does this reflect biological suppression, competition between representations, normalization, sparse coding artifacts, or the model’s way of removing irrelevant information?
35. Recover context-conditioned circuits
The circuit used for a gene in one cell type may differ sharply from the circuit used in another. Build methods that return conditional circuits and identify stable cores, context-specific branches, and switches between alternative computations.
36. Measure circuit completeness and faithfulness
A proposed circuit should reproduce the model’s behaviour when run independently or when substituted into the original model. Develop tests for how much performance, calibration, and robustness the circuit preserves across inputs.
37. Edit circuits with local side effects
Can we remove a technical artifact, disease-associated shortcut, or unsafe capability while preserving unrelated functions? Circuit editing should report both the intended change and a broad audit of collateral damage.
38. Find universal biological circuits
Do independently trained models converge on equivalent circuits for translation, secretion, immune activation, differentiation, or DNA repair? Cross-model convergence would provide evidence that a circuit reflects durable structure in the data-generating process.
V. Sparse features and representation geometry
Sparse autoencoders have exposed interpretable features in protein, single-cell, microscopy, pathology, and physiological models. Their outputs are still method-dependent: two autoencoders trained on the same activations can recover different feature sets, and different dictionary sizes can split or merge concepts.
39. Make sparse features stable across random seeds
Measure which features recur across independent training runs and which are arbitrary decompositions. Develop alignment methods, consensus dictionaries, or objectives that reward stable recovery without forcing trully ambiguous concepts into artificial agreement.
40. Understand feature splitting and merging
A broad immune-response feature at one dictionary size may become several cytokine, interferon, and stress features at another. We need a hierarchical account of how features divide, combine, and relate across levels of resolution.
41. Align features across models
Create methods for matching functionally equivalent features between different architectures, model sizes, training datasets, and modalities. A shared feature may be rotated, split, distributed, or implemented through a different circuit, so simple vector similarity will often fail.
42. Recover rare biological concepts
Rare cell states, transient developmental events, unusual pathogens, and uncommon diseases may contribute little to an average reconstruction objective. Train specialised dictionaries or sampling schemes that recover rare features without hallucinating structure from a handful of examples.
43. Interpret negative and absence-based features
A feature may represent the absence of a pathway, loss of differentiation, missing structural elements, or suppression of a response. These concepts are harder to annotate because biological databases mainly record positive associations.
44. Learn nonlinear and multiscale features
Standard sparse autoencoders represent activations as linear combinations of features. Biology contains spatial patterns, sequence motifs, interactions, trajectories, and nested hierarchies. Develop feature models that preserve this richer structure while remaining causally testable. I tested bilinear autoencoders, without much success so far.
45. Connect sparse features to global geometry
Sparse features describe recurring directions, while manifold methods describe the shape of whole representation spaces. Determine when these views agree, when they capture different information, and whether sparse features can serve as coordinates on biological manifolds.
46. Make topological findings protocol-robust
Clusters, branches, loops, and curved manifolds can change with preprocessing, distance metric, sample density, and dimensionality reduction. A topological claim should survive plausible analysis choices, matched null models, and external datasets.
47. Separate sequence semantics from cellular semantics
A single-cell model may appear to know gene function because its gene embeddings inherit information that a protein-sequence model could already supply. Every claim about cell-derived knowledge should compare against sequence-only, expression-only, and annotation-only baselines.
48. Interpret genuinely unknown features
Large language models can generate plausible names for almost any gene list. Build annotation pipelines that show evidence, counterevidence, nearest known concepts, causal behaviour, and experimental tests, while allowing the label “currently unknown.”
VI. Extracting standalone biological algorithms
In one my study, I report a compact algorithm representing blood cell development extracted from scGPT through frozen attention operators. The extracted algorithm runs 34.5 times faster than a standard multilayer probe, uses roughly 1,000 times fewer trainable parameters, and retains interpretable cell lineage factors. This is a promising demonstration, and independent replication is now very valuable.
49. Independently replicate the hematopoietic result
Apply the same pipeline to independently processed datasets, alternative checkpoints, other single cell foundation models, and strict donor-held-out evaluations. Replication would establish whether the method is general or unusually well matched to one model and developmental system.
50. Extract algorithms for more bioinformatics tasks
Candidate targets include cell typing, developmental ordering, gene-module scoring, cell-cell communication, variant-effect prediction, protein-function annotation, binding prediction, sequence alignment, structural motif detection, and spatial-neighbourhood analysis.
51. Export native operators directly
Search for attention heads, convolutional filters, state-space operators, graph-message-passing rules, or small feature circuits that can be copied out of a model with little or no retraining. Direct export gives especially strong evidence that the computation was already present.
52. Translate neural computations into equations or rules
Can an internal circuit be expressed as a scoring function, differential equation, logical program, graph algorithm, or short piece of conventional code? Symbolic extraction would make biological assumptions easier to inspect and modify.
53. Eliminate target-dataset retraining
A compelling extracted algorithm should work on new data using fixed parameters or minimal calibration. Otherwise, performance may come from the newly trained adaptor rather than from knowledge recovered from the foundation model.
54. Test extreme distribution shifts
Evaluate extracted algorithms across donors, laboratories, platforms, tissues, diseases, species, and evolutionary distances. The most interesting result would be an algorithm that transfers further than the original model’s usual downstream pipelines.
55. Compare extraction against purpose-built methods
For each task, compare predictive performance, calibration, speed, memory, data requirements, interpretability, and maintenance cost. A compact extracted algorithm should earn its place against mature specialist bioinformatics tools.
56. Compose extracted algorithms
Can separate exported modules for cell identity, developmental state, signalling, and perturbation response be joined into a larger transparent system? Composition may reveal whether the original model contains reusable biological subroutines.
57. Preserve uncertainty and safety during extraction
Compact tools can make capabilities cheaper and easier to distribute. Algorithm extraction should include a review of whether compression amplifies access to potentially dangerous biological capabilities.
VII. Novel biological discovery and laboratory validation
Interpretable features have already been proposed for filling missing protein annotations, identifying model-driving biomarkers, refining networks, mapping cell states, and guiding experimental design. The decisive next step is prospective validation in experiments selected before the result is known.
58. Discover biomarkers that survive model removal
Use interpretability to identify a compact set of measurable variables that are associated with some condition (like disease or age), then build a conventional statistical model directly from raw biological data. Validation should occur in an external cohort with no dependency on the original foundation model.
59. Discover previously unrecognised cell states
Find internal features or geometric regions that identify reproducible cell populations absent from existing taxonomies. Require coherent molecular programmes, recurrence across donors, and confirmation through independent measurements or functional experiments.
60. Find hidden transition states
Developmental and disease processes may pass through brief intermediate states that standard clustering misses. Representation geometry could identify bottlenecks, branches, or sparse regions and suggest markers for experimentally isolating those cells.
61. Investigate unannotated circuit hubs
Exhaustive circuit mapping can reveal highly connected features with no clear database annotation. These hubs are promising discovery targets, although they may also encode technical properties of datasets. Characterise them through targeted perturbations and independent datasets.
62. Map conserved and divergent mechanisms
Compare equivalent internal circuits across tissues and species. Stable cores may reveal conserved biological programmes, while specific points of divergence may identify tissue-specific or lineage-specific regulatory innovations.
63. Prioritise drug targets through selective circuit control
Identify perturbations that suppress disease-associated features while preserving healthy-state features. Of course, experimental validation should test both efficacy and specificity, since “strongly changes the model” is a rather adventurous drug-development criterion.
64. Discover side-effect and toxicity programmes
Trace how candidate interventions activate stress, immune, metabolic, or cell-death circuits. This could expose likely liabilities before expensive animal studies, provided the predicted programmes transfer to real experimental systems.
65. Design experiments from representation geometry
Use uncertainty, curvature, sparse regions, and disagreements between models to choose new samples or interventions. Compare this strategy against random selection, expert selection, diversity sampling, and ordinary active learning.
66. Automate discovery without automating self-deception
Agentic systems could search thousands of feature, pathway, and circuit hypotheses. They also create a powerful multiple-testing machine. Automated pipelines need held-out data, explicit null models, correction for repeated search, preregistered stopping rules, and independent reviewer agents.
VIII. Training better models for mechanistic biology
Interpretability sometimes reveals that some models never learned the desired mechanism. In that case, increasingly elaborate microscopes will not manufacture a missing object. Model objectives, data, and architectures may need to change.
67. Pretrain directly on interventions
Include gene knockouts, gene activation, drug treatments, environmental changes, and combinations during pretraining. Then test whether causal direction and regulatory effects become easier to recover from internal representations.
68. Use counterfactual training objectives
Ask models to predict how the same biological system would behave under alternative interventions. Counterfactual objectives may encourage representations of mechanisms that remain stable across changing conditions.
69. Train on time and lineage information
Add time courses, lineage tracing, developmental ancestry, and repeated measurements. These data may help models distinguish causes, consequences, transient responses, and stable cell identities.
70. Build multimodal models
Jointly model DNA, chromatin, RNA, protein abundance, metabolites, morphology, spatial position, and perturbations. Interpretability should reveal which modalities contribute unique information and where the model learns cross-modal causal links. I am training such models now, so you can join!
71. Add causal inductive biases
Explore directed graph modules, explicit intervention variables, structural causal models, dynamical systems, and asymmetric message passing. Compare these designs against ordinary transformers under equal data and compute.
72. Make models interpretable by construction
Use modular pathways, sparse bottlenecks, hierarchical representations, explicit gene sets, and constrained interactions during training. The key question is how much interpretability can be gained without forcing the model’s performance to deteriorate.
73. Compare architecture families mechanistically
Transformers, graph neural networks, state-space models, diffusion models, variational autoencoders, and neural differential equations may learn different biological abstractions. Evaluate both predictive performance and the recoverability of causal mechanisms.
74. Remove technical variation during training
Develop objectives that suppress batch, laboratory, platform, and donor artifacts while preserving real biological heterogeneity. Interpretability can then verify whether deconfounding succeeded or merely hid the artifact in a less obvious subspace.
75. Derive scaling laws for causal knowledge
Measure how regulatory direction, perturbation transfer, circuit coherence, algorithm extractability, and discovery rates change with model size and data diversity. Predictive loss may improve smoothly while causal knowledge remains flat, or vice versa.
76. Use synthetic causal curricula
Generate artificial cells, gene networks, proteins, or evolutionary systems with exactly known mechanisms. Train models from simple to complex systems and test whether interpretable representations track the known causal structure.
IX. Replication, transfer, and benchmark design
Replication has special importance here because model training, feature decomposition, biological preprocessing, and reference databases all introduce degrees of freedom. Network-method rankings have already shown meaningful sensitivity to evaluation choices.
77. Replicate across random seeds
Repeat model training, feature extraction, circuit tracing, and downstream evaluation across seeds. Report which conclusions remain stable even when individual features rotate, split, or disappear.
78. Replicate across checkpoints and model sizes
A mechanism found in one checkpoint may be transient or scale-specific. Track equivalent features and circuits across small, medium, and large versions of the same model family, like for example for Geneformer.
79. Replicate across architectures
Test whether findings from a transformer survive in graph, state-space, convolutional, or variational models trained on comparable data. Convergence across architectures is stronger evidence than recurrence within one model family.
80. Replicate across datasets, laboratories, and platforms
Run the same analysis on data from different sequencing technologies, sample-processing protocols, hospitals, and laboratories.
81. Replicate across donors, tissues, diseases, and species
State the boundary conditions of every mechanism. Some circuits may be universal, some human-specific, some tissue-specific, and some artifacts of one immortalised cell line.
82. Replicate across intervention types
Compare gene knockout, partial knockdown, overexpression, chemical inhibition, dosage variation, and environmental changes.
83. Standardise leakage-resistant evaluation
Biological data contain near-duplicates, related donors, homologous sequences, shared studies, and annotation leakage. Benchmarks should split data according to the scientific generalisation claim, including donor, laboratory, tissue, family, or evolutionary-distance splits.
84. Publish negative replications and versioned atlases
Feature atlases should record model versions, data versions, autoencoder settings, annotation databases, and confidence. Failed replications should be searchable alongside successes.
X. Beyond ordinary single-cell gene expression
I focus mostly on single cell models, mostly because I believe there is much more knowledge to extract from them. But mechanistic interpretability is already spreading across proteins, microscopy, pathology, spatial biology, and physiological signals. Each modality offers different ground truth and different opportunities for scientific discovery.
85. Recover protein mechanisms from sequence models
Move from features correlated with known annotations to circuits that compute binding, catalysis, structural stability, localization, or allosteric effects. Validate proposed mechanisms using mutational scans and biochemical assays.
86. Interpret protein-generation decisions
Which internal features cause a generative protein model to favour a fold, active site, binding interface, or taxonomic family? Can steering alter one property while preserving stability and unrelated functions?
87. Decode genomic regulatory grammar
Genome models may learn promoter structure, enhancers, splice signals, chromatin boundaries, long-range interactions, and evolutionary constraints. Identify the internal algorithms that combine motifs over thousands or millions of bases.
88. Interpret RNA foundation models
RNA models combine sequence, secondary structure, modification, localization, expression, and interaction information. Open problems include recovering folding rules, binding programmes, splicing mechanisms, degradation signals, and context-dependent regulatory elements.
89. Map spatial tissue circuits
Spatial models can represent which cells influence their neighbours and how tissue architecture constrains signalling. Interpret them to recover local communication rules, niches, boundaries, and disease-associated neighbourhoods.
90. Extract morphological programmes from microscopy models
Sparse dictionary methods can already recover cell types and perturbation-related morphology. The next step is to identify causal visual programmes, connect them to molecular pathways, and design experiments around previously unnamed morphological states.
91. Interpret structural and interaction models
Study how protein-structure and molecular-interaction models represent geometry, physical constraints, confidence, alternative conformations, and binding. This may reveal computational approximations that can be exported into faster structural algorithms.
92. Trace cross-modal circuits in multi-omics models
When a model predicts gene expression from chromatin or proteins from RNA, which internal pathways carry the information? Cross-modal circuit tracing could expose real biological coupling as well as shortcut mappings.
93. Extract signalling, metabolic, and whole-cell computations
Extend interpretability to models of signalling networks, metabolic flux, cellular simulations, and virtual cells. The long-term target is a comprehensive hierarchy connecting molecular events to cell behaviour.
XI. Biosecurity and safety auditing
My take is that you can’t do security audit of bio models without mechinterp. Genome language models have now been used to generate complete viable bacteriophage genomes, and a separate work found that fine-tuning could partially restore virus-related capabilities in a model whose original training data excluded certain viral sequences.
94. Define risk-relevant internal capabilities
Specify which internal abilities matter for biosecurity: for example, they may relate to understanding host range, immune interaction, functional sequence constraints, molecular evasion, or whole-genome coherence.
95. Locate risk-relevant features and circuits
Can interpretability identify internal representations associated with hazardous biological functions before researchers rely on broad output evaluations? Such features would need causal validation and careful access controls.
96. Detect pre-capability indicators
A model may develop partial components of a capability before succeeding on an end-to-end benchmark. Track precursor features, reusable subcircuits, and sudden changes in representation geometry across training checkpoints.
97. Test the robustness of training-data exclusion
When sensitive categories are removed from pretraining, determine which related abstractions remain. Models may reconstruct missing capabilities from neighbouring biological domains, evolutionary regularities, or later fine-tuning.
98. Monitor capability restoration during fine-tuning
Interpretability could show which dormant representations become reactivated or recombined during adaptation. This may support earlier detection than waiting for a model to succeed on a complete misuse-relevant task.
99. Develop mechanistically verified unlearning
After removing sensitive data or capabilities, test whether the relevant circuits have actually changed. Output suppression alone may leave recoverable internal knowledge intact or move it into a different representation.
100. Red-team interpretability-based safeguards
Assume that an adversary knows the auditing method. Test whether fine-tuning, representation rotation, feature splitting, model merging, or alternative prompting can bypass feature-level monitors.
101. Use interpretability to improve sequence screening
Internal representations may help identify functional similarity even when raw sequence similarity is low. Evaluate whether interpretable features can complement existing screening systems without producing unacceptable false-positive rates.
XII. Biology as a testbed for general mechanistic interpretability
Biological models offer unusually rich ground truth: known pathways, protein structures, evolutionary relationships, perturbation experiments, lineage information, and measurable phenotypes. They are also often smaller than frontier language models. This makes them useful model (but also real) organisms for mechanistic interpretability itself, provided biological databases are not mistaken for perfect ground truth.
102. Create “model organisms” for mechanistic interpretability
Select a small set of biological models, datasets, and tasks for repeated, exhaustive study. The equivalent of E. coli or Drosophila for interpretability would let methods accumulate on shared systems rather than restarting from zero in every paper.
103. Build synthetic biology tasks with exact ground truth
Generate data from known regulatory networks, evolutionary processes, spatial systems, and biochemical simulators. Then test whether interpretability methods recover the exact mechanisms used to generate both the data and the model’s predictions.
104. Compare interpretability methods head-to-head
Run attention analysis, probing, gradients, sparse autoencoders, transcoders, causal tracing, causal abstraction, geometry, and circuit discovery on identical models. Evaluate accuracy against shared causal and biological targets.
105. Test methods under biological redundancy
Biological systems contain duplicated genes, parallel pathways, compensation, feedback, and many-to-one mappings. These properties create a demanding benchmark for whether interpretability can recover distributed and degenerate computations.
106. Exhaustively analyse very small biological models
Train small transformers or graph models on controlled biological tasks and map every relevant neuron, feature, edge, and intervention. Complete explanations on small systems can test assumptions before scaling methods to billion-parameter models.
107. Release shared activation and intervention datasets
Store model activations, sparse features, patching results, perturbation outcomes, and annotation mappings in standard formats. This would let researchers compare methods without repeatedly paying the full inference cost.
108. Build agentic research loops with independent quality gates
Automate hypothesis generation, analysis, replication, and report production while separating executor, critic, statistician, and biological-review roles. Hold-out datasets and preregistered tests should remain inaccessible to the hypothesis-generating agent until evaluation.
109. Just apply modern mechinterp techniques to bio models and see if the results are better
Manifolds, j-space, natural language autoencoders, compositional methods whatever—it is (usually) easier to test it on bio models as compared to LLMs.
If you start working on it, please get in touch with me to coordinate!