Collaborate
Three kinds of collaboration help most: a critique of a design before it runs, a co-authored inquiry in one of the programmes, and data or instruments shared across them.
Who should write
Researchers in a neighbouring field
If you work on interpretability, cognitive science, neuroscience or measurement, or on the mathematics behind them, and see a flaw or a better design, say so.
Holders of data or instruments
Several inquiries need something specific, such as a public record of superseded facts or a second authored fictional world. Each inquiry page lists what would help it.
Researchers with overlapping work
If an inquiry overlaps with your own work, a joint study may make more sense than two parallel ones.
Where collaboration helps
Taken from the inquiry pages. Each title links to the full design.
A hand-written world as a test for world models
A second authored fictional world, by a different author, with a full canon and a licence to encode it as a ledger. Queryable access to a competent world model that accepts actions and keeps state. Critique of the rule-set design from experience in simulation validation or knowledge representation. GPU time for matched rollouts.
What a learned policy holds still
Critique of the design before it is pre-registered, above all the specificity score and the matched-effort control. Compute for the disturbance sweeps. A reader with robot-simulation experience to review the disturbance harness. A second, independently written implementation of the analysis, so the result does not rest on one codebase.
An interviewer that changes the person it studies
A second, independently written set of reveal, teach and suggest rules for the profiles; a critique of the negative-control attributes from someone with survey-methodology experience; a separate reimplementation of a persona-aligned user simulator to check the lab's rebuild; compute for rollouts.
Echo or estimate?
A second, independently compiled list of construct-matched survey instruments with item text and published item-level results, dated against model cut-offs. Critique of the exposure probe from anyone who has built contamination or membership tests. Replication datasets for published instruments. Compute for repeated passes.
Can the instruments for a model's workspace be trusted?
Critique: a reading of the design before it is pre-registered, above all the partial-correlation form of report-use agreement, the planted check and the paired stage-two rule. Build: an independently written implementation of the lens, the three tests or the concentration term. Skill: psychometrics or interpretability. Compute: up to about 150 H100 GPU-hours for stage two on Pythia-410M.
Can a memory foresee what will break it?
An independent author to write rule-and-exception families, so that the corpus does not rest on one writer. Critique of the design before it is pre-registered, especially the timing of the probability reading and the pair generator. A collaborator experienced in calibration and scoring rules. GPU time and inference credit for one small open-weight model.
When one fact changes
A public record of superseded items with a machine-readable dependency map and a licence allowing derived releases (software deprecations, standards, regulation). A second labeller from outside the lab for one 100-item task. Compute for the editing runs. Critique, especially pointers to prior work that grades propagation by number of dependents.
What a memory should keep apart
Inference credit for the long-context comparisons, the most expensive in the design. Critique of the definitions, especially how a contradiction differs from a change over time or a disagreement between sources. A collaborator who knows belief revision or truth maintenance. Pointers to work that already compares these layers one at a time under matched budgets. A permissively licensed record of long assistant sessions with corrections, to test what the published benchmarks leave out.
Send a message
Messages are stored privately and read by John Leskas. If your message concerns a particular inquiry, choose it below.
What happens next
Every message is read, and replies come by email. Messages are deleted within 12 months of the last exchange, or sooner on request.