01 / The open toolkit
Tools for your research.
The transformer model, language, datasets, gridworld, harness, guides, and documentation are available for independent experiments and extensions.
Explore the toolkitIndependent research / AI systems
Small models.
Clear questions.
A small independent research lab studying how AI systems work and what it takes to make them more capable.
Micro-Transformer is both an open-source toolkit and an ongoing research project. Others can use the tools independently; the Lab uses them to develop and test new capabilities, with a longer-term direction towards much larger models, autonomy, and selfhood.
Build / Inspect / Test / Understand
small models
clear ideas
real progress
clear ideas
real progress
inspect
test
refine
test
refine
How the work fits together
One project. Three connected parts.
The toolkit is available for others to use. The Lab’s own research builds on it. The wider programme takes those questions beyond the original model.
02 / The research project
A system to develop and test.
The Lab uses Micro-Transformer to investigate existing capabilities, develop new ones, and test what each change contributes against controlled baselines.
Explore the research03 / The wider direction
Larger models. Deeper questions.
Extend the investigations to much larger models and explore autonomy, selfhood, and mechanisms that might support Autonoetic Selfhood.
Explore the directionThe Research Lab
Questions worth testing.
Within the Lab, Micro-Transformer is a system being investigated as well as a tool for investigation. The research develops, extends, and tests capabilities such as learning, memory, prediction, and coordination through controlled comparisons. Later work will investigate these mechanisms in much larger models and test whether the findings hold at scale.
These questions describe the research direction. They are not claims of demonstrated capabilities.
01 / Understanding the model
What has it learned?
When a model gets an answer right, what supports that answer—and where does its apparent understanding break down?
- Use tasks with answers that can be checked independently.
- Compare familiar examples with cases that require a different combination of known elements.
- Inspect failures as closely as successful answers.
A correct prediction is a starting point for investigation.
02 / Building capability
What makes a difference?
What would memory, prediction, or coordination contribute to a system’s ability to carry out a defined task?
- Test a specific change against a matched baseline.
- Separate the effect of a mechanism from extra computation or information.
- Keep the task and success criteria fixed through the comparison.
Start with a question narrow enough to answer, then investigate whether the findings hold in larger models.
03 / Connecting mechanisms and minds
What would count as evidence?
How can ideas about cognition and machine minds lead to predictions that distinguish competing explanations?
- State what a proposed mechanism should change.
- Consider simpler explanations for the same behaviour.
- Distinguish task performance from evidence of a persistent, usable self-model.
This question guides the investigation of autonomy and Autonoetic Selfhood.
The working method
Ask a clear question. Make a controlled comparison. Record what happened and what remains unresolved. Use those findings to guide further research with larger models.
The Micro-Transformer toolkit
The tools behind the questions.
The open-source toolkit makes the transformer model, formal language, data generation, Micro-World gridworld, harness, and documentation available for others to run their own research. Researchers, learners, and developers can use and extend these components independently of the Lab’s research programme.
The model, formal language, data generation, harness, and Micro-World grid simulation have separate responsibilities. Experiments can connect them through explicit adapters; the baseline transformer does not automatically act as an agent in the gridworld.
Explore the toolkit components
01
Transformer
A small causal transformer trained from scratch, with architecture and inference code you can inspect and change.
Explore the model02
Language & data
A readable formal language, reference interpreter, and procedural datasets with answers that can be checked.
Explore the language03
Micro-World
A separate gridworld with observations, actions, and replay. Connect a model through an adapter defined by your experiment.
Explore Micro-World04
Harness & guides
Run the frozen checkpoint, inspect example prompts and known failures, and follow the guides to build your own experiments.
Explore the harness
Start with the quickstart
Open Playground
View the source on GitHub
For learners, researchers, and developers who want a small system they can run locally, inspect, and extend.
About the Research Lab
Small enough to inspect. Built to investigate.
Micro-Transformer Research Lab is a small independent lab with three connected parts: an open toolkit for others to use, an ongoing research project developing and testing capabilities in Micro-Transformer, and a wider investigation into larger models, autonomy, and selfhood.
The approach starts with systems small enough to inspect, change, and test, then scales to much larger models for further research. Findings from small systems guide those investigations; whether they hold at scale must be tested.
The longer-term research explores Autonoetic Selfhood: how a system might connect memories of its past, its present state, and possible futures to an ongoing model of itself. The Lab investigates what memory, prediction, self-modelling, and autonomy could contribute. This remains a research aim; fluent self-descriptions alone are insufficient evidence.
Essays and commentary develop and question the ideas behind the work. These are distinct from experiment reports, which document methods, observations, limitations, and alternative explanations.
Read on Substack