My literature review, agentically
Building a map of continual-learning ideas with Claude Code and Obsidian, Timothée Lesort
Why?
In a matter of months, coding agents went from autocomplete curiosities to collaborators able to navigate, modify and test entire codebases. Software engineers were the first to feel that shift, but it goes well beyond software: the same tools are changing how researchers search, read, connect and synthesize information — and with it, the very way a scientific field can be explored and understood.
At the imec AI.labs, alongside our work on AI research itself, we try to understand which new methodologies these systems make possible, and how they can help us do better research, faster. It is not only a question of productivity: if the way we access and organize knowledge changes, the way we think about our research problems may change too.
This post is a field report on one such experiment: a simple bibliography system, built to assess how well agentic systems can help a researcher get a grip on a specific research domain, here continual learning. Beyond the tool itself, the goal is to share what worked, what did not, and what this small project suggests about the research-oriented knowledge systems we could build in the future.
Literature Review
A large part of research time goes into reading scientific papers, taking notes, extracting ideas, organizing and reorganizing them. This tedious work helps to build a mental map of the field, and gives us basic bricks to manipulate concepts, be innovative, and share what we know. Unfortunately, it takes a lot of time and iteration: a paper often has to be read and analyzed several times before it is correctly processed and positioned in the literature.
As a researcher, I tried multiple times, during my PhD, my postdoc and later, to build a consistent set of notes gathering ideas and summaries. It never ended up anywhere satisfying. Papers usually cover multiple ideas and concepts; I never found a consistent way to connect them, and I kept changing my mind about how to organize the notes. Or I would realize they did not contain all the information I wanted, and end up re-reading the papers to check against the ground truth rather than my (laborious) notes. In the end I was writing things down just to memorize, and had given up on the big knowledge base that would always be too messy. This digging again and again through the literature was particularly tedious when writing surveys: remembering which paper does what always came down to eyeballing PDFs by hand.
With LLMs and agents, analyzing and summarizing papers became almost instantaneous, and organizing notes became easy: you want your files organized by topic rather than by author? Just ask. You do not remember which article talked about concept Y? Point the agent at your folder of PDFs and ask.
When I saw Karpathy's LLM wiki post, a second brain built as an Obsidian vault (i.e. a collection of notes linked together) maintained by an agent, I thought I could reuse this system for my research literature map. So I followed a similar path with Claude (Fable 5) in Claude Code and Obsidian, to extract the ideas from papers and organize the literature of my research field: continual learning.
Building a Research Obsidian Vault with Fable 5
I created a PROJECT.md file explaining my goal and scope: a map of the ideas and opinions in continual learning papers. Then I seeded the system by asking my agent to go through recent major conferences (NeurIPS, ICLR, ICML) to find continual learning papers, ingest them, and then continue with the referenced papers. An ingestion is the processing of one paper by the agent: it downloads the PDF, extracts the text, summarizes it in a paper note, then extracts ideas to either connect the paper to existing ideas or create new ones. I also ingested the papers I knew from continual_learning_papers, the repository I co-maintained. As of October 2026, the vault holds 549 papers from various venues, universities and author communities, creating a large graph of ideas, authors and papers.
What makes this more than a pile of summaries is how a paper connects to an idea. Each link is typed: a paper argues for an idea, argues against it, nuances it, or merely tests it without taking a side. And each link carries the experimental conditions under which the paper makes its claim (class-incremental or task-incremental, from scratch or pretrained, how much replay, which benchmark), because in continual learning results flip from one setting to another: "replay is necessary" holds in some scenarios and not in others. The other rule is that the vault describes and never judges: it records who says what, under which conditions, and never decides who is right. If the evidence is lopsided, the tables show it, and the reader draws the conclusion.
I was careful not to ingest the whole literature in a single pass, so I could check that the system was creating links between ideas and taking reasonable notes. So I mostly ran waves of 10-15 papers at a time, then queried the system, looked at the summaries of papers I already knew, and plotted some figures to check whether the data was usable. I had some hallucination issues in the beginning: when the agent could not find a paper, it would write notes from memory. After tuning the "analyze- paper" skill of my agent, the system ran smoothly: a paper without an accessible full text now gets no note at all, and each wave ends with a spot-check of a sample of the agent's claims against the source PDF. The vault can be visualized as a graph with Obsidian or a specialized plugin, but I found it much more convenient to just ask Claude to use the data to create HTML pages with plots, for example to map author opinions to ideas or method connections (see the carousel below). I found the results visually good, inspiring and convenient to explore thanks to interactive plots. It is also powerful to just ask questions like "what are the recent papers on topic X" and see a detailed answer with references coming from the vault. As for coverage: 80% of the "builds on" links, from a paper to the earlier work it extends, point to papers that are themselves in the vault, so the corpus captures most of the lineage its papers rely on. With one month of a Claude Max subscription, I achieved what would have taken me more than a year of consistent note-writing.
Using and Steering the System
I was surprised by the number of ideas extracted: 683 for 549 papers, more than one per paper, and more than half of them (359) are engaged by a single paper. Originally, I deliberately did not push it too much on merging ideas, but I think there is room for improvement on that end, at least to get better modularity in ideas and avoid entangling concepts and subjects. For example, an idea like "regularization works in the domain-incremental vision setup" should be disentangled into an approach, a scenario and a data type. The vault's rules already say that the setting belongs in the condition attached to a link, not in the idea itself; the agents just did not apply that rule consistently.
Another interesting outcome is about the edges between papers and ideas: 1537 are labelled argues for, 465 nuances, and only 98 argues against. Is the field that consensual? Partly, but the ratio also reflects how the map was built. Every idea enters the vault through a paper that proposes it, so support is recorded by construction. Opposition only appears if the agent reads a paper's related-work section looking for what it disputes, and papers rarely state their disagreements straightforwardly. So the ratio says as much about the extraction as about the field. The most debated questions are "replay is necessary for class incremental learning" and "protecting important weights prevents forgetting", which are in my opinion two sides of the same coin: both address how to learn features that discriminate between temporally distant data. Whether those features can be learned without replay is the whole point of both debates. A paper arguing that replay is necessary in class-incremental learning is implicitly arguing that protecting weights is not enough, and the other way around.
Overall, while I found the system really useful for digging into the literature and the community and generating high-level overviews, I do not think it can be built without supervision: it needs someone to drive the vision, steer the system and arbitrate the granularity of ideas. Consequently, I would not trust a survey entirely written from these notes, and if I had to write a new one I would probably still do it by hand, assisted by this system. The prose written automatically often feels to me like an imitation of writing rather than real writing; it tends to sound empty and without opinion.
Also I do not fully trust individual paper notes in the vault: I believe they carry sufficient information to build a relevant map of the field, but they can lack precision, and they do not replace eventually looking at the original paper. One trick that makes checking Claude's answers against the original paper much faster: have Claude open the source PDF with the relevant passages highlighted. It is quite efficient to verify and understand new concepts or ideas within their original context. And the whole system is highly adaptive, so anyone can tune it to their own field and preferences, continue the fully agentic approach, or mix manual and agentic approaches.
Am I only superficially efficient with my agents?
Since memorizing the literature usually goes through the painful process of note-taking, a difficult question arises: will I stop understanding papers because I no longer write my notes? Am I only superficially efficient? First, I am convinced that not all the manual labor of writing was useful for memorizing. In the pre-agentic world, once a concept is correctly memorized, there is still work left to finish the notes, improve the template, rework them, reorganize them. Second, I distrust the feeling of having understood something, so I keep returning to the paper and discussing it with the agent until I am sure I get it. I just make note-taking faster, and the querying system gets me simple answers faster. Third, I think my understanding will improve beyond the simple faster-answer system, through cross-cutting analysis. It becomes trivial to plot a graph of who works on one topic, at which university, who collaborates with whom, so I gain field understanding in addition to paper understanding. The final vault represents the field — its ideas, its trends, its connectivity — and can be used to investigate it. My feeling is that this is really powerful for processing and producing original and creative ideas in research. The results have never been the notes themselves, but what we get out of them.
Next Steps
Developing this system made one thing crystal clear: agentic AI can help a lot in collecting, aggregating and delivering complex knowledge. It offers a new way to enlarge the memory of researchers, and to support them in being more knowledgeable about their field, its ideas, and its people.
This methodology is not limited to the scientific literature. It can be extended to the internal knowledge of companies and research institutes. Over the decades, such organizations accumulate an enormous amount of internal knowledge (reports, experiments, designs, presentations, discussions) that is rarely fully exploited. Building such systems on top of it could turn researchers into "super researchers", backed by an extended memory of everything their organization has learned.
One main question remains, though: does it allow researchers to be more creative? Or, on the contrary, will it create some kind of self-censorship? Doing research is also, sometimes, about naively redoing what has already been done in the past, but in the light of the current state of the art. An idea that failed ten years ago may succeed today with new tools, new data or new insights. If an extended memory always reminds us that "this was already tried", we may stop exploring paths that are worth revisiting. Understanding how to design these systems so that they feed creativity rather than constrain it is, in our view, one of the most interesting open questions for the future of research-oriented knowledge systems.