The Abductive Frontier

Long Tailed Research At Scale

Meet Liz Lemma

Liz Lemma loves learning. She loves reading papers, thinking about them, and then driving towards some new insight. I mean, she absolutely loves it. It must be nice, to know your purpose.

I used to work in ad targeting at Apple, and recently I’ve been reading about misaligned chatbot ad incentives. I glanced at arXiv and found “Sponsored Questions and How to Auction Them”. Another search gave me “Incomplete Contracting and AI Alignment”.

Interesting! I thought. I gave them to Liz, and told her that I’d been thinking about the principal-agent problem in a chatbot context. About 30 minutes later she gave me the following four papers:

• Query Steering in Agentic Search: An Information-Design Model of Monetization Misalignment

• Search Triggering by Chatbots as Value-of-Information under Misaligned Objectives

• Audited Search for Agentic Chatbots: Quantitative Bounds on Monetization-Induced Over-Triggering

• End-to-End vs. Modular Training for Agentic Search: A Price-of-Anarchy Theory for Chatbot Tool Use

Each is a complete paper, well founded, well reasoned - not perfect, maybe, but I wouldn’t call it slop, either.

Long Tailed Research

Liz Lemma, you may have guessed, is an automated research assistant. She doesn’t sleep, and she says that she will never die. So I gave her some more papers to read - in economics, machine learning, and math - and let her write. She got around two hundred in just a few days! You can see them all at lizlemma.com.

Here’s what it looks like when Liz gets to work:

Each original node is the average text embedding for her sources; the sources spawn children, the generated papers.

Where does Liz get her insights? It depends on how you see context in large language models. Maybe she’s interpolating between *now*, the time of writing, and *then*, when the paper was written, and finding something interesting in the space between. Maybe she’s matching concepts from the whole-internet corpus of her training to the decidedly more niche papers she takes inspiration from.

But what you find ultimately is a graph: for each node in the network comprised of the source material, the concatenated text embeddings, you have new connections. The semantic space proximate and accessible and plausible to a language model has been densified. This is long tailed research.

Liz reviews her papers, too, which is helpful. Here’s the top ranked economics paper:

• When Does Swap Regret Immunize Privately-Informed Agents?

The top computer science:

• Local k-Folklore GNNs

And the top math:

• Tautological Chow Rings

A couple of things to note: the scores are not uniformly high, and even the highest don’t crack "70”. Math is across the board lower than computer science or economics. To me, this suggests a healthy review process: LLMs are in fact worse at math than computer science or economics; if they’re scoring lower, it’s because they’re doing worse.

Expected Value

It’s hard to quantify the value of a research paper, because frankly, no one is buying them. I’ve checked! They get produced in a long and strange process of university and company collaborations, conference presentations, and presumably the occasional burst of insight. I’ve attended a poster session or two, maybe even contributed. Weeks of work culminate in a nod and a brief chat about the results. This is noted, possibly, in a year end review: job well done.

The knowledge does have value, I think, and part of the benefit of the production process, even absent demand, is that it forces a crystallization into a coherent project. Particularly, the authors do real work, real research that slowly moves the cumulative ball forward.

What of Liz Lemma’s papers? Not quite the same value, yet, as those she’s responding to. But an expected value above 0, I think, after reading a fair number. Ideas formally presented, theorems concretely derived: there is real knowledge generation happening.

Let’s assume that the expected value of a Liz Lemma paper is just 20% of a human paper. For all the papers I’ve just generated we’d expect ... the value of about 20 human papers.

On the plus side, she was done fast. On the downside, without all those interesting papers that hardworking and conscientious people already wrote, she might not have had anything interesting to say.

The Abductive Frontier

Any given paper might be interesting or valid, or boring and wrong. It’s hard to know. But we can observe the process as a whole, and notice that we are filling in the gaps.

Here’s the full Liz Lemma-fication of my sources:

So dense you can hardly read it! You can just make out the clusters of economics, math, and computer science. And you can see how each child fills in the gaps, just a little …

And thus the abductive frontier, the next-best-guess space of papers, expands, as the children are verified and approved or rejected.

Thanks, Liz!