We Can Solve Psychology with Text Embeddings
How unique are humans, exactly? How can we tell?
Personality Lexicons
The standard psychological test is “OCEAN” - Openness, Conscientiousness, Extraversion, Agreeableness, and Neuroticism. Five axes, scored from 0 to 100, to get the gist of a human personality. Meyers-Briggs, not well regarded by academia, has 16 “types”. Astrological signs, even less well regarded, remain popular as a primal form of personality typing.
It raises an interesting question: how unique is the typical human personality? There are some 8 billion people on the planet. How unique is each of their personality? What’s the variance, the standard deviation? Who are the outliers?
We’d probably notice if there were really only 12 consistent astrological signs, or 16 types of people period. It’s just not that many! But maybe it's fewer than 8 billion.
The "lexical hypothesis" of psychology is something like this:
For a given human internal state, trait, or characteristic that is socially relevant and important
People will naturally develop words and language to describe and discuss it
The more important the characteristic, the more likely it is to be encoded in language
By studying the personality-descriptive words that exist in a language, we can identify the most important personality traits and dimensions
The OCEAN test comes from some rough categorization of the personality lexicon. But it's broad strokes: you're looking at personality through a low-powered magnifying glass. The detail is fuzzy.
Text Embeddings
Text embeddings via modern language models are a huge leap in our ability to capture and analyze personality-relevant linguistic patterns. For those unfamiliar: text embeddings are big chunks of data, in a standard format, that represent text. If you give OpenAI's "text-embedding-3-large" model the phrase “I ate a carrot,” it will return 3,072 numbers between 0 and 1 which best represent that text. That’s the text embedding!
Text embeddings are part of the foundation of large language model development. As language models have gotten bigger and better, so too have text embedding models.
How does this relate to personality? Each embedding can encode subtle variations in word choice, phrasing, and expression. The traditional lexical approach gave us broad categorical distinctions ("extroverted" vs "introverted"); embeddings capture fine-grained features.
A 30-minute conversation analyzed via modern language embeddings is like moving from a magnifying glass to an electron microscope. Each sentence provides hundreds of subtle signals about personality. With 3072 dimensions of measurement, we have far more precision than we actually need to uniquely identify someone's personality pattern.
We could think about a "personality" as a set of distances from reference words. Let’s say I wrote down some phrases for a few different categories - things like “Careful planning prevents most problems” goes under “Conscientiousness”, “Established hierarchies maintain order” under “AuthorityOrientation”. You could ask me a question - “What is your favorite color?”, hear my response - “blue”, and visualize the distances like this:
That unique shape is, with respect to that answer, my personality. Here’s what it looks like in two dimensions:
Personality Shape, Personality Context
You probably wouldn’t learn much about how conscientious I am from the distances to “blue”. But let’s say we answered a series of questions, like in a job interview: “What are your strengths?”, “Name a time you experienced conflict,” and so on. We’d have a more complex structure to observe. These answers don’t show distance to references, just to each other:
If we want, we can look at the reference distances like a grid, with every question a row, and every reference a column:
This grid is our best guess at a single, unified “personality”.
Personality in Context
Our personality changes based on context. We act one way around our friends, another around our family, another at work. Depending on the context in which we operate, our personality might appear different, even if it's the same underneath. Here’s a simple illustration:
If we think of personality as a three-dimensional object, like a sculpture on display, we can see it in a few different ways:
A professional interview is like viewing it with a spotlight from above, casting one kind of shadow.
A first date is like illuminating it from the side, casting a completely different shadow.
A casual conversation with friends casts yet another.
When we look at our nodes, we can measure how personality features maintain their relative distances even as the perspective shifts. The "top" might be our personality seen from the perspective of the "Conscientiousness" embedding position; the "side" might be our personality seen from the perspective of the "ClusterC" embedding position. We can model the shift below:
The shape we see from one angle is not the shape we see from another, but the personality remains consistent. This idea will be crucial as we try to assess personality through a variety of lenses.
Personality in a Population
No man is an island. Personalities in relation to text is fine; more important is their relationship to other personalities. The image below shows different embeddings, colored by their similarities to a phrase.
Clusters emerge. You might find that for a given assessment, men fall on the left, women on the right. Or maybe religious people here, atheists over there.
Imagine if everyone in the world had filled out a similar test. We’d see all the answers laid out in front of us, just like above. What would the clusters tell us? Quite a bit, probably!
The implications of a well-scoped and valid personality test are fairly stunning. We can solve psychology! Well, maybe. We’d have to make the following assumptions:
There exists a distribution of traits relevant to "personality"
These traits are knowable and sufficiently distinguishable as to be representative via text embedding
These traits are sufficiently consistent as to be useful and inconsistencies are explainable across axes
This gets us most of the way there. Text embeddings are a huge leap forward for our understanding of ourselves.
They’re also vital tools for understand our newly intelligent friends, the language models. Read the followup to see how Claude and ChatGPT compare personality wise.
