Are Personality Tests the New Horoscopes? (Patreon)

[This is a transcript with links to references.]

Are you extroverted, a social butterfly? Or more the introverted type who lingers around in a corner of the room? Are you agreeable? Or quick to argue? Do you like your routine? Or do you get easily bored? What’s your personality?

The internet is full of personality tests. Self-help gurus use them for dating advice. Employers use them to decide whom to hire. Schools use them to recommend students what to study. And travel companies use them to suggest travel locations. *I recommend if you have a personality, you should subscribe to this channel.

Are personality tests scientific? Or are they just a modern version of horoscopes? Did you know that Google reportedly only looks for one personality trait and now wonder which one it is? That’s what we’ll talk about today.

The Oxford Dictionary of Psychology defines personality as the total of behavioural and mental characteristics that are distinctive of an individual. It’s the pattern of thoughts, feelings, and behaviours that make every person different.

Let’s take this definition apart. The first relevant part of the definition is that it’s a *pattern. Other behaviours may be distinctive, but they change from day to day, to hour to hour. They might depend on what you ate or how much you slept or what you read in the news in the morning. We call that “mood,” not personality.

The relation between mood and personality is like that between weather and climate. Personality is the long-term average. Averaged over a few couple of years, some people and places are a little bit more rainy than others, or a little bit more sunny, or a little bit more stormy.

The second important part is that it’s a *distinctive pattern. I tend to get tired at night and sleep regularly. That’s arguably a behavioural characteristic, and it’s also a pattern, but it’s one common to all humans, so it’s got nothing to do with my personality. It’s not distinctive.

How long does a distinctive pattern have to last to count as a part of your personality? Loosely speaking, if it’s less than a few weeks it doesn’t count. A few months or years, and you can call that part of your personality.

If that sounds a bit vague it’s because it is. This is psychology after all, not physics. Psychologists however like to up their science game by measuring things. That’s why they’ve come up with several ways of quantifying personality types.

If such a test actually measures an aspect of your personality, then it should be compatible with the definition we just looked at. This means first of all that it should be something that’s specific to you, and not a general characteristic of human behaviour.

You also want to make sure that the test tells you something about what the person *does, not just about what they say. So you need an independent way to assess the answers. You can do this by asking other people or by asking the person for verifiable details from their life. It’s called the “validity” of the test.  

Third, the test should identify a pattern, so the results should remain reasonably stable when you retake the test after a few days or weeks. That’s the “reliability” of the test.

And fourth, you want to look for traits that are aspects of human psychology, not of social or cultural context. This property of a test is called “universality”. Though calling it “universality” is maybe somewhat of an exaggeration, not like we’re going to ask people in Andromeda whether they like partying. That a test is universal doesn’t mean that the *traits shouldn’t depend on cultural context – they usually do – but that reliability of the test doesn’t depend on it. Basically, the traits should exist wherever you go. In a nutshell, Marmite is not a personality type.

The two most widely used psychology tests are the Myers-Briggs Type Indicator and the Big Five Inventory. You can take simple versions of both of those tests for free online, though they’ll usually try to get you to pay for a detailed assessment. Maybe watch this video to the end before you spend money on those tests. We’ll first look at what science has to say about Myers Briggs and the Big Five, and then we’ll look at what we can learn from this.

The Myers-Briggs test assesses you on four different scales, each of which has a letter. That’s the scale from extroversion to introversion , sensing to intuition , thinking to feeling  and judging to perceiving. The result gives you a four-letter label. All together there are 2 to the 4, so 16 personality types. And that’s all the maths there’ll be in this video.

This test was created in 1943 by two American writers, Isabel Briggs Myers and her mother Katharine. The father was a physicist, which might have been why they felt a need to understand human personality types. But the overarching goal of their test was to help find employment for women while men were fighting in World War 2.

Here’s me taking a free online test of the Myers-Briggs type. I come out being INTJ, that’s introverted, intuition, thinking, judging. And this particular brand of test also thinks I’m a little turbulent, whatever that means. I think my mum would agree.

In any case, according to the result page I am “rational and quick-witted” with an “uncanny knack for seeing right through phoniness and hypocrisy” and that I “may struggle to find people who can keep up with [my] nonstop analysis of everything around [me].”

That’s quite accurate and I’m very flattered which isn’t so surprising because this is the most flattering way to rephrase the answers I just gave to all those questions. A less flattering version is that I’d be the kind of person who makes jokes at a funeral if I wasn’t too introverted to talk to anyone in the first place.  

I’ve taken this test a few times over the past 20 years and my result hasn’t changed, but turns out that for many people this isn’t the case. An old version of the test was found to reproduce the classification for 47 percent of people at re-testing after five weeks. This might sound worse than it is since there are 16 categories. If the results were basically random, the chance to reproduce the result would be one over 16 so about six percent.

However, a newer version of the test was found to reproduce the results for 60 to 75 percent of test-takers *per trait, which altogether gives a reliability of 25 percent at best. Other studies have found similar values. It’s usually less than a half of people for whom the classification remains unchanged.

You see, the issue with the Myers-Briggs test is that it starts from the premise that people tend to the extreme ends of these scales, so it’s an either-or distinction. You’re either extrovert or introvert. You either think or feel. And so on.

But in reality, most people are non-extreme on those traits and cluster towards a mean value that’s near the middle, though not exactly at the centre. We all both think and feel, both judge and perceive, and rely on both our senses and our intuition. And few people are as extremely introverted as I. In fact studies have shown that extraversion in the population is pretty much a normal distribution with most people clustering in the middle.

To remedy this problem, Myers-Briggs tests often exclude a completely neutral answer to force people to pick a side. But when forced to pick a side, people do this randomly, and then their personality type looks unstable over time.

Now you could say, well, then why not just use sliding scales for the four characteristics, rather than packing people into those 16 camps? Alright, but that still doesn’t solve the issue that those characteristics weren’t particularly descriptive of individuals in the first place. It’s like trying to describe a book with its weight and electrical resistance. Sure you can do that. But does it tell you anything about the book that matters?

This is why the Myers-Briggs test, while extremely popular, has a bad reputation among psychologists. The psychologist David Pittenger says for example that the test “does not conform to many of the basic standards expected of psychological tests” and “there is no evidence that the [Myers Briggs test] measures anything of value.”

According to the science journalist Angus Chen “psychologists say the questionnaire is one of the worst personality tests in existence”. And in a 2012 article in the Washington Post, Carl Thoresen, at the time chairman of a company who sells those tests and professor of psychology at Stanford, admitted that he hasn’t even mentioned the test once in any of his roughly 150 papers, in part because, quote “it would be questioned by my academic colleagues”, end quote.

That’s pretty damning.

That said, you won’t be surprised to hear that the Myers-Briggs classification isn’t very predictive of anything, certainly not job performance. For example, in 2015, a team from the UK and Norway looked at a sample of over 6000 adult managers who completed the Myers-Briggs test and who also provided details about long it took for them to be promoted. They found that shorter times to promotion were correlated with being Extraverted and a Sensing Type, while Feeling and Perceiving types took longer to be promoted. However, the correlations are so weak they’re all compatible with zero.

Okay, you may say, maybe the Myers Briggs test doesn’t quite capture human personality, but no reason to give up on the idea entirely.

Let’s therefore look at the next most popular test, the Big Five Inventory, also known as the 5-Factor Model. It was developed in the 1960s at an Air Force Base in Texas, probably to assess new recruits. It was created by compiling every word that could be considered a personality trait and creating simple questions to inquire about them. Traits that seemed to be related – like “talkative” and “sociable” – were collected in five categories that gave the test its name: extraversion, conscientiousness, agreeableness, neuroticism, and openness to experience.

Extraversion we already met previously. Openness to experience is pretty self-explanatory. Individuals high in conscientiousness tend to be careful, responsible, and organized. Those high on neuroticism tend to be anxious, depressed, angry, or insecure.

Agreeableness doesn’t mean you agree with everything, but rather it’s a measure of tolerance, modesty, or sympathy for other people’s problems. And for the non-native English speakers, “disagreeable” doesn’t mean that someone likes to disagree, it’s what you say of fish that gave you food poisoning.

Here’s me taking the Big Five Test. I’m neither friendly nor cheerful, which won’t be news for you. I’m very agreeable because I said I worry about homeless people, and I excel in conscientiousness because I’ve asked my team to cross off weekly tasks on a 30-point checklist so my head doesn’t explode.

Ok, so well, it measures something, arguably, but is it useful? A 2014 meta-analysis looked at the Big Five retest reliability in 67 studies involving about 15 thousand people in total, almost half of them from the US. The median reliability was almost 82 percent, which is okay I guess, seeing that we’re talking about psychology.

Extraversion scales resulted in the most dependable scores, whereas agreeableness scales had a slightly smaller reliability. A big five test on about 300 indigenous people in a Bolivian village, however, found a reliability comparable to that of the Myers-Briggs test, so, maybe reliable, but not universally so.

What about the specificity of the Big Five test? Does it actually tell you something about people, individually?

Some studies have indeed found that the Big Five traits correlate with a number of different, individual aspects of life. For example, conscientiousness is correlated with good health and academic performance. Neuroticism is negatively correlated with job performance. Agreeableness is positively correlated with happiness, quality of life, and the ability to collaborate with and lead people. Though, generally, none of those correlations are particularly strong, usually in the range of low to moderate strength.

But the Big Five test quite plausibly tells you something about job performance. After all, a lot of the questions are basically asking for how you work: Do you complete tasks? Do you procrastinate? You’d expect this to tell you something about people’s job performance, wouldn’t you. Which brings up the question who’d answer “no” in a job interview when asked whether they complete tasks. It’s an issue with those tests in general: How do you know people give honest answers? Do you ask them if they’re honest?

Self-reported traits might reflect the personality that people would like to have rather than the one they do in fact have. But while this means you shouldn’t take their answers at face value, their answers might still correlate with some aspects of their behaviour.

This brings us to the next question, which is what those numbers tell us to begin with. Is your personality something you’re born with, or something that you’re raised into, or something that changes throughout your life? Well, it’s a mix of all those.

Several studies have found that the heritability of the Big Five traits lies between thirty and sixty percent. For example, a 2014 meta-analysis found that the heritability of neuroticism and extraversion scores was at 48 and 49 percent, respectively. Another 2015 meta-analysis found a heritability between 30 and 40 percent for all five traits.

However, the way the genes play out depends very much on personal history, which is why all those personality traits can change substantially throughout life. For example, the study on the reliability of the Big Five test that I mentioned earlier also found that reliability decreased with retest time after long periods.

This is broadly speaking compatible with other studies which found that environmental influence on the big five personality traits increases with age. For example, a 2014 meta-analysis involving over 21 thousand sibling pairs found that genetic influence on the Big Five traits stabilises near the age of thirty. Environmental factors are almost non-existent in childhood, contribute about as much as genetic factors by midlife, and then stabilise in old age.

If you look at any website where you can pay for a personality test, they’ll claim that their customers find the test to generally be accurate and descriptive of their personality. Indeed, I am myself inclined to say the results of those tests are fairly accurate.

This isn’t terribly surprising though because test results are a flattering version of a description you just gave yourself. Of course, people are inclined to say it’s accurate if it says nice things about them. There’s also the problem that people are unlikely to pay for being told they’re ignorant jerks.

Horoscopes exploit the same human weakness: We want to believe nice things about ourselves. It’s known as the Barnum effect, named after the showman P.T. Barnum, who was known for his ability to attract large crowds to his circus acts with generic flattery.

As a physicist I’m not particularly impressed by all this personality testing. As we’ve seen, if correlations exist at all, they tend to be small. Indeed, personality is such a complicated mix of genes, experience, sociology, and culture, that I find it extremely questionable there will ever be a good classification of it. Nonetheless I’m quite fond of personality tests. Let me explain.

I used to read horoscopes with my grandma. Neither of us actually believed in what they said. But they gave rise to interesting thought experiments. What if today we actually made a surprising discovery about a good friend? Who might it be? Have I really been spending too much time in a dream world recently? Maybe I should indeed come to a decision this week.  

I like personality tests for a similar reason. They are an occasion for self-reflection. They tell you who you are and ask you who you want to be.

It’s a curious fact that we are thrown into this world, not knowing much about ourselves. Growing up is very much about getting to know yourself, and finding your place in this world. If personality tests help you get there, I think that’s a good thing. And that one personality trait that Google is allegedly looking for in hiring decisions? Curiosity.

As we were working on this video, we found a bewildering variety of new personality tests. They go by names like HEXACO, the NEO Personality Inventory, the Whole Trait Theory, or the Cybernetic Big Five. It was just too much to cram it all into one video. But if you think the topic is interesting, let us know in the comments, and we’ll have a closer look at those newer tests.