How psychometric testing transformed my startup
I was bad at talking to people. That's a slow death sentence for a founder: to build something people want, you need to talk to people. I'm social and have no aversion to talking, but I struggled to learn through a casual conversation by listening, engaging, and asking good questions.
As a result, we built things people didn't want. I floundered for a couple years, until I found something that finally started working.
Before finding something that worked, I tried a lot that didn't work:
- I read books about talking to users, and talked to hundreds of potential customers over Zoom.
- My co-founder and I took trips to visit customers in person.
- I went to conferences to talk to potential customers in person.
- We went through Y Combinator, and tried applying other founder's advice on how to talk to customers[1].
One day, my therapist – who is also effectively my executive coach – mentioned something interesting. She noticed that I excel at tasks that involve learning by reading, but struggle with tasks that require learning through conversation. A discrepancy like that is often a sign of an underlying issue, like an auditory processing disorder. She suggested a comprehensive psychometric evaluation: a full day test where a psychometrician measures aspects of your cognitive function.
The evaluation
The evaluation started with the psychometrician – in this case, a psychologist with a PhD – talking with both me and my partner. After about 45 minutes, my partner was dismissed and we dove into the 1:1 evaluation.
The core of the test was the WAIS-IV, an adult intelligence test that measures different areas of cognitive function. Some of it was similar to tests I'd taken as a child, like rearranging blocks to recreate designs that I was shown. Most were new to me, like listening to dozens of words ("elephant", "cabbage", ...) and answering questions about what I heard.
After the lengthy evaluation and a weeks-long wait for the results, I learned that my therapist was right: I'm measurably bad at learning through conversation, but in the top 1% of learning by reading[2].
I learned that my working memory is horrible, so I struggle to remember what people say the first time they say it – 98% of people are better at this than me. After the third time something is repeated, I reliably remember it – only 16% of people scored higher. That's like being blind for "remembering what people say, the first time they say it" – and I had no idea.
I was shocked. I hadn't assumed that I might struggle with the core tenets of conversation, like hearing and remembering what the other person says. I always identified as a "smart person" – I was grouped with the fastest learners in school, finished my undergrad in computer science in 2.5 years, and started a venture-funded startup in computational biology. I didn't feel socially inept, either; I always had strong friendships and relationships.
But tests don't lie. I was really, really bad at the most core tenet of learning from people: remembering what they say.
Post-evaluation
The psychometrician gave me a twelve page report, with a list of possible next steps ranging from drugs to tai chi. The most game-changing alteration I made to my life was medication. After some experimentation, we found a good combination of two medications[3] that sufficiently increased my cognitive function without changing my personality.
It was like wearing glasses for the first time. Learning through conversation started to work. I could remember new information after hearing it just once, and started to be truly open to listening to what others were telling me[4].
I practiced the basics of learning through conversation with my partner: how to really listen, pay attention to subtle cues, engage, and ask good questions. I had known the theory, but now it started to work in practice.
I started to hear what customers were saying, picking up on the now-obvious signals of the polite but unenthused: a lack of urgency is a lack of need; "it costs too much" with no alternative is really "this doesn't solve an important enough problem". There are still challenges, but we're finally starting to build things people want at a higher success rate. Features we build are now matched to customer demand, and they are used shortly after going live.
Talking to customers is more of an ego hit now, because I realize my ideas are bad more often. But it's also more rewarding, as I've started to genuinely learn from them and leave conversations with a different perspective.
An aside: technical founders and Y Combinator
I think something similar might be true for many technical founders that go through Y Combinator. They're great at learning by reading, and comparably bad at learning through conversation – so some focus on a technical skill that they can learn by reading, like software engineering, and struggle to make the transition to learning by talking to people.
Conclusion
It's weird how universally this changed my life, even outside of work.
Prior to re-learning how to have a conversation, I never talked to strangers sitting next to me on a plane. I feared either talking too much or having nothing to say.
How much changed became clear on a plane ride from Boston. I helped the woman sitting next to me find the power outlet between our seats, and we accidentally wound up talking for the entire trip. It was one of the best conversations of my life: my perspective on religion, school, and cities was different by the end of the conversation. We trusted each other enough by the end of the conversation that I dropped her – a former stranger – at her house on my way home from the airport.
I think it's possible to be happy without learning through conversation; happy engineers, actuaries, and academics who don't learn well from conversation exist, just like there are happy people with uncorrected vision. But I'm surprised I didn't realize this about myself earlier, and I wonder how many other people are in a similar situation.
Thank you Paul Butler, Cameron Ferguson, and Andrew Israel for reviewing drafts of this.
Y Combinator makes it easier to learn how to be a startup founder by reading. But that only gets you so far (for now): to learn about your customers, you need to actually talk to them. ↩︎
I'm avoiding using the diagnostic labels and focusing on what's measurable in this post, because I think the labels distract more than they add value in this context. For categories related to learning by reading (e.g. symbol search, vocabulary, and information) I scored in the top 0.04-1%. For categories related to learning through conversation, my performance was much more variable – reaching as low as the bottom 2%. ↩︎
I'm happy to share more context if you reach out to me individually (first name at last name dot com). ↩︎
Or in medical terms, I had more working memory, attention, and cognitive flexibility to work with. ↩︎