Last week, Diogo Almeida, one of the people behind the research that eventually led to ChatGPT, took his company TypeSafe out of stealth with a pretty bold idea: maybe we've been using the wrong kind of AI for a whole bunch of automation. PromptMates was lucky enough to get early access to their first model, jev. Shout out to Matt Texeira for helping make that happen.
I've spent the last couple of days playing with it, and I want to be careful here. I'm not trying to become one of those people who declares every new AI thing the future of work 36 hours after it launches. But I do think this could be a pretty interesting unlock for HR and Talent.
The easiest way I can explain TypeSafe is that it's trying to create a third lane. On one side, you have the giant conversational LLMs we all know: Claude, ChatGPT, Gemini, and so on. They're really smart generalists. You can give them a messy problem, ask follow-up questions, have them reason through something, write something, analyze something, and generally do a whole bunch of different jobs. On the other side, you have traditional classifiers, much narrower systems built to recognize one specific thing.
TypeSafe calls what it's building a "System One" model. jev doesn't really chat with you. You give it something to judge and it gives you a typed answer with a probability attached. Essentially: yes, and I'm 96% confident. Or no, and I'm 74% confident.
That sounds way less impressive than Claude writing a strategy document, until you start thinking about how much of our work is actually made up of tiny judgment calls. Does this contract contain a non-standard clause? Does this HR case need escalation? Does this expense report violate policy? Does this offer letter deviate from the approved template? Does this interview feedback contain language that needs a human to look at it?
That's when I started paying attention
The really interesting part of TypeSafe's pitch wasn't just that jev could answer those questions cheaply. They were also making a stronger argument around the reliability of the probability itself. This is where I had to go learn a new word: calibration.
Here's the ELI5 version. Imagine a weather app tells you there's a 70% chance of rain on ten different days. If the app is well calibrated, it should actually rain on roughly seven of those days. That means the 70% number is useful. A badly calibrated system might constantly tell you it's 95% confident while being wrong all the time. The answer might still look impressive, but you can't safely build automation around that confidence score.
That distinction becomes really important if you're building an agent. Maybe anything above 90% gets handled automatically. Anything between 70% and 90% gets another check. Anything below 70% gets kicked back to a human. If you're going to build that system, the confidence number has to mean something.
TypeSafe's argument was basically that this was an area where jev had an advantage over traditional LLMs. So I decided to test it.
This is probably my favorite part of the whole thing, because I'm absolutely not an ML engineer. I started by doing exactly what I keep telling people in PromptMates to do. I gave all of the TypeSafe documentation to my LLM of choice and basically said, explain this thing to me like I'm five. Literally ELI5.
Once I understood what TypeSafe was claiming, I asked the next question: how would we actually test whether that claim is true?
We landed on a really simple binary judgment: is this piece of text positive or negative? It's not an HR use case, and that was intentional. TypeSafe is still in beta and doesn't currently have the enterprise controls I'd need before sending it candidate information, employee data, contracts, performance reviews, PII, or anything else remotely sensitive. Everything in this test used public benchmark data.
Then I wanted two levels of difficulty
For the easy test, we used 200 movie review snippets from SST-2. Movie reviews felt perfect because sentiment is usually pretty clear. "This movie was incredible. Go see it." Positive. "This was two hours of my life I'll never get back." Negative. Obviously there are exceptions, but generally speaking, movie reviews give you a pretty clean yes/no sentiment problem.
And honestly, the first test was almost too easy. All three systems were very good. jev got 96% correct. GPT-4.1 got 96% correct. Claude Opus got 98% correct.
So then I wanted to make it harder. For round two, we used 300 examples from TweetEval. Tweets are messy. There's sarcasm, slang, inside jokes, mixed sentiment, and people saying something positive while clearly meaning the opposite. Exactly the kind of stuff where a model probably shouldn't be 99% sure of everything.
That made tweets a much better test of the confidence scores. Every system got the exact same examples. We ran all 500 through jev, GPT-4.1, and Claude Opus 4.8. For each one, we looked at a few things: did it actually get the answer right, how good was the probability it assigned, how well did its confidence line up with reality, how long did it take, and what did it cost?
Along the way I learned a couple more ML terms: Brier score and Expected Calibration Error, or ECE. You really don't need to know the math behind either one for this conversation. The important part is that lower is better, and ECE specifically helps answer the weather-app question: when the model says it's 80% confident, does that confidence actually behave like 80%?
On the harder tweet test, jev got 94.3% correct. Claude Opus got 94%. GPT-4.1 got 95.3%. Pretty damn close.
Then I looked at calibration. jev's ECE was 0.069. Claude was 0.096. GPT-4.1 was 0.050. Again, lower is better.
TypeSafe didn't magically get these decisions right while the giant LLMs got them wrong. GPT-4.1 was actually slightly better calibrated than jev in my test.
That matters, because I don't think we should repeat a vendor claim just because the technology is new and interesting.
But then I got to the part that made me much more interested in the product. It was stupid cheap.
For the 300 harder examples, jev cost roughly $0.004. Not four dollars. Four tenths of a cent. GPT-4.1 cost around $0.073. Claude Opus cost around $0.30.
Obviously one 300-item benchmark isn't going to destroy anybody's AI budget. But scale that thinking. What happens when an agent needs to make 100,000 little decisions? Or a million? What happens when every candidate, every document, every ticket, every message, every workflow step has one or more judgment calls sitting inside it?
Suddenly the model you choose for those tiny decisions matters a lot
jev was also the fastest of the three in my testing, with median latency around 400 milliseconds versus roughly 700 milliseconds for GPT-4.1 and more than 1.2 seconds for Opus.
So TypeSafe didn't win the test the way I expected. It didn't prove that traditional LLMs can't make trustworthy decisions. It proved something else: you may be able to get essentially the same quality of small judgment for a fraction of the cost.
For those of us sitting inside HR, Talent, People Ops, Finance, Legal, and the rest of G&A, that is not a small thing. We don't exactly have revenue-generating AI budgets raining down on us.
The testing also changed how I think about the actual HR use cases. My first instinct was to make a giant list: contract scanning, certification checks, candidate routing, PII detection, interview QA, expense reports, HR tickets, comp approvals. Then I started applying what I learned from the test, and a lot of those disappeared.
There are really three questions I'd ask before using something like jev.
First, is this actually a judgment or is it just a rule? "Is this certification expired?" is a date comparison. "Are all the approvals present?" is a checklist. "Is this candidate already in the ATS?" is a database query. "Is this comp request outside the approved range?" is math. Those don't need AI. Using an AI model for those is basically taking a spreadsheet formula and making it probabilistic.
Second, does the use case touch sensitive data? Right now, that removes a huge chunk of real HR applications for me. I'm not sending employee files, I-9s, medical documentation, background checks, performance reviews, comp information, or candidate PII through a beta external model.
There's also an interesting catch with something like PII detection. "Does this document contain PII?" sounds like a perfect yes/no model question, except you have to send the document somewhere to ask the question. You may have just created the exact data leak you were trying to prevent. That job belongs with a local privacy gate before the information ever leaves your environment.
Then you get to the third question: is this genuinely fuzzy, high volume, and valuable enough that confidence-gating can save a human time?
That's the sweet spot
Outreach QA might be one of the best examples. Imagine a sourcing agent generates 10,000 outbound messages. Before each one goes out, something asks: is this personalized enough, does this contain an unsupported claim, did we accidentally mention the wrong company, does the tone look appropriate?
If the model is highly confident everything looks good, send it. If confidence drops, tap the recruiter on the shoulder. Now the recruiter isn't reviewing 10,000 messages. They're reviewing the weird ones. That is a fundamentally different way of thinking about human-in-the-loop AI.
Candidate relevance is another potential example. Risky language in job descriptions. Potentially HR ticket routing once the content has been properly de-identified.
There's one other thing from my testing that matters here too. jev tended to be a little more conservative in some of those middle confidence ranges. That means if I set the system to only auto-act above 90%, jev may kick more things back to the human than another model would.
That's not necessarily bad. It fails toward human review. But if the entire business case is saving recruiter time, you need to know that when you set your thresholds.
The real takeaway wasn't TypeSafe
I didn't know how to evaluate this thing when Matt gave us access. I'm not technical enough to read a model paper and immediately understand where something belongs in an architecture. So I used AI to help me understand AI.
I had it teach me what TypeSafe actually does. I had it explain calibration to me like I was five. I had it help me think through how you could test the claim. Then I brought the thing I do know really well to the table: HR and Talent.
The people building these models know models better than we do. They do not know recruiting better than recruiters. They do not know the thousand weird little judgment calls buried inside your company's HR workflows. We do.
So as all these new tools show up, I don't think our job is to sit around waiting for somebody to package one up as "AI for HR." Try the new thing. Ask stupid questions. Make the model explain itself. Question the marketing claims. Test it on something safe. Figure out where it works. Figure out where it absolutely should not be used.
Then bring your knowledge of the work to it. That, increasingly, feels like the job.
And if we can eventually use a model that handles a huge percentage of those little HR and TA judgment calls for literal fractions of a penny, for teams that have historically struggled to get budget for any of this stuff, that could be a pretty cool thing.