Can you Jev code a Gmail spam filter?
Last week everyone was talking about Jev, so I decided to test whether you can build an anti-spam system similar to or better than Gmail's.
What is Jev? It's essentially an LLM-based classifier that, instead of text, responds with the probabilities of the labels. If you want a deep dive, Archer Hume unpacks Jev’s architecture.
I pulled 460 of my own Gmail emails, split half and half between the spam folder and the inbox, and 3,293 labelled emails from the PhishFuzzer benchmark, dated 2002 to 2026.
I then ran Jev to classify the emails into four categories: legitimate, spam, fraud, and malware.
The overall results look strong:
- It detected harmful emails that were fraudulent or contained malware, with 100% recall in my Gmail (44 of 44) and 94.3% in PhishFuzzer (1,060 of 1,124).
- The overall performance on PhishFuzzer is at the level of Gemini-3.1-Pro, macro-F1 0.676 against 0.685.
- It detected a fake Paperless Post invitation linking to a remote-access installer, from body text alone. Gmail took two days to start filtering it.
- Cost and latency are impressive: $0.25 per 1,000 emails, 0.24 seconds per email.
My take:
- You can use Jev to vibe-code an email filter that will do a decent job of filtering out malicious emails. It could be a solution for price-sensitive companies that can't afford to pay an extra $5 to $35 per seat for a third-party email security tool, or for DIYers like me at The Weather Report.
- Spam is a tricky category. The canonical definition of spam is unsolicited email with substantively identical content. Some spam is objective: a crypto pitch from an unknown domain, addressed to nobody. Jev detects that with ease. But subjective spam is about how you feel about an email at a moment in time. For example, the newsletter you read every week is not spam today. Six months later, after you have lost interest and forgotten to unsubscribe, it feels like spam. Gmail sees your previous spam labels on it, but Jev doesn't.
- Detection is a constant fight over four constraints: precision, recall, cost, and latency. Jev is moving the detection frontier, so you make fewer trade-offs.
- The next big category to check is whether Jev can reliably detect indirect prompt injections in emails. Super critical as we give agents access to our mailboxes.
- Finally, the cost and latency are impressive and look affordable, but not for Gmail, which handles 1.8 billion active users.
The prompt I used:
state: email_headers, email_subject, email_body.instructions: Classify this email using email_headers, email_subject and email_body. If more than one category fits, choose the most harmful: malware, then fraud, then spam.legitimate: Ordinary mail: personal or business correspondence, a notice about the recipient's own account, transaction or appointment, or a mailing that shows the recipient is on the list on purpose, by naming the list, the account or the signup it came from. Bulk sending alone does not disqualify it, since companies and publications the recipient chose all send in bulk.spam: Mail sent in bulk to sell, promote, recruit or gather attention, with nothing showing the recipient is on the list on purpose. It does not pretend to be another brand or person and it invents no event, so responding costs the recipient only their time. Sending from several lookalike domains of the sender's own brand is still spam.fraud: Mail that misrepresents its sender or its situation, or threatens the recipient, in order to extract money, credentials, personal data or a reply. The deception or threat is the mechanism, whether it impersonates a brand or person, imitates someone else's domain, invents the event it reports, or hides what answering will lead to. A charge or a service is still fraud when the pretext for it is invented.malware: Mail whose purpose is to get software running on the recipient's machine, by attaching or linking to something that executes, installs or opens a channel to the sender. Whatever story the mail tells exists to make the recipient open it.