Step-by-step guide Your First AI Agent in a Weekend: inbox triage + lead follow-up →
The average worker receives 117 emails a day, most of them skimmed in under a minute, according to Microsoft's telemetry (Microsoft Work Trend Index, June 2025, from data that excludes EU tenants). Somewhere in that pile sit the few that matter: a new client asking about work, a supplier delay that threatens a deadline, a customer with a problem. We wanted to know how much of the sorting and chasing an AI assistant could take on without ever speaking for anyone, so we built one and scored it on 50 test emails whose correct answers we wrote down before the first run.
What an AI inbox assistant does, and what it doesn't
Strip away the word "agent" and the pattern is simple: read each email, decide what kind it is, then do the one thing you've agreed should happen for that kind. Ours does four jobs:
- Sort. Label every email Lead, Client, Supplier, Admin or Later, and add a second label when it's urgent.
- Capture. Copy each new inquiry into a lead sheet, with the details the email actually states.
- Draft. Write a reply to each new lead in your style and leave it in Drafts, inside the lead's thread.
- Chase. Every weekday morning, draft a polite follow-up for leads that have been quiet for three days, and stop after the second.
What it doesn't do is send. Every step that writes to a customer creates a draft. AI drafts are mostly good, but now and then they get a name or a date wrong, and a ten-minute morning review catches that.
Why the follow-up matters more than the sorting
Sorting saves minutes; following up is where the income is. In March 2011, Harvard Business Review published an audit of 2,241 US companies that timed how quickly each answered a web inquiry (HBR, March 2011). Of those, 37% replied within an hour, but 23% never replied at all. A separate study by the same authors, covering 1.25 million leads, found that firms that tried to make contact within an hour were nearly seven times as likely to qualify the lead (a meaningful conversation with a key decision maker) as those that waited even an hour longer. Their conclusion: companies "should be responding at internet speed".
The data is American and dated, and no assistant can promise a reply time. But the mechanism holds: an inquiry that waits goes cold, and a reply already waiting in your drafts makes a quick answer easier.
How the build fits together
| Automation | Starts when | What it leaves you |
|---|---|---|
| Sorter | A new email arrives | One category label, plus AI/Urgent when needed |
| Lead catcher | An email gets the Lead label | A row in the lead sheet and a reply in Drafts |
| Follow-up assistant | Every weekday at 08:00 | A follow-up draft for each lead quiet for 3 days or more |
We built it in Zapier with its AI step, AI by Zapier, choosing Claude 5.5 Haiku from the model menu, plus Gmail and Google Sheets. There's no code and no API key. Multi-step automations and AI steps need a paid Zapier plan or its trial. Most lessons below apply to any automation tool.
Start with a rules card, not a prompt
The most useful hour of the build is spent outside Zapier, writing a rules card: one page that defines each category and the urgent flag, and says what happens to each. The detail does the heavy lifting. A line like "a client asking about their own invoice is Client, not Admin" turns a vague sorter into a dependable one. Write down every case you hesitate over, because the model will hesitate there too. The card goes into the prompt word for word, and when the assistant gets something wrong later, you change the card, not the automation.
What we measured on 50 test emails
Our worked example is a fictional three-person interior design studio in Amsterdam. We wrote 50 realistic emails for it, including awkward ones like a phishing-style "verify your account" message and a lead writing for the second time. The correct answers were fixed before the first run, in November 2026, and never changed.
| Measure | Result |
|---|---|
| Categories right | 49 of 50 (98%) |
| Urgent emails caught, first wording | 4 of 10 |
| Urgent emails caught, after rewriting one sentence | 10 of 10, with 3 false alarms |
| Lead names and timelines right | 14 of 14 leads |
| Invented lead details in non-leads | 0 of 36 |
| Emails sent by the assistant | 0 |
The one miss was a supplier's "new paint colours" promotion, filed under Later. Our own rules card put promotions under Later, so the fault was ours: the fix is one more line on the card, not a cleverer prompt. Fifty emails is a modest test and your inbox isn't ours, so treat these numbers as a benchmark, not a forecast.
The urgency lesson: concrete triggers beat vague thresholds
Our first rule called an email urgent if it needed action "within 2 working days". The sorter caught only 4 of the 10 urgent emails. We had asked for a one-sentence reason with every decision, and the reasons showed the problem: the model was judging the category and ignoring the deadline. We rewrote that one sentence as concrete triggers:
- Deadlines. A date in the next seven days, or words such as "this week", "tomorrow" or "before Friday".
- Things going wrong. Damage, a delay or a wrong delivery on a live project.
- Waiting leads. A prospect kept waiting, or one who wants work to start within weeks.
- Expiries. Something the business depends on that expires or needs approval within seven days.
Nothing else changed. The second run caught all 10, with 3 false alarms: a lead asking about December dates, a request to reschedule and samples on loan "until Saturday". That trade is worth making, because a false alarm costs a glance and a missed complaint can cost a client. Two habits came out of it: always ask the model for its reason, and change one sentence at a time, then test again.
No invented details: leave it empty rather than guess
The lead catcher fills in name, company, project, location, timeline and budget hint. The risk with any language model is that it fills a blank with something plausible. Two choices stopped that in our test: the prompt says to use only facts written in the email and to leave a field empty rather than guess, and every field except the email address is optional. A required field pushes the model to produce an answer even when there isn't one. Across the 36 test emails that weren't leads, not one invented lead detail appeared.
The reply prompt is as strict: no prices, fees or availability dates. One test draft came back at 93 words, with two qualifying questions and nothing promised.
Three Zapier pitfalls we hit
- The auto-generated output fields were wrong. Zapier writes the AI step's output fields from your prompt. In three of our four AI steps it got them wrong, adding required fields that only echoed the input and missing most lead fields. Check them by hand.
- "New Labeled Conversation" passes only a snippet. It fires once per thread, which sounds ideal, but in our test it passed only a thread ID and a short snippet, with no sender or full text. "New Labeled Email" passes the whole message.
- Duplicates are a lookup problem. "New Labeled Email" also fires when a lead replies, so a plain "add row" step logs the same lead twice. Google Sheets' "Lookup Spreadsheet Row" with create-if-missing, keyed on the email address, solved it: run twice on the same email, it found the existing row and added nothing.
Keep a person on the send button
There's no send step in any of the three automations. The lead catcher ends with Gmail's "Create Draft Reply", and in our test that step returned a single label: DRAFT. One thing to know while you build: Zapier's help centre notes that testing an action step in the editor performs the action in your app (Zapier, updated November 2026). Test a send step and the email goes out, so build on a test inbox.
The matching routine takes about ten minutes a morning: open Drafts, read each reply next to the original, fix what sounds off and send it, mark the lead as waiting in the sheet, then deal with anything labelled urgent.
One more check before you connect a real mailbox. Every email the assistant reads goes to an AI model, so health or client data needs a legal basis and a proper agreement before it goes through any AI tool. Many practices keep clinical mail in a separate inbox and point the assistant only at the general one. Because we build to healthcare standards, that separation is our default too.
What it costs to run, counted in tasks
Zapier meters usage in tasks. According to the same help article, a task is a successful action; triggers, filters, paths, Formatter and Looping steps don't count, and nor do steps a filter skipped. AI steps cost more on higher model tiers; the Standard tier we used cost 1 task per run.
| Automation | Tasks | Per |
|---|---|---|
| Sorter | 2, plus 1 if urgent | Email read |
| Lead catcher | 3 | New lead |
| Follow-up assistant | 1, plus 1 per waiting lead, plus 2 per follow-up drafted | Morning |
The cheapest email is the one the sorter never sees, so exclude newsletters and promotions in the trigger's Gmail search. Zapier has also announced different rules for its newer Next Gen Zaps from 1 December 2026, so check the current article before you choose a plan.
Build it, buy it, or hand it over
If you mostly want summaries and quicker replies, and get only a handful of inquiries a month, the AI built into your mail app may be enough. Build your own when leads matter to your income, when your categories are specific to your business, or when you want to control where your data goes.
Hand it to an engineer when several people answer the mailbox, when replies should draw on your calendar or files, or when the data needs a stricter setup than a general-purpose automation tool. That's the kind of system we build as custom AI applications and integrations.
We've written the whole build up as a step-by-step guide: Your First AI Agent in a Weekend, inbox triage and lead follow-up edition, with every Zap setting, the nine prompts we used, the 50-email test method and a checklist. Other titles are in our guides catalogue. Even without it, the essentials fit in one line: write the rules card first, score a test set before you switch on, and let the assistant draft while a person presses send.
Frequently asked questions
What is an AI inbox assistant? An AI inbox assistant is an automation that reads each incoming email, decides what kind it is using rules you write, and takes one agreed action, such as adding a label, logging a lead in a spreadsheet or drafting a reply. A well-designed one creates drafts only, so a person reviews and sends everything that goes to a customer.
How accurate is AI at sorting business email? In our test of 50 realistic emails, Claude 5.5 Haiku running inside Zapier put 49 in the right category. Urgency was harder: a vague rule caught 4 of 10 urgent emails, while a rule listing concrete triggers caught all 10, with 3 false alarms. Results depend heavily on how clearly your rules are written, so score your own test set before you switch anything on.
Should an AI assistant send emails automatically? We advise against it. Drafts are usually good, but a model can occasionally get a name wrong, misread a date or strike the wrong tone. Leaving every reply and follow-up as a draft costs about ten minutes of review a day and means no customer receives a message nobody has read.
Can I use an AI inbox assistant with patient or client data? Only with care. Every email the assistant reads is sent to an AI model, so health or client data needs a legal basis and a proper data processing agreement before it goes through any AI tool. A common approach is to keep sensitive mail in a separate inbox that the assistant never reads.