How to Use Confidence-Based Approval to Control What Your AI Sends Automatically

The most common question about AI email handling is not "can it write a good reply?" It is "what happens if it gets one wrong?" That question is fair, and a serious solution has to answer it with more than reassurance. The answer lies in how much freedom the AI has for each kind of message, and who decides.
Confidence-based approval is one way to organise this. Instead of a single switch between "a person reads everything" and "the AI sends everything", it lets the system's own level of certainty influence what happens next. This article explains the idea, how to set it up sensibly and how to keep it under review. It describes how InboxFirma approaches the topic and what you should agree on during setup. It does not promise specific accuracy figures, because those depend on your mail and your configuration.
The basic idea
Every incoming email can be judged on two things: what kind of request it is and how sure the system is that it understands the request and can answer it properly. The second part is the "confidence". It is an estimate of certainty, not a guarantee of correctness. A high value means the message closely resembles cases the system has handled well, with clear information to answer from. A low value means the message is unusual, ambiguous or missing something.
Confidence-based approval connects that estimate to a decision about what happens next. In InboxFirma, the principle has three outcomes:
- High confidence: the reply may be sent automatically, according to the rules you have set.
- Medium confidence: the reply is prepared and submitted to a person for approval.
- Low confidence or an ambiguous case: the message is routed to a person with a note, and no automatic reply is sent.
Alongside this sits a rule that does not depend on confidence at all: legal and sensitive mail, such as letters from lawyers, authorities or the police, is routed to a human and is never answered automatically, however sure the system might be.
Why not just approve everything?
Manual approval of every reply is the safest setting, and it is a sensible place to start. But it has a cost. If a person has to read and click through hundreds of routine replies, the time saved by drafting is partly spent on checking. The queue gets shorter, but the team still handles each message.
Confidence-based approval aims to focus people's attention where it is needed. Routine messages with clear answers need less attention. Unusual ones need more. In a well-configured setup, people spend their time on the cases that benefit from judgement, rather than on confirming the hundredth identical receipt acknowledgement.
Why not send everything automatically?
The opposite extreme has an obvious weakness. A system can be confidently wrong, and a wrong reply sent without review may reach a customer before anyone notices. Some categories also carry consequences that make automatic sending inappropriate regardless of confidence, for example commitments about money, contract terms or anything with legal weight.
That is why confidence is one input, not the whole rule. The categories, the conditions and the exceptions are decided by people.
Step 1: Start with approval everywhere
When you begin, set every category to approval. The AI prepares replies and your team checks and sends them. This phase is not wasted time. It gives you:
- a real picture of how good the drafts are for your mail,
- information about which categories are reliable and which need work,
- and a team that has seen what the system does and does not do.
Record how often a draft is sent unchanged, how often it is edited and how often it is rejected. These numbers drive the next steps.
Step 2: Decide which categories could ever be automatic
Not every category should be a candidate for automatic sending. Go through them and ask:
- What is the worst realistic outcome of a wrong reply? A wrong opening-hours answer is a minor inconvenience. A wrong statement about a contract is a different matter.
- Is the answer fixed or does it require judgement? A confirmation of receipt is fixed. An assessment of a complaint is not.
- Does the reply commit you to something? Anything that promises, refuses or changes something should be treated with more caution.
- Does it depend on data the system may not have? If the answer relies on information outside the email, such as a status in another system, automatic sending only makes sense where that data is connected reliably.
Typical candidates for eventual automatic sending are simple acknowledgements, standard information that does not change, and clearly structured requests. Typical non-candidates are complaints, legal topics, disputes, pricing commitments and unusual requests.
Step 3: Set the thresholds together
For categories that are candidates, the next question is how sure the system has to be before it sends on its own. This is a configuration decision, and it should be taken deliberately:
- A higher threshold means fewer automatic replies and more approvals. It is more cautious.
- A lower threshold means more automation and more reliance on the system's judgement.
There is no universal number. The right level depends on your categories, your tolerance for error and how well the drafts have performed in the approval phase. In InboxFirma, thresholds are agreed with you during setup, and they can differ by category. They can also be adjusted later.
The interactive demo of confidence-based sending on the how it works page illustrates the principle with a simulated inbox, including a confidence display and the three outcomes. Its percentages, timings and example data are illustrative and are labelled as a simulation. They are not claims about the accuracy or settings of a live system.
Step 4: Decide whether replies wait before sending
Some teams like to add a short delay before an automatic reply leaves, so that a person can stop it if something looks wrong. The simulation on the how-it-works page shows this idea with a countdown and a stop button. Whether such a delay is used in your setup, and how long it should be, is part of the configuration discussion. It is a trade-off between speed and an additional safety margin, and different teams make different choices.
Step 5: Keep hard rules independent of confidence
Some rules should never be loosened by confidence. Write them down explicitly:
- Legal and sensitive mail always goes to a person.
- Certain topics, such as complaints with potential consequences or anything involving authorities, always go to a person.
- Replies that make financial or contractual commitments need approval.
- If a message contains an attachment the system cannot interpret reliably, it goes to a person.
These rules protect against the cases where a confident answer is still the wrong kind of answer.
Step 6: Review what was sent automatically
Automatic does not mean unobserved. Build a habit of sampling what the system has sent on its own:
- Look at a selection of automatically sent replies each week at the beginning, then less often as trust builds.
- Pay particular attention to replies in newly automated categories.
- Check cases where the customer answered with a complaint or a correction. These are early warning signs.
- Compare corrections made in the approval queue with the replies that went out automatically.
A monthly report on automation rate, time saved and volume helps to see the overall picture, but it does not replace reading actual examples. The point of review is to catch drift early: a category that worked well may start to fail when your business changes a policy or a product.
Step 7: Adjust, in both directions
The configuration is not a one-way street. Review it regularly and be willing to move categories either way:
- Towards more automation: a category with consistently good drafts, few edits and no complaints may move from approval to automatic sending, or the threshold may be lowered slightly.
- Towards more control: a category that produces corrections, a change in your offering or a new type of request may need to move back to approval.
Treat both directions as normal. A team that feels free to tighten a rule when it is not sure ends up with a more trustworthy setup than one that treats automation levels as a one-way ratchet.
Step 8: Assign ownership
Someone needs to own the rules. In practice that means:
- a named person who decides on thresholds and categories,
- a way for team members to flag drafts that were wrong, so that the feedback reaches that person,
- and an agreed rhythm for review, for example monthly.
Without ownership, thresholds tend to stay where they were set on day one, regardless of how experience develops.
What confidence-based approval cannot do
It is worth being clear about the limits:
- It is not a guarantee. A system can be confident and still wrong. That is why hard rules, sampling and review matter.
- It does not replace judgement. Cases that need a person should be routed to a person, whatever the confidence.
- It does not work without good inputs. If the system lacks the information needed to answer, a high confidence is not meaningful. Good knowledge, good examples and, where needed, connected data are part of the setup.
- It does not decide for you. The categories, thresholds and exceptions are your decisions. The software applies them consistently.
A simple example of how rules might look
To make this concrete, here is a fictional example, purely for illustration:
- Confirmations of receipt: automatic when confidence is high; otherwise approval.
- Questions about opening hours and contact persons: automatic when confidence is high; otherwise approval.
- Order or case status: approval at first; automatic later only where the data is connected and reliable.
- Changes to personal or contract data: always approval, and where an integration exists, any change in the connected system follows your approval rules.
- Complaints, legal topics and anything involving authorities: always a person, never automatic.
Your own list will look different. The structure, however, tends to stay the same: some categories can become automatic over time, some always need approval and some always go to a person.
Getting started
If you want to see how this could look for your mail, begin with a simple inventory of your categories and a first view on which ones could one day be automatic. Our overview of the AI email automation features explains what the system does, and the 30-day AI email pilot is a structured way to start with approval and learn from real mail. You can also talk to us about how the rules would be configured for your team.
The aim is simple: automation that you can explain to your colleagues and your customers, because you decided what it may do and you can see what it did.


