Skip to content
Faidhi Fahmi.Contact
All selected works
AITechnicalData

Answering support before it becomes a ticket

Organisation
iLyF · Easy, Instant Insurances
Role
Co-founder & Chief Product Officer · AI support lead
Period
Malaysia

Built AI support across WhatsApp, Telegram, and web, with clear safety rules and human handoffs. Product signals help the team reach customers before they report a problem.

At a glance

Situation
Support volume was growing with the user base, and most of it was the same twenty questions. Most failures never became tickets at all. They became quietly closed apps.
Stakes
The options were hiring support linearly or answering automatically, and a bot that answers wrongly about a policy somebody paid for destroys more trust than it saves cost.
My role
Co-founder and Chief Product Officer. Led the assistant, curated knowledge base, escalation boundary and signal loop that reaches customers before they complain.
Constraints
Regulated product answers with zero tolerance for confident errors, three live channels, and detection built only from tools already on the bill.
What changed
  • 70–80% of routine tickets handled without a human, around the clock
  • Support headcount stayed flat while the user base grew
  • Failed payments and stuck quotes were reached proactively instead of waiting to be reported
Read this if
You want AI in support that deflects real work with guardrails, not another demo.
of routine tickets handled without a human
70–80%of routine tickets handled without a human
first response on every channel
24/7first response on every channel
support headcount as the user base grew
Flatsupport headcount as the user base grew
product events on one Segment schema, one identity
50+product events on one Segment schema, one identity
contact on friction and error signals, before a ticket existed
Outboundcontact on friction and error signals, before a ticket existed

How the system actually worked

The escalation boundary, the loop that closes it, and the lane that starts without the customerTwo ways in. Reactive: questions arrive on WhatsApp, Telegram and web. Retrieval runs over a curated knowledge base, not open generation. An escalation boundary then decides whether the assistant may answer at all. If there is a validated answer in scope, it answers, which covers seventy to eighty percent of routine tickets around the clock. If not, it escalates to a human with the full conversation attached, so the customer never restarts their story. Every escalation is logged as a knowledge-base gap and flows back into the curated content, which is what improved deflection, not prompt tuning. Proactive: a friction or error signal, assembled from one Segment identity, a Mixpanel drop-off and a Sentry exception, passes a bar for whether it is worth interrupting someone for. If it is, the system reaches out first, saying what happened, what was already fixed and the one step left; if not, it is logged rather than messaged. Whatever the customer replies arrives back on their usual channel and re-enters the same assistant.REACTIVE · CHANNELSWhatsAppTelegramWebRetrieval overcurated KBnot open generation1Escalation boundarymay it answer?2Answers fromvalidated content70–80% of routine tickets, 24/7in scopeHuman, with the fullconversation attachedthe customer never restartsout of scope3logged as a knowledge-base gap → written into the KB4PROACTIVE · THE SIGNAL LOOP STARTS IT, BEFORE ANYONE WRITES INFriction or errordetectedone Segment identity,a Mixpanel drop-off,a Sentry exception5Worthinterrupting for?6Reach out firstwhat happened, what we fixed,the one step left for themyesLogged, not messagedbelow the barwhatever they reply lands back in the same assistantThe curated knowledge basethe assistant's quality ceiling is set by the documentation, not the model

Scroll the figure sideways to read it, or turn your phone.

  1. Retrieval over curated content, not open generation. The assistant answers from validated material or it does not answer, which is what stops it saying something confident and wrong about a policy somebody paid for.
  2. The boundary is drawn explicitly: which classes of question it may attempt at all, decided in advance rather than discovered in production.
  3. An escalation carries the full conversation, so the customer never has to restart their story with a human.
  4. Every escalation is a knowledge-base gap, not a model failure. Closing those gaps moved deflection far more reliably than prompt tuning ever did.
  5. The signal is assembled, not observed in one place: Segment carries one identity into every tool, Mixpanel says where the customer stalled, Sentry says what broke underneath them. Neither analytics nor error tracking can name a person, a step and a cause on its own.
  6. Every signal we could detect was a signal we could message on, which is exactly the temptation to resist. A cool-down, a suppression list and a hard rule against interrupting a live human conversation keep the list of situations short.
Two ways into the same assistant. One path the customer starts, and retrieval decides whether we may answer at all. The other the signal loop starts, before anyone has typed a word. Everything the assistant could not answer became a knowledge-base gap, and that backlog, not prompt tuning, is what drove the next improvement.
Segment, Mixpanel and Sentry read as one signalThe app and web emit over fifty product events into Segment, which defines one event schema and one identity. Mixpanel shows where the customer stalled: funnel drop-off, retry loops and time on step. Sentry shows why: the exception, release or insurer call that timed out. Both converge on one identity record, naming the person, step and cause. The record ranks the engineering queue by customers affected rather than error count. It also passes through a gate for named situations such as payment failure, repeated quote attempts, a crash during purchase or an abandoned renewal. The system can then contact the customer on the channel they were already using. Everything else goes to a human with the session context attached. Cool-downs and suppression rules prevent unwanted messages.INSTRUMENTED ONCEApp and web50+ product eventsSegmentone event schema,one identity, fannedout to every tool1TWO READINGS OF ONE SESSIONMixpanelwhere they stalled:funnel drop-off, retries,time on stepSentrywhy it stalled:exception, release,the insurer call that timed outJoined on one identitythe person, the stepand the cause,in one record2Engineering queueranked by customersaffected, not byerror count4A namedsituation?3Deterministic rulespayment failure, quote retried 3x,crash mid-purchase,renewal opened and droppedReach out insidethe sessionon the channel they were on,carrying the fixStraight to a humanfull session context attachedanything elseCool-down · suppression list · never while a human is already mid-conversationthe bar for interrupting someone stayed deliberately high

Scroll the figure sideways to read it, or turn your phone.

  1. Segment is the unglamorous piece that makes the rest possible: one event schema defined once, so “quote started” and “payment failed” mean the same thing in every tool and carry the same identity. Wire Mixpanel and Sentry into the app separately and there is nothing to join a year later.
  2. A funnel drop tells you people left, not whether the product broke. An exception tells you something broke, not whether anyone abandoned because of it. Joined on one identity, they name the person, the step and the cause in the same record.
  3. The list of situations worth interrupting someone for stayed deliberately short. Anything the rules could not name went to a human with the session attached rather than being guessed at.
  4. Same stream, a second use: the engineering queue ranked by customers affected rather than by error count, which is a different order from the one an error tracker sorts by.
The detection half. Segment is the unglamorous middle. Without one schema and one identity, a Mixpanel drop-off and a Sentry exception are two dashboards describing strangers. The short list of named situations is deliberate. Being able to see a problem does not always justify messaging the customer.

Stack & practices

  • OpenAI
  • Retrieval over curated KB
  • Segment
  • Mixpanel
  • Sentry
  • WhatsApp
  • Telegram
  • Web

Got a problem shaped like this one?

A short conversation is usually enough to see if there's a fit.

Email meRead another case study