We deleted our bug report form and let the chatbot do it instead
How MCAFax turns "hey this isn't working" into a triaged, deduplicated, severity-rated ticket — and the exact prompt you can paste into your own AI helper to build the same thing.
The ticket that never got written
Here's the thing nobody tells you when you're building software alone.
Your users are telling you what's broken. Constantly. They're just not telling you in the place you built for it.
We had a "Report an issue" link. Bottom of the dashboard, small text, you had to scroll. In the entire time it existed, we got a handful of submissions. Meanwhile I'd go read chat logs and find seven people in one week saying some version of "it's not letting me send" — and not one of them clicked the link. Not one.
Because that's not how anyone actually behaves. Nobody who is annoyed at your product stops being annoyed, opens a new tab, fills out a form with a dropdown for "issue category," and writes a clear reproduction case. That's a thing engineers imagine users do. What users actually do is complain in the nearest text box to their face and then leave.
For us, the nearest text box was the AI chat assistant on the dashboard. So we stopped fighting it. We deleted the form and taught the chat to file the ticket.
That change did more for our bug backlog than any process I've tried. Not because the tickets got better — because they started existing at all.
This post is the whole build. Nothing held back. The severity model, the deduplication logic, the data model, the security rules, the deploy steps, the stuff that broke. And at the bottom, the complete prompt we use to install this thing into a new project, which you can copy and run against your own codebase.
Why "just add a form" is the wrong instinct
Two reasons, and the second one is the one people miss.
One: the form loses reports. Covered above. Friction between "I'm annoyed" and "I filed something" kills the report. Chat has zero friction because they were already typing.
Two: the form produces garbage even when it works. A form gets you whatever the user chose to type into a box, which is usually four words and no context. A chat assistant is already in the conversation. It knows what page they're on, what they were trying to do three messages ago, what plan they're on, whether they already tried the obvious fix. It can ask one clarifying question in a way that doesn't feel like paperwork.
So the report gets filed AND it's a better report. That's the whole pitch.
There's a third thing that only showed up after we ran it a while: the assistant resolves a big share of "bugs" before they become tickets. A real chunk of what looks like a defect is a user error or a known behavior. The assistant answers it, and no ticket gets created. Your queue ends up containing mostly real problems. That alone was worth building it.
The shape of the thing
Three pieces. That's it.
- Detection + a tool in the chat. The assistant notices a complaint, tries to help first, and if it's a real defect, calls a tool called
report_issue. - A server-side handler that does the actual thinking: computes severity, checks whether this is a duplicate, and either creates a ticket or appends to an existing one.
- An admin queue where I work the tickets.
The important architectural decision, and I'd fight anyone on this one: the model reports observations, the server makes decisions.
The model does not pick the severity. The model does not decide what's a duplicate. The model supplies structured signals — is this blocking, does a workaround exist, does it touch money, does it touch data, does it touch security, how frustrated is this person — and then deterministic server code turns those into a rating.
Why: because if you let the model rate severity directly, it drifts. Same bug, two conversations, two different ratings, depending on how upset the user sounded. Then your queue sort order is meaningless and you stop trusting it. Deterministic scoring off model-supplied signals gets you consistency without giving up the model's ability to read a messy conversation.
The severity system
Four levels. We call them P1 through P4 because that's what everyone calls them and inventing new names for this would be a waste of everyone's time.
| Level | Name | What it means |
|---|---|---|
| P1 | Critical | Security or data exposure, money moving wrong, or the core action of the product is broken for everybody. |
| P2 | High | User is blocked with no workaround, or it affects money or data for a single user. |
| P3 | Medium | Blocked but there's a workaround, or the behavior is degraded and confusing. |
| P4 | Low | Cosmetic, copy, minor annoyance, feature request wearing a bug costume. |
The scoring rules run in this order, server-side:
affects_security→ P1, always, no exceptions, no discussion.affects_moneyOraffects_data→ P2 floor.blockingAnd no workaround → P2 floor.blockingWith a workaround → P3 floor.- Everything else → P4.
"Floor" matters — a later rule can raise it, nothing lowers it.
Auto-escalation is the part that actually earns its keep
Static severity is a snapshot of one person's bad afternoon. What you actually want to know is how many people are hitting this.
So every time a new report merges into an existing open ticket, the count goes up. And:
- 3 Distinct users → bump one severity level
- 8 Distinct users → bump another
- Caps at P1
Every bump gets logged with a reason and a timestamp, so when I open a ticket that says P2 I can see it started at P4 and climbed because eleven people hit it.
That's the thing I actually look at now. Not severity. The occurrence count next to it. One person complaining is noise, and I say that with love. Eleven people complaining in a day is a deploy that broke something.
Admins can override severity manually. That sets a severityLocked flag and freezes auto-escalation on that ticket, so my judgment doesn't get overwritten by a counter.
Deduplication — the part that's genuinely hard
If you don't solve this, the system eats itself. One bad deploy, forty users hit it, and you wake up to forty tickets that all say the same thing and you can't see anything else in the queue.
We do a two-stage match. Before creating anything, the handler pulls open tickets in the same product area, from the last 30 days, with status open or in_progress.
Stage one — fingerprint. Normalize the area, the route or page, and a stripped-down symptom string. Lowercase it, strip out numbers, IDs, names, and anything in quotes. If the fingerprints match exactly, merge. This is cheap, it's instant, and it catches most of the volume — because forty people hitting the same broken button on the same page produce nearly identical fingerprints.
Stage two — semantic. No fingerprint match, so we take the new title and description plus the titles of the candidate open tickets, send them to the model in one cheap call, and ask: is any of these genuinely the same problem, or is this new? Return an ID or return null.
And here's the rule that matters most in the entire system:
When unsure, create a new ticket.
A duplicate ticket costs me thirty seconds to merge by hand. A wrong merge silently buries a real bug inside an unrelated one, and I never see it again. Those are not symmetric mistakes, so we tune the whole thing toward over-creating. Be biased toward the cheap error.
Merging never overwrites the original. It appends an occurrence record — this user, their description, their context, the timestamp. The original ticket's description stays as written. Then the escalation check re-runs.
This matters for a reason that isn't obvious until you've done it wrong: the second and fifth and twelfth reports often contain the detail that makes the bug reproducible. If you throw them away because "we already have that one," you throw away your best diagnostic material.
How the assistant is supposed to behave
The system prompt additions, more or less as we run them:
Notice complaints that aren't labeled as complaints. People don't say "I would like to report a defect." They say: this isn't working, it won't let me, I keep getting an error, nothing happens when I click, it's been stuck since yesterday, did you guys change something. Plus the pure-frustration message with no actual request in it.
Try to help first. If it's user error or known behavior, solve it and file nothing. Not every complaint is a bug and a queue full of non-bugs is a queue you stop reading.
Ask at most two clarifying questions. What were you trying to do, and what happened instead. That's it. If you interrogate someone who's already annoyed, they leave, and now you've made it worse than the form.
Confirm before writing. Show a one-line summary of what's about to get filed and get a yes. We already had a confirm-before-action pattern for every other tool in our assistant, and this follows it. Users should never find out later that a chat message became a database record.
Close the loop like a human. Give them the ticket ID in plain language. If it merged into an existing ticket, say so — "a few other people have hit this too, it's already flagged and being worked on." That's better than a new ticket number, not worse, and it should feel that way. Never let it read like their report got tossed.
Never promise a fix date. I don't care how easy it looks.
The tool
One tool on the assistant, following whatever tool-calling pattern your codebase already uses.
name: report_issue
description: File a user-reported problem, or attach this report to an
existing open ticket describing the same problem.
input:
area: enum from config (your product areas)
title: short imperative summary, max 80 chars, no user PII
description: what they were doing, what they expected, what happened
severity_signals: {
blocking: bool // cannot complete the task at all
workaround_exists: bool
affects_money: bool // billing, payments, anything financial
affects_data: bool // loss, corruption, wrong data shown
affects_security: bool // exposure, auth bypass, wrong-user data
reported_frustration: 0-3
}
user_context: { userId, plan/role, page/route, browser, timestamp }
Note what's not in there: no severity field. The model can't set it. That's the point.
The data model
One collection. Ours is Firestore, but this maps to any database.
id
businessKey
area
title
description
severity
severityLocked
severityHistory[]
status // open | in_progress | waiting_on_user
// | resolved | wont_fix | duplicate
fingerprint
occurrences[] // { userId, description, userContext, at }
occurrenceCount
distinctUserCount
createdAt
lastReportedAt
resolvedAt
assignedTo
internalNotes[]
sourceConversationId
mergedInto // nullable
Three rules around it:
Server-side writes only. Clients never write tickets directly. Lock it in your security rules. If a client can write to this collection, a client can write P1 tickets all day, and your queue is now a spam target.
Index it. You'll want (status, severity, lastReportedAt) and (area, status, createdAt). If you're on Firestore you'll find out you need these when your queue query fails with a link to create the index — just click the link, it builds it for you.
Strip PII on the way in. Store the user ID, not an email dump. And run a scrub on the description field before write — people paste things into chat. Card numbers, account numbers, passwords, all of it. If your ticket table becomes a place where sensitive data accumulates by accident, you've built a liability instead of a tool.
The admin queue
Nothing fancy. A table, sorted by severity descending, then most recently reported.
- Severity as a colored pill: P1 red, P2 orange, P3 yellow, P4 gray.
- Filters on severity, status, area, date range. Search on title.
- A count badge on anything with more than one occurrence. This is the single most useful element on the screen. It's how you spot a bad deploy in about four seconds.
- Ticket detail shows: the original description, every occurrence in chronological order with its user context, the full severity history including auto-bump reasons, internal notes, and controls for status, severity override, assignment, and manual merge.
- Full edit and delete from the UI.
That last one is not a nice-to-have. If fixing a mislabeled ticket requires opening the database console, you will not fix mislabeled tickets. I learned that the hard way on a different part of this product — anything I can't do from my own admin screen effectively doesn't get done.
If you wire up a notification target, fire it on new P1s and on anything that auto-escalates into P1. Nothing else. The second you get notified about P3s you start ignoring notifications.
Setting it up
Here's the actual sequence. I'm writing this for someone who is not a career engineer, because I'm not one either.
1. Run the prompt. Paste the prompt at the bottom of this post into your AI coding assistant inside your project's repo. It starts by interviewing you — your stack, your product areas, whether you already have a chat assistant, where your admin screens live. Answer honestly, and correct it when it guesses wrong about your codebase. Don't let it start coding until the summary it gives you matches what you actually want.
2. Give it your product areas. This is the one input that determines whether the whole thing is useful. Five to twelve functional areas of your product that a complaint could be about. For us that's things like sending, broker lookup, letters, account, billing. If your areas are too broad ("app," "website") your dedup and filtering are worthless. Too narrow and the model can't pick reliably. Aim for the list you'd use to describe your product to someone in one minute.
3. Configure it. Everything business-specific lives in one file — ticketing.config.js. Business name, product-area enum, severity thresholds, dedup window, notification target, admin auth check, collection names, assistant tone. Nothing business-specific goes anywhere else. That's what makes it portable.
4. Deploy the backend. If you're on a serverless setup like ours, frontend changes deploy on push but backend functions need an explicit deploy command. These are two separate paths and it is extremely easy to push your code, see your UI change, assume you're done, and spend an hour confused about why the tool call is failing. Deploy the backend. Then check.
5. Create the indexes. Run a query against the queue. If it fails, your database will usually hand you a link that creates the index it wants. Click it, wait a minute or two for it to build.
6. Lock the security rules. Confirm clients can't write to the tickets collection. Ask your assistant to show you the rules diff and read it yourself.
7. Set up admin access. Whatever gate your admin app already uses — a custom claim, a role, an allowlist. Reuse it. Don't invent a second auth system for this.
8. Seed test tickets. One at each severity level, and one with five occurrences on it. You want to see the queue populated before real tickets show up, because that's when you'll notice your severity colors are unreadable or your sort is backwards.
9. Test the whole loop yourself. Open the chat as a normal user. Complain about something. Watch it ask its questions, watch it confirm, watch the ticket land. Then complain about the same thing from a second account and confirm it merges instead of creating a second ticket. That second test is the one people skip and it's the one that matters.
What broke for us
Being straight with you about this, because every build-in-public post that claims it went smoothly is lying.
Dedup was too aggressive at first. We tuned the semantic matcher to be helpful and it started merging things that were merely adjacent — two different failures on the same page became one ticket. We lost a real bug for about a week inside another ticket. Fixed by requiring a confident match and defaulting to create. Over-create. Always.
Severity got gamed by tone. Early version let the model weigh frustration heavily. Turns out the loudest user does not have the worst bug. Frustration is still captured as a signal — it's useful context when I'm deciding what to work on — but it no longer moves the rating on its own.
The assistant filed tickets for things it should have just answered. First week it was trigger-happy — every question that contained the word "can't" became a ticket. Fixing this was a prompt problem, not a code problem: try to help first, and only file if it's genuinely a defect.
We didn't strip PII initially. Somebody pasted a full application into chat while describing a problem, and it went into a ticket description verbatim. Caught it fast, but that's exactly the kind of thing that quietly turns a helpful internal tool into a data-handling problem you didn't sign up for. Build the scrub in from day one.
Making it portable
I run more than one business. I did not want to build this four times.
So the whole thing lives in one folder on the backend and one on the frontend, and every business-specific value lives in that single config file. Installing it into the next project is: copy the two folders, edit the config, deploy, create indexes, done. The tool handler, the dedup logic, the severity scoring, and the admin UI never change between businesses.
If you're building anything for yourself more than once, do this from the start. The discipline of "nothing business-specific outside the config file" costs you almost nothing while you're writing it and saves you a rewrite every single time you reuse it.
The prompt below is written to enforce that, so if you run it as-is you get the portable version by default.
The prompt
Paste this into Claude Code or your AI assistant of choice, inside the repo of the project you want to add this to.
# BUILD PROMPT — AI Chat Support Ticketing System (Portable)
## STEP 0 — INTERVIEW ME FIRST (do not write any code yet)
Before building anything, read my codebase and then ask me these questions one
batch at a time. Do not assume answers. Do not start coding until I've answered.
1. Business identity — What is this business called, what does it do, and who
are its users (customers, staff, both)?
2. Stack — Confirm what you found: frontend framework, backend/runtime,
database, hosting, auth. Tell me what you detected and let me correct you.
3. Existing chat — Is there already an AI chat assistant in this app? If yes,
show me the file(s) and the system prompt so you extend it rather than build
a second one. If no, you'll scaffold one.
4. Product areas — What are the 5–12 functional areas of this product that a
complaint could be about? (e.g. login, billing, sending, search, uploads,
notifications.) I'll give you the list; you'll use it as the ticket `area`
enum.
5. Who works tickets — Is there an existing admin/internal app? Where should
the ticket queue live? Do admins already have an auth gate / custom claim I
should reuse?
6. Notification — Where do I want to be told about a Critical ticket: email,
SMS, Slack, or just the admin queue?
7. AI provider — Which model/API key is already wired up in this project, and
is there a secret manager already in use?
Then summarize what you're about to build in one screen and wait for my go.
## STEP 1 — WHAT TO BUILD
A support ticketing system driven by the AI chat assistant, not by a "Submit a
bug" form. When a user complains in chat that something isn't working, the
assistant handles it conversationally and quietly turns it into structured data.
Three pieces:
A. Chat-side complaint detection + ticket tool
B. Ticket store with dedup and severity
C. Admin ticket queue UI
Build them in that order, and make each one work end-to-end before moving on.
## STEP 2 — PORTABILITY REQUIREMENT
This is being installed across several different businesses of mine. So:
- Put every business-specific value in ONE config file — ticketing.config.js
(or the equivalent for the detected stack) — at the root of the ticketing
module. That file holds: business name, product-area enum, severity
thresholds, notification target, admin auth check, collection or table names,
and the chat assistant's tone.
- Everything else must be generic. No business logic hardcoded in the tool
handler, the dedup logic, or the UI.
- Keep the whole thing inside a single folder (e.g. /ticketing on the backend
and /ticketing on the frontend) so I can copy that folder into the next
project and only edit the config file.
- Write a short INSTALL.md in that folder: what to change in the config, what
env vars/secrets are needed, what to deploy, and how to grant admin access.
Assume the person reading it is me and I'm not a deep tooling expert — give
exact file paths and exact terminal commands.
## STEP 3 — COMPLAINT DETECTION IN CHAT
Add to the assistant's system prompt:
- Users often report problems mid-conversation without labeling them as bug
reports. Watch for: "this isn't working", "it won't let me", "I keep getting
an error", "nothing happens when I click", "it's been stuck since yesterday",
"did you guys change something", plus frustration with no explicit ask.
- When that happens: FIRST TRY TO ACTUALLY HELP. If it's a known behavior or
user error, resolve it and do not open a ticket.
- If it looks like a real defect, ask at most TWO clarifying questions — what
they were trying to do, and what happened instead. Never interrogate.
- Then call the ticket tool.
- CONFIRM BEFORE WRITING. Show the user a one-line summary of what you're about
to file and get a yes. This matches the existing confirm-before-action
pattern in my chat assistant — follow it exactly.
- After filing, tell them the ticket ID in plain language and what happens next.
If it was merged into an existing ticket, say so — "a few other people have
hit this too, it's already flagged" — never make it sound like their report
was discarded.
- Never promise a fix date.
## STEP 4 — THE TOOL
Add one tool to the assistant, following the same tool-calling pattern already
in this codebase.
name: report_issue
description: File a user-reported problem, or attach this report to an
existing open ticket describing the same problem.
input:
area: enum from config (product areas)
title: short imperative summary, max 80 chars, no user PII
description: what the user was doing, what they expected, what happened
severity_signals: {
blocking: bool // user cannot complete the task at all
workaround_exists: bool
affects_money: bool // billing, payments, anything financial
affects_data: bool // loss, corruption, wrong data shown
affects_security: bool // exposure, auth bypass, wrong-user data
reported_frustration: 0-3
}
user_context: { userId, plan/role, page/route, browser, timestamp }
The tool handler — not the model — computes severity and does dedup. The model
supplies observations; the server decides.
## STEP 5 — SEVERITY RATING
Four levels. Server-side scoring, deterministic, defined in config so I can
retune per business.
P1 Critical — Security or data exposure, money moving incorrectly, or the
product's core action is broken for everyone.
P2 High — Blocking with no workaround, or affects money/data for one user.
P3 Medium — Blocking but a workaround exists, or degraded/confusing behavior.
P4 Low — Cosmetic, copy, minor annoyance, feature request in disguise.
Scoring rules:
- affects_security → P1, always.
- affects_money or affects_data → P2 floor.
- blocking && !workaround_exists → P2 floor.
- blocking && workaround_exists → P3 floor.
- else → P4.
- AUTO-ESCALATION: every time a duplicate is merged into an open ticket,
increment occurrenceCount. At 3 distinct users, bump one level. At 8 distinct
users, bump one more. Cap at P1. Log every auto-bump with reason and timestamp
so I can see why a ticket moved.
- Admins can manually override severity; a manual override sets
severityLocked: true and freezes auto-escalation for that ticket.
## STEP 6 — DEDUP (create new vs. add to existing)
Before creating anything, the handler must check open tickets in the same area,
with status open or in_progress, from the last 30 days (window in config).
Two-stage match:
1. FINGERPRINT — normalize area + route/page + a stripped-down symptom string
(lowercase, drop numbers, IDs, names, quotes). Exact fingerprint match → merge.
2. SEMANTIC — if no fingerprint match, send the new title/description plus the
candidate open tickets' titles to the model in a single cheap call and ask
for the ID of a genuine duplicate or null. Require a confident match; default
to creating a new ticket when unsure. A wrong merge hides a real bug — a
duplicate ticket is cheap, a swallowed report isn't.
On merge, do NOT overwrite the original ticket. Append an occurrence record:
occurrences: [{ userId, description, userContext, at }]
occurrenceCount: n
distinctUserCount: n
lastReportedAt: timestamp
Then re-run the severity escalation check.
## STEP 7 — DATA MODEL
One collection/table, `tickets`:
id, businessKey, area, title, description,
severity, severityLocked, severityHistory[],
status: open | in_progress | waiting_on_user | resolved | wont_fix | duplicate,
fingerprint,
occurrences[], occurrenceCount, distinctUserCount,
createdAt, lastReportedAt, resolvedAt,
assignedTo, internalNotes[],
sourceConversationId,
mergedInto (nullable)
Rules:
- Write only through the server-side handler. Clients never write tickets
directly — lock this down in the security rules and show me the rules diff.
- Index on (status, severity, lastReportedAt) and on (area, status, createdAt).
Tell me the exact command or console steps to create them.
- Store no more PII than needed: user ID, not email dumps; strip anything that
looks like a card number, SSN, or password out of description before write.
## STEP 8 — ADMIN QUEUE
In the admin app (or a new admin-gated route if none exists):
- Table of tickets, default sort: severity desc, then lastReportedAt desc.
- Severity shown as a colored pill (P1 red, P2 orange, P3 yellow, P4 gray).
- Filters: severity, status, area, date range. Search on title.
- A visible count badge on any ticket with occurrenceCount > 1 — this is the
signal I actually care about.
- Ticket detail: full description, every occurrence with its user context in
chronological order, severity history with auto-bump reasons, internal notes,
and controls to change status, override severity, assign, and merge into
another ticket manually.
- Full edit and delete from this UI — I never want to open the database console
to fix a ticket.
- If config has a notification target, fire it on any new P1 and on any
auto-escalation into P1.
## STEP 9 — HOW TO WORK
- Build A, then B, then C. After each, stop, tell me exactly what to run to test
it locally, and wait for me to confirm it works before continuing.
- Any backend/function changes: remind me they need an explicit deploy command
and give me the exact command. Frontend changes deploy on push — keep the two
paths clearly separate in your instructions to me.
- Back up any file before you rewrite it, and run a syntax check after.
- Don't touch unrelated files.
- At the end, give me a checklist of everything that must be true for this to
work in a fresh project, and one seeded test ticket at each severity level so
I can see the queue populated.
Coming next: tickets that fix themselves
Everything above still ends with a human — me — opening a queue and doing the work.
The next piece we're building is the part where the AI reads its own tickets and closes the ones it can. Not every ticket. But a real share of what lands in that queue is something a model with repo access and the right guardrails can diagnose and patch: a broken link, a copy error, a missing null check, a validation rule that's stricter than it should be, a config value that drifted.
The interesting problems there aren't the fixes — they're the guardrails. What is an AI allowed to touch without a human looking. How do you prove a fix actually fixed the reported behavior and didn't just make the error message go away. What happens when it's wrong. How does the person who filed the ticket find out it's resolved.
That's the next post. We'll publish the same way — the full approach, the failures, and the prompt.
If you get stuck
The prompt above is genuinely everything. Run it and you should get a working system.
But "run this against your codebase" assumes you have a codebase, a deploy path, a database with security rules, and an admin app to hang a queue on. Plenty of people reading this have the first one and not the rest, and that's usually where it stalls — not the ticketing logic, the infrastructure underneath it.
If that's you, email info@mcafax.com. Tell me what you're running and where it broke. I've set this up more than once and I'm happy to help you get the plumbing right — we're not selling infrastructure consulting, I just think more small operators should have tools like this and the setup step shouldn't be the thing that stops you.
Take the prompt. Build it. It's yours.