master-ai/SENTRY_AS_PRODUCT.md

# Sentry-as-Product — Proposal

> Today's Sentry wiring catches errors in **the Vibn platform**.
> The bigger opportunity is wiring Sentry into **every project Vibn
> ships**, then feeding those errors back into the user's AI chat.
> Difference between "an AI that codes" and "an AI that owns the
> product."

## TL;DR

Today, when a Vibn user's deployed app crashes for real users:

```
real user → site 500s → user closes tab, never tells founder
                    → founder finds out hours/days later (or never)
                    → AI in Vibn chat has zero idea anything is wrong
```

The fix is to make every Vibn project ship with Sentry pre-wired,
then expose the error feed to the AI as a tool. Total effort:
**~8 hours**, in 4 stages, each independently shippable.

| Stage | Capability | Effort | Unlocks |
|---|---|---|---|
| 1 | Auto-provision a Sentry project per Vibn project on first deploy | ~3 hr | Real-user errors captured at all |
| 2 | Bake Sentry into every scaffold template | ~2 hr | Capture works without user setup |
| 3 | Add `project_recent_errors` MCP tool for the AI | ~2 hr | AI can answer "is anything broken?" |
| 4 | Auto-surface unresolved errors at chat-turn start | ~1 hr | AI proactively offers fixes |

Total: **~8 hr**, no new infra (we already have Sentry org access,
Coolify env API, scaffold templates, MCP tool registry).

---

## Why this is the right next investment

### The current loop is broken at the seam between user and platform

Vibn's value proposition is "the AI is your technical co-founder."
That promise breaks the moment the AI's last commit causes a real
user error and the AI doesn't know about it. The current loop:

```
1. User describes feature in chat
2. AI ships code
3. AI says "deployed, give it a try"
4. (silence)
5. Real users hit edge cases → 500s → bounce
6. Founder eventually notices via support ticket / analytics dip
7. Founder pastes error back to AI
8. AI fixes
```

Steps 4–6 are dead air for the founder, **and the AI cannot help
during them.** This is the gap that separates Vibn from "any IDE
with an LLM."

### What it looks like with this proposal shipped

```
1. User describes feature in chat
2. AI ships code
3. AI says "deployed, give it a try"
4. Real users hit edge cases → 500s → Sentry captures
5. (Founder opens Vibn chat 3 hrs later for unrelated reason)
6. AI: "Hey — checkout has 500'd for 3 users in the last hour
        because `customer.email` is undefined on
        app/checkout/route.ts:47. Want me to fix it?"
7. AI fixes, deploys, marks issue resolved in Sentry
```

The AI becomes the on-call engineer. This is what "technical
co-founder" actually means and we are 8 hours away from it.

### Why now (not Phase 4)

- The Sentry wiring we just shipped for vibn-frontend gave us:
  - A working Sentry org (`vibnai`)
  - An auth token with project-management scope
  - Verified knowledge that the build args / source maps flow works
  - A working `withSentryConfig` recipe in `vibn-frontend/next.config.ts`
- All of those are reusable for stage 1 and 2 of this proposal.
- Doing this **before** the beta means user projects start emitting
  error data on day one, so by the time we're debugging real beta
  user pain, we have a month of history to reason about.
- Doing it after the beta means we'd have to retroactively
  instrument projects that have already been deployed for weeks.

---

## Stage 1 — Auto-provision a Sentry project per Vibn project (~3 hr)

**Goal:** when a user creates a Vibn project, the platform creates a
matching Sentry project under the `vibnai` org and stashes the DSN
+ auth token in Coolify env vars on the user's app.

**What gets built:**

1. **A `provisionSentryProject(projectId, name)` helper** in
   `vibn-frontend/lib/integrations/sentry.ts`. Calls Sentry's
   `POST /api/0/teams/vibnai/{team}/projects/` with the project
   slug, returns the DSN.
2. **Hook into project-create flow** — on first successful deploy,
   call the helper and write the resulting DSN + auth token into
   Coolify env vars (`NEXT_PUBLIC_SENTRY_DSN`,
   `SENTRY_AUTH_TOKEN`) for that app via the same Coolify API we
   used today.
3. **Idempotency** — if the Sentry project already exists, fetch
   its DSN instead of creating a duplicate. Same project name
   convention every time: `vibn-{workspace}-{projectSlug}`.
4. **Storage** — store `sentryProjectSlug` and `sentryAuthTokenId`
   on the Postgres `projects` row so we can look them up later
   without re-walking the Sentry org.

**Risk:** Sentry's API rate-limits team-project creation. We bypass
this by reading-before-writing, so the only API cost on subsequent
deploys is one GET.

**Definition of done:** create a fresh Vibn project → check Sentry
org → see a project named `vibn-{ws}-{slug}` → check Coolify env on
that app → see DSN populated.

---

## Stage 2 — Bake Sentry into every scaffold template (~2 hr)

**Goal:** every Next.js / Vite / etc. starter template Vibn ships
already has Sentry wired up. User does nothing.

**What gets built:**

1. **For each scaffold template in `vibn-frontend/lib/scaffold/`**,
   add the same files we shipped today:
   - `instrumentation.ts`
   - `instrumentation-client.ts`
   - `app/global-error.tsx` (Next.js) / equivalent boundary (Vite)
   - `next.config.ts` wrapped with `withSentryConfig` (Next.js)
   - `vite.config.ts` with `sentryVitePlugin` (Vite)
   - `Dockerfile` ARG declarations for `NEXT_PUBLIC_SENTRY_DSN` +
     `SENTRY_AUTH_TOKEN`
2. **Add `@sentry/nextjs` (or `@sentry/react` + `@sentry/vite-plugin`)
   to each template's `package.json` `dependencies`.**
3. **Document in template README** that Sentry is pre-wired and the
   user doesn't need to do anything.

**Risk:** Sentry's wrapper sometimes interacts badly with custom
build configs (e.g. monorepos, custom webpack rules). Mitigation:
the `errorHandler` we set today (`console.warn` instead of throw)
ensures source map upload failures don't break builds.

**Definition of done:** scaffold a fresh Next.js project from Vibn
templates → deploy → throw a test error → see it in Sentry,
de-minified.

---

## Stage 3 — Expose error feed to the AI as MCP tools (~2 hr)

**Goal:** the AI can ask Sentry "what's broken in project X?" and
get a real answer.

**What gets built:**

Three new MCP tools in `vibn-frontend/lib/ai/vibn-tools.ts`:

1. **`project_recent_errors { projectId, since?, limit? }`**
   - Returns: `[{ id, title, count, lastSeen, culprit, level }]`
   - Default `since`: 24h. Default `limit`: 10.
   - Filters to unresolved issues only.
   - Implementation: read `sentryProjectSlug` off the project row,
     call Sentry's `GET /api/0/projects/{org}/{slug}/issues/`.

2. **`project_error_detail { projectId, issueId }`**
   - Returns: `{ stacktrace, breadcrumbs, request, user, replay_url }`
   - Implementation: Sentry's `GET /api/0/issues/{id}/events/latest/`.

3. **`project_error_resolve { projectId, issueId }`**
   - Side-effect: marks the issue resolved in Sentry.
   - Used by the AI after it ships a fix and confirms via tests.
   - Implementation: Sentry's `PUT /api/0/issues/{id}/` with
     `status: "resolved"`.

**Auth:** token storage is per-project (from Stage 1's `projects`
row). Each project's AI sees only its own project's errors. No
cross-project leakage.

**Definition of done:** in a Vibn chat for a project with known
errors, ask the AI "any errors lately?" → AI calls
`project_recent_errors` → shows real list.

---

## Stage 4 — Auto-surface unresolved errors at chat-turn start (~1 hr)

**Goal:** the AI doesn't wait to be asked. When the user opens a
chat and there are unresolved errors, the AI mentions them on the
first turn.

**What gets built:**

In `vibn-frontend/app/api/chat/route.ts`, at the start of each chat
turn (before calling the model):

1. Call the same `project_recent_errors` logic Stage 3 exposed.
2. If `count > 0`, prepend a synthetic system message:

```
[PROJECT HEALTH]
{N} unresolved Sentry issues in the last 24 hours:
- {title} (×{count}, last seen {time}) — {culprit}
- ...

If the user's first message is unrelated to these, you may still
proactively mention them: "Quick FYI before we get into that —
{X} has been failing for users."

If their message IS about a broken thing, prefer the matching
Sentry issue's stack trace over guessing.
```

3. Only fire this once per N chat turns (configurable, default 1
   per session opening) — we don't want to spam every turn.

**Risk:** false alarms (Sentry issue from yesterday's deploy that
no one cares about anymore) make the AI annoying. Mitigation:
tighten the `since` window to the last 6h, and only surface issues
with `count >= 2` (one-off errors don't count).

**Definition of done:** intentionally break a deployed user
project, open chat, type "what's up?" → AI's first response
mentions the issue, with file path.

---

## Out of scope for this proposal

- **User-owned Sentry orgs.** Some users will eventually want their
  own Sentry account, not the shared `vibnai` org. Ship-later;
  doesn't block the loop. Easy retrofit because storage is already
  per-project.
- **Performance / Tracing data.** Sentry also captures spans /
  traces. Useful for "this endpoint is slow" but not the urgent
  product loop. Ship-later.
- **Front-end UI for errors in Vibn.** A "Health" tab showing the
  Sentry feed in the Vibn UI is nice but not required for the AI
  loop to work. Ship-later.

---

## Recommendation

Add a **Phase 2.9 (Sentry-as-product loop)** to `BETA_LAUNCH_PLAN.md`
covering Stages 1–4 as a single bundle. Estimate: **8 hr engineering**.

This is the second-highest-leverage item still ahead of beta,
behind only the deploy-failed webhook (which is 30 min). Every
hour spent here directly upgrades the value of every other beta
test session that follows it.