173 lines
5.0 KiB
Markdown
173 lines
5.0 KiB
Markdown
# QA Issues Found - Table Stakes Implementation
|
|
|
|
## 🐛 **Issue #1: Extension Linked Status Not Passed to AI** (CRITICAL)
|
|
|
|
**Problem:**
|
|
- `link-project` API updates `projects.extensionLinked = true`
|
|
- But `ProjectChatContext` doesn't include `extensionLinked` field
|
|
- AI doesn't know extension is linked, so can't update `collectorHandoff.extensionLinked`
|
|
- Checklist never shows extension as linked
|
|
|
|
**Root Cause:**
|
|
`lib/server/chat-context.ts` doesn't include `extensionLinked` in the context object passed to LLM.
|
|
|
|
**Impact:**
|
|
- User links extension via UI
|
|
- AI never acknowledges it
|
|
- Checklist stays incomplete
|
|
- Auto-transition may never trigger
|
|
|
|
**Fix:**
|
|
Add `extensionLinked` to `ProjectChatContext.project` and pass `projectData.extensionLinked` to LLM.
|
|
|
|
---
|
|
|
|
## 🐛 **Issue #2: Collector Handoff Missing from Context Type** (MEDIUM)
|
|
|
|
**Problem:**
|
|
`ProjectChatContext.phaseHandoffs` type is:
|
|
```typescript
|
|
Partial<Record<'extraction' | 'vision' | 'mvp' | 'marketing', PhaseHandoff>>
|
|
```
|
|
|
|
But we're storing `'collector'` handoffs. This is a TypeScript type mismatch.
|
|
|
|
**Impact:**
|
|
- Type error (may not catch at runtime in JS)
|
|
- Context builder won't expose existing `collector` handoff to AI
|
|
- AI can't see its own previous checklist state
|
|
|
|
**Fix:**
|
|
Update type to include `'collector'`:
|
|
```typescript
|
|
Partial<Record<'collector' | 'extraction' | 'vision' | 'mvp' | 'marketing', PhaseHandoff>>
|
|
```
|
|
|
|
---
|
|
|
|
## 🐛 **Issue #3: Phase Transition Uses Wrong Field** (MEDIUM)
|
|
|
|
**Problem:**
|
|
Auto-transition updates:
|
|
```typescript
|
|
currentPhase: 'analyzed'
|
|
```
|
|
|
|
But `resolveChatMode` checks for `phaseData.canonicalProductModel` to determine if we're in extraction mode, not `currentPhase`.
|
|
|
|
**Impact:**
|
|
- Project transitions to `analyzed` phase
|
|
- But mode resolver might still return `collector_mode` if no extractions exist
|
|
- AI might not actually switch to extraction prompt
|
|
|
|
**Fix:**
|
|
Either:
|
|
1. Update `resolveChatMode` to also check `currentPhase` field
|
|
2. Or update auto-transition to set a field that mode resolver checks
|
|
|
|
---
|
|
|
|
## 🐛 **Issue #4: Context Source Types Missing** (LOW)
|
|
|
|
**Problem:**
|
|
`knowledgeSummary.bySourceType` counts items by type, but doesn't explicitly include counts for:
|
|
- `extension_chat` (from browser extension)
|
|
- `github_code` (from GitHub)
|
|
|
|
**Impact:**
|
|
- AI can tell if GitHub is *connected* (via `githubRepo`)
|
|
- But can't tell if extension has *sent any chats* yet
|
|
- May incorrectly think extension is "not working"
|
|
|
|
**Fix:**
|
|
Add explicit source type detection in context summary.
|
|
|
|
---
|
|
|
|
## 🐛 **Issue #5: Conversation History Indentation Error** (SYNTAX)
|
|
|
|
**Problem:**
|
|
`app/api/ai/chat/route.ts` lines 41-67 have indentation issues from recent edits.
|
|
|
|
**Status:** Already caught by editor, needs cleanup.
|
|
|
|
---
|
|
|
|
## 🐛 **Issue #6: ExtensionLinked vs Extension Data** (DESIGN)
|
|
|
|
**Problem:**
|
|
- `extensionLinked` is a boolean flag on project
|
|
- But doesn't actually verify extension is *sending data*
|
|
- User could link, then uninstall extension
|
|
|
|
**Impact:**
|
|
- Checklist shows "Extension linked ✓"
|
|
- But extension isn't actually working
|
|
- False sense of completion
|
|
|
|
**Fix (Future):**
|
|
- Add `lastExtensionActivity` timestamp
|
|
- Show "Extension active" vs "Extension linked but inactive"
|
|
- Collector checks for recent activity, not just linked status
|
|
|
|
---
|
|
|
|
## 📊 **Priority Order:**
|
|
|
|
1. **🔴 Critical - Issue #1**: Extension status not passed to AI
|
|
2. **🟡 Medium - Issue #2**: Type mismatch for collector handoff
|
|
3. **🟡 Medium - Issue #3**: Phase transition field mismatch
|
|
4. **🟢 Low - Issue #4**: Source type granularity
|
|
5. **🟣 Cleanup - Issue #5**: Indentation
|
|
6. **🔵 Future - Issue #6**: Active vs linked detection
|
|
|
|
---
|
|
|
|
## 🛠️ **Fixes To Apply:**
|
|
|
|
### Fix #1: Add extensionLinked to context
|
|
|
|
```typescript:lib/server/chat-context.ts
|
|
project: {
|
|
id: projectId,
|
|
name: projectData.name ?? 'Unnamed Project',
|
|
currentPhase: projectData.currentPhase ?? 'collector',
|
|
phaseStatus: projectData.phaseStatus ?? 'not_started',
|
|
githubRepo: projectData.githubRepo ?? null,
|
|
githubRepoUrl: projectData.githubRepoUrl ?? null,
|
|
extensionLinked: projectData.extensionLinked ?? false, // ADD THIS
|
|
},
|
|
```
|
|
|
|
### Fix #2: Update phaseHandoffs type
|
|
|
|
```typescript:lib/server/chat-context.ts
|
|
phaseHandoffs: Partial<Record<'collector' | 'extraction' | 'vision' | 'mvp' | 'marketing', PhaseHandoff>>;
|
|
```
|
|
|
|
### Fix #3: Update mode resolver to check currentPhase
|
|
|
|
```typescript:lib/server/chat-mode-resolver.ts
|
|
// After checking for knowledge and extractions
|
|
if (projectData.currentPhase === 'analyzed' || (hasExtractions && !phaseData.canonicalProductModel)) {
|
|
return 'extraction_review_mode';
|
|
}
|
|
```
|
|
|
|
### Fix #5: Clean up indentation
|
|
|
|
Run prettier/format on `app/api/ai/chat/route.ts`.
|
|
|
|
---
|
|
|
|
## ✅ **Testing After Fixes:**
|
|
|
|
1. Create new project
|
|
2. Upload document → verify checklist updates
|
|
3. Connect GitHub → verify checklist updates
|
|
4. Link extension → **verify checklist updates** (currently broken)
|
|
5. AI asks "Is that everything?" → User says "Yes"
|
|
6. **Verify auto-transition to extraction mode** (currently may not work)
|
|
7. Verify AI switches to extraction prompt
|
|
|