Using AI to QC-QA a Real Web App — 8 Categories, 163 Items — Score Jumped from 49% to 89%
An experiment using AI deep research to find QC-QA checklists for web projects yielded 236 prompts, 32 tools, and 195 checklist items across 8 categories. Then those were used to audit a real web app — covering Functional Testing, Security, Performance, Accessibility, SEO, and 3 more categories — with every actual prompt included.

Functional Testing — From 56% to 87% with AI
The Starting Point — Why Build a QC-QA System with AI?
The team had someone doing QC-QA on web projects — but the problem was: slow checks, incomplete coverage, and no clear standard for what "complete" even means.
Multiple web projects worth 10 million baht and up. A single bug slipping into production could mean hundreds of thousands in losses. But having humans check everything manually — that's 2-3 days per release.
The goal: find a way to make the QC-QA team faster and more thorough, using AI.
That was the challenge. And so the AI got asked.
Deep Research Prompt — The First Question Asked to AI
In the world of QC/QA for web applications, mobile applications, and web design for projects worth 10 million baht and above — what QC/QA processes are needed? Do a deep research and show me everything, both free and paid. Include prompts for AI-assisted auditing, tools that actually work, and checklist items that need to be covered.
The AI deep research came back in 15 minutes — with results beyond expectations.
236 prompts. 32 tools. 195 checklist items. — from just a single question.
But raw data this massive — just dumping it in front of the team would be overwhelming. It needed to be organized into something actually usable.
Getting AI to Categorize — 8 Categories
From the 236 prompts, 32 tools, 195 checklist items gathered: 1. Separate free vs paid 2. Categorize by scan type and sub-items 3. Separate by tech stack (Next.js, React, Vue, etc.) 4. For quick workflow, separate by system category 5. Create main menu based on number of categories
AI came back with 8 categories — fully organized with item counts for each.
Planning
10 itemsDevelopment
21 itemsFunctional Testing
19 itemsPerformance
19 itemsSecurity
20 itemsAccessibility
16 itemsSEO & Analytics
18 itemsDeployment
15 itemsEach category has ready-made prompts — just hit Copy and paste into Cursor. Complete with recommended tools + install commands. The team doesn't need to memorize anything or Google stuff themselves.
What Do the 19 Items Check?
Today's deep dive is into the Functional Testing category with 19 items — because this is the category that was actually used on the ScanlyIQ project and uncovered 5 critical bugs.
Functional Testing breaks down into 4 groups:
Group 1 — E2E Testing (10 Items)
| # | Topic | Severity | Tool |
|---|---|---|---|
| 1 | Happy Path for All User Flows | CRITICAL | Playwright |
| 2 | Form Validation | HIGH | Playwright |
| 3 | Authentication Flow | CRITICAL | Playwright |
| 4 | Authorization (RBAC) | CRITICAL | Playwright |
| 5 | CRUD Operations | HIGH | Playwright |
| 6 | Search & Filter | MEDIUM | Playwright |
| 7 | Pagination | LOW | Playwright |
| 8 | File Upload/Download | MEDIUM | Playwright |
| 9 | Payment Flow | CRITICAL | Playwright |
| 10 | Email Notifications | MEDIUM | Mailtrap |
Group 2 — Cross-Browser (4 Items)
| # | Browser | Severity | Tool |
|---|---|---|---|
| 11 | Chrome | HIGH | Playwright |
| 12 | Firefox | MEDIUM | Playwright |
| 13 | Safari | MEDIUM | Playwright |
| 14 | Edge | LOW | Playwright |
Group 3 — Responsive Design (4 Items)
| # | Screen Size | Severity | Tool |
|---|---|---|---|
| 15 | Mobile (320-480px) | HIGH | Responsively |
| 16 | Tablet | MEDIUM | Responsively |
| 17 | Desktop | MEDIUM | Responsively |
| 18 | Large Screen | LOW | Responsively |
Group 4 — Error Pages (1 Item)
| # | Topic | Severity | Tool |
|---|---|---|---|
| 19 | Error Pages: 404/500 | MEDIUM | Playwright |
Every item has a ready-made prompt — just hit Copy and paste into Cursor to let AI audit the project.
Real Audit Results — 5 Critical Bugs Found Hiding
The prompts from the Functional Testing category were pasted into Cursor and Claude was told to audit the actual ScanlyIQ project — not just read the checklist and tick ✅ boxes.
AI read every file of code, checked every route endpoint, examined middleware and auth logic — then gave back scores.
The results? Gut-wrenching.
The score before fixing was just 56% — only 110 out of 190 possible points. This was a project that was "thought to be" ready for deployment.
ZONE 1 — Critical Issues Caught by AI
5 critical issues — without auditing, these would definitely have shipped to production:
🔴 ZONE 1 — Risky Fixes (Must Test on Staging First)
| # | Issue | Why It's Risky |
|---|---|---|
| 1 | Payment race condition — stripeSessionId might be null when webhook fires | User pays but doesn't get credits |
| 2 | Google OAuth bypasses isActive — banned user can still login via Google | Ban system doesn't work |
| 3 | allowDangerousEmailAccountLinking: true | Attacker can hijack accounts |
| 4 | Credit deduction race condition — concurrent requests overdraw | User can spend more credits than they have |
| 5 | File delete removes from disk before DB | If DB fails → file is gone but DB still shows it |
Item 1 is huge — Payment race condition. If the Stripe webhook fires before the system saves the stripeSessionId, the user pays but doesn't get credits. When would anyone find out? When the user comes complaining.
Item 3 is even scarier — allowDangerousEmailAccountLinking was turned on. The name literally says "dangerous" but nobody noticed. AI caught it.
ZONE 2 — Safe to Fix Right Away
🟢 ZONE 2 — Safe to Fix Now (No System Impact)
| # | Issue | Fix | Effort |
|---|---|---|---|
| 1 | /api/auth/register missing → can't register | Create new endpoint | L |
| 2 | Email functions never called | Add sendEmail calls | S |
| 3 | No error.tsx / not-found.tsx | Create custom 404/500 | S |
| 4 | File size limit mismatch (50MB vs 20MB) | Make them match | XS |
| 5 | OKLCH CSS has no fallback | Add HSL fallback | M |
| 6 | No browserslist | Add to package.json | XS |
| 7 | Large screen has no max-width | Add max-w-7xl | XS |
AI Fixed All 12 Items — In a Single Session
The instruction to AI: "Can you fix all of these?"
AI fixed all 12 items. From creating the register endpoint, fixing the payment race condition, closing the OAuth vulnerability, creating error pages, all the way to adding CSS fallbacks — everything in a single session.
| # | What Was Fixed | Before → After |
|---|---|---|
| 1 | Register endpoint | Missing → Registration works + password hashing + referral + bonus |
| 2 | Error pages | Default Next.js → Custom 404/500 in Thai |
| 3 | File size mismatch | Server 20MB ≠ Client 50MB → Matched |
| 4 | Payment race condition | Webhook can't find session → Fallback via metadata |
| 5 | OAuth security | dangerousLinking: true → false |
| 6 | OAuth ban bypass | No isActive check → Check in JWT callback |
| 7 | Credit race condition | Check outside transaction → Moved into $transaction |
| 8 | File delete race | Delete disk first → Soft-delete DB first |
| 9 | Email notifications | Not sending → Sends welcome + job completed |
| 10 | OKLCH CSS fallback | OKLCH only → HSL fallback + @supports |
| 11 | browserslist | Missing → last 2 versions, >0.5% |
| 12 | Large screen max-width | Unlimited → max-w-7xl mx-auto |
Final Scores — Per Item
| Topic | Score | Result |
|---|---|---|
| E2E: Happy Path | 0 → 8/10 | ✅ Register works now |
| Form Validation | 5/10 | ⚠️ Mismatch fixed |
| Auth Flow | 3 → 8/10 | ✅ Register + OAuth fixed |
| Authorization (RBAC) | 9/10 | ✅ Already great from the start |
| CRUD Operations | 6/10 | ⚠️ Some operations missing |
| Search & Filter | 8/10 | ✅ |
| Pagination | 6/10 | ⚠️ Storage pagination added |
| File Upload/Download | 5/10 | ⚠️ Upload ✅ Download ❌ |
| Payment Flow | 6 → 8/10 | ✅ Race condition fixed |
| Email Notifications | 1 → 7/10 | ✅ Emails now sent |
| Chrome / Edge | 9/10 | ✅ |
| Firefox / Safari | 6 → 8/10 | ✅ CSS fallback added |
| Responsive (Mobile/Tablet/Desktop) | 8-9/10 | ✅ |
| Large Screen | 5 → 8/10 | ✅ max-width added |
| Error Pages | 1 → 8/10 | ✅ Custom 404/500 |
Why Prompts Beat Expensive Tools — And the Workflow
Once upon a time, a QA tool costing tens of thousands per month was purchased — it could only check the "surface". Lighthouse scores, broken links, SSL expiry. That's it.
But a good prompt + AI that can read code? It found race conditions in the code. It found OAuth bypasses. It found email functions that were created but never actually called.
This is something no tool can do — unless there's a senior developer sitting down to do a code review.
❌ Traditional QA Tools
- ❌ Only checks the surface (UI, performance, SEO)
- ❌ Doesn't read source code
- ❌ Can't find race conditions / logic bugs
- ❌ Costs tens of thousands/month
- ❌ Learning curve required
✅ AI + Good Prompts
- ✅ Reads every line of code
- ✅ Finds race conditions, auth bypasses, dead code
- ✅ Fixes things directly, not just reports
- ✅ Cost: just the AI subscription
- ✅ Instructed in plain language
The Actual Workflow — 5 Steps
Step 1 — Open QC Dashboard → Pick a Category
Open Enterprise 360 QC-QA Tools → Go to the Prompts tab → Select a category (e.g., Functional Testing)
Step 2 — Hit Copy Prompt
Each category has a ready-made prompt — just one button to Copy Prompt
Step 3 — Paste into Cursor (Cmd+L)
Open Cursor → Cmd+L → Paste prompt → Tell AI to audit
Step 4 — AI Audits + Returns Results
AI reads every file, checks every endpoint, then returns results as Zone 1 (risky) + Zone 2 (safe to fix) + scores
Step 5 — Tell AI to Fix
Tell AI "fix everything" → AI fixes code → runs build → verifies
All 5 steps took less than 1 hour total — for auditing + fixing 19 topics and 12 issues. Doing it manually? 2-3 days.
Try It Yourself — Prompt + FAQ
Here's the actual prompt used — copy and paste into Cursor:
[ROLE] You are a Senior QC Engineer auditing a web project [PROJECT] Framework: nextjs | Path: ./ [URL] Dev: http://localhost:3000 | Prod: (not set yet) [SCOPE] Audit only — do not modify any files Audit "🧪 Functional Testing" — 19/19 items: 1. [CRITICAL] E2E: Happy Path for All User Flows 2. [HIGH] E2E: Form Validation 3. [CRITICAL] E2E: Authentication Flow 4. [CRITICAL] E2E: Authorization (RBAC) 5. [HIGH] E2E: CRUD Operations 6. [MEDIUM] E2E: Search & Filter 7. [LOW] E2E: Pagination 8. [MEDIUM] E2E: File Upload/Download 9. [CRITICAL] E2E: Payment Flow 10. [MEDIUM] E2E: Email Notifications 11. [HIGH] Cross-Browser: Chrome 12-14. [MEDIUM-LOW] Cross-Browser: Firefox, Safari, Edge 15. [HIGH] Responsive: Mobile (320-480px) 16-18. [MEDIUM-LOW] Responsive: Tablet, Desktop, Large Screen 19. [MEDIUM] Error Pages: 404/500 [OUTPUT] ===== Audit Summary ===== ## ZONE 1 — Risky fixes (backup/test staging first) ## ZONE 2 — Safe to fix (no system impact) ## Score — Before: XX% → After: XX%
Change [PROJECT] to your framework and path, change [URL] to your dev URL — then tell AI to go.
FAQ — Frequently Asked Questions
Q: Do you need to know code to use this prompt?
Nope. Just open Cursor with the project to audit, copy the prompt, paste it, and tell AI to go — it reads all the code automatically. Results are written in plain language anyone can understand.
Q: Does this work with any language/framework?
Absolutely. This prompt is designed for web apps on any framework — Next.js, React, Vue, Angular, PHP, Django. Just change the [PROJECT] Framework.
Q: Are 19 items enough? What about Security, Performance?
Functional Testing is just 1 of 8 categories. Security (20 items), Performance (19 items), SEO (18 items) are in separate categories — each with its own prompt. Use the QC Dashboard to open them one at a time.
Q: Is it safe when AI fixes code? Won't it break more stuff?
Always audit first. Tell AI to audit only (do not modify files) → review results → if it looks good, then tell it to fix. And commit at every step in case rollback is needed.
Q: Why not just use Playwright directly instead of AI code review?
Playwright checks "behavior" (what happens when a button is clicked). AI checks "structure" (does the code have race conditions? is auth secure?). Best to use both — but if only one could be chosen, AI code review finds deeper problems.
Planning — From 33% to 85% with AI
Why Does Planning Need Auditing? — It's the Foundation
After running the Functional Testing category and bumping ScanlyIQ from 56% to 87% — curiosity kicked in: "What else would turn up if other categories were audited?"
Opened the QC Dashboard to the Planning — 10 items category. This one doesn't check for code bugs — it checks whether the project has the infrastructure to scale. Are there tests? CI/CD? Monitoring? Security headers?
The results? Even worse than Functional Testing.
33 out of 100. A project that "works" but has zero foundation — not a single test file, no CI/CD, no monitoring, and an ESLint config so broken it was useless.
What Do the 10 Items Check?
[ROLE] You are a Senior QC Engineer auditing a web project [PROJECT] Framework: nextjs | Path: ./ [SCOPE] Audit only — do not modify any files Audit "📋 Planning" — 10/10 items: 1. [MEDIUM] Define Browser/Device Matrix 2. [HIGH] Define Performance Budget (LCP<2.5s, CLS<0.1) 3. [HIGH] Define Accessibility Standard (WCAG 2.1 AA) 4. [MEDIUM] Define SEO Requirements 5. [CRITICAL] Define Security Requirements (OWASP Top 10) 6. [HIGH] Plan Test Strategy (Unit/Integration/E2E) 7. [MEDIUM] Define Code Quality Standards 8. [MEDIUM] Set Error Budget & Monitoring Plan 9. [MEDIUM] Design CI/CD Pipeline 10. [LOW] Define Definition of Done (DoD)
Scores Before Fix — Per Item
| # | Topic | Priority | Score | Status |
|---|---|---|---|---|
| 1 | Browser/Device Matrix | MEDIUM | 5/10 | ⚠️ Has browserslist but no documentation |
| 2 | Performance Budget | HIGH | 2/10 | ❌ No budget/tracking |
| 3 | Accessibility (WCAG 2.1 AA) | HIGH | 3/10 | ⚠️ Only basic aria-invalid |
| 4 | SEO Requirements | MEDIUM | 4/10 | ⚠️ Has metadata but no robots/sitemap |
| 5 | Security (OWASP Top 10) | CRITICAL | 7/10 | ✅ Auth + Zod + Rate Limit excellent |
| 6 | Test Strategy | HIGH | 0/10 | ❌ Not a single test file |
| 7 | Code Quality | MEDIUM | 5/10 | ⚠️ TS strict is good but ESLint broken |
| 8 | Monitoring | MEDIUM | 1/10 | ❌ No error tracking at all |
| 9 | CI/CD Pipeline | MEDIUM | 2/10 | ❌ Only has Docker |
| 10 | Definition of Done | LOW | 4/10 | ⚠️ Exists in CLAUDE.md but not formal |
Total: 33/100 — The item that scored 0/10 was Test Strategy. A project with 56 API routes but not a single automated test.
ZONE 1 — Fixes That Could Break Things
🔴 ZONE 1 — Must Test on Staging First
| # | Issue | Why It's Risky |
|---|---|---|
| 1 | No Security Headers (CSP, HSTS, X-Frame-Options) | HSTS forces HTTPS immediately — if SSL isn't ready, site becomes inaccessible |
| 2 | ESLint config broken | Fixing may surface tons of errors that need follow-up fixes |
ZONE 2 — Quick Wins to Fix Right Away
🟢 ZONE 2 — Safe to Fix Now (No System Impact)
| # | Issue | Fix | Effort |
|---|---|---|---|
| 1 | No robots.txt | Create src/app/robots.ts | XS |
| 2 | No sitemap.xml | Create src/app/sitemap.ts | S |
| 3 | No metadataBase | Add to layout.tsx | XS |
| 4 | No Viewport export | Add to layout.tsx | XS |
| 5 | No Prettier | Install + configure | S |
| 6 | No test framework | Install Vitest + write first test | M |
| 7 | No CI/CD | Create GitHub Actions workflow | M |
| 8 | No monitoring | Install Sentry SDK | M |
| 9 | No Structured Data | Add JSON-LD | S |
| 10 | No Performance Budget | Create document defining targets | XS |
AI Fixed All 10 Items — Results
The instruction to AI: "Go ahead and do everything" — same approach as Functional Testing.
AI completed all 10 items in a single session:
| # | What Was Done | Before → After |
|---|---|---|
| 1 | Fixed ESLint config | Broken (Oops! Something went wrong) → Working + found 15 warnings |
| 2 | Installed Prettier | Missing → .prettierrc + format:check script |
| 3 | Added Security Headers | 0 headers → HSTS, X-Frame-Options, X-Content-Type + 3 more |
| 4 | Added metadataBase + viewport | Missing → Complete meta, OG, icons, locale |
| 5 | Created robots.ts + sitemap.ts | Missing → Both dynamically generated |
| 6 | Added Structured Data | Missing → JSON-LD WebApplication schema |
| 7 | Installed Vitest + first tests | 0 tests → 9 unit tests all passing |
| 8 | Created CI/CD | Missing → GitHub Actions (lint + type-check + test + build) |
| 9 | Installed Sentry | Missing → client + server + edge config + error capture |
| 10 | Created Performance Budget | Missing → document with LCP/CLS/INP targets + bundle budget |
Scores After Fix — Per Item
| Topic | Score | Result |
|---|---|---|
| Browser/Device Matrix | 5 → 7/10 | ✅ Has browserslist + Tailwind breakpoints |
| Performance Budget | 2 → 8/10 | ✅ Has document + clearly defined targets |
| Accessibility | 3 → 6/10 | ⚠️ Still needs more auditing |
| SEO Requirements | 4 → 9/10 | ✅ robots + sitemap + JSON-LD + OG complete |
| Security (OWASP) | 7 → 9/10 | ✅ Added 6 security headers |
| Test Strategy | 0 → 8/10 | ✅ Vitest + 9 tests + CI/CD runs automatically |
| Code Quality | 5 → 9/10 | ✅ ESLint working + Prettier + TS strict |
| Monitoring | 1 → 8/10 | ✅ Sentry on all 3 layers |
| CI/CD Pipeline | 2 → 9/10 | ✅ GitHub Actions full pipeline |
| Definition of Done | 4 → 7/10 | ✅ Has quality gates + budget |
Total: 33/100 → 85/100 — up 52 points. The biggest jumps were Test Strategy (0 → 8) and CI/CD (2 → 9).
Lessons Learned from the Planning Category
Planning isn't just "making plans" — it's the project's foundational infrastructure. A project that works but has no tests, no CI/CD, no monitoring — that's like a house you can live in but has no foundation piles.
The funny part is AI pointed out that ESLint — which should be the "first line of defense" for code quality — had been broken the whole time without anyone noticing. Running pnpm lint gave "Oops! Something went wrong!" but nobody noticed because builds still passed.
Another thing: 0 test files for 56 API routes. Not because of laziness — but because "not knowing where to start". AI set up Vitest, wrote the first tests, set up CI/CD — all in 1 session. Now every push to GitHub automatically runs lint + type-check + test + build.
Security Testing — From 71% to 100% with AI
Why Is Security Scarier Than Other Categories?
Functional Testing finds bugs that break systems. Planning finds structural gaps. But Security? — it finds vulnerabilities that let others steal data.
For projects worth 10 million and up, if user data leaks — it's not just the cost of fixing the system, it's losing all trust.
Opened the QC Dashboard to the Security — 20 items category and ran it against ScanlyIQ:
71% sounds decent — but for Security, "decent" isn't enough. Failing just one item could mean getting hacked.
What Do the 20 Items Check?
[ROLE] You are a Senior QC Engineer auditing a web project [PROJECT] Framework: nextjs | Path: ./ [SCOPE] Audit only — do not modify any files Audit "🛡️ Security" — 20/20 items: 1. [CRITICAL] OWASP ZAP (0 High/Critical) 2. [CRITICAL] SQL Injection Test 3. [CRITICAL] XSS Test 4. [HIGH] CSRF Protection 5. [HIGH] Content Security Policy (CSP) 6. [HIGH] HTTPS Everywhere 7. [HIGH] HSTS Header 8. [MEDIUM] X-Content-Type-Options 9. [MEDIUM] X-Frame-Options 10. [MEDIUM] Referrer-Policy 11. [LOW] Permissions-Policy 12. [HIGH] Cookie Flags (Secure/HttpOnly/SameSite) 13. [HIGH] Rate Limiting on API 14. [HIGH] File Upload Validation 15. [CRITICAL] Dependency Scan (0 Critical) 16. [HIGH] SSL Grade A+ 17. [MEDIUM] SRI (Subresource Integrity) 18. [CRITICAL] Secrets Scan (0 leaks) 19. [HIGH] Input Validation client+server 20. [MEDIUM] Security Headers Score A+
Note: 6 items (#1, #16, #17, #20 and 2 others) require a deployed URL or external tools — AI can audit 14/20 items via code review + pnpm audit
Scores Before Fix — Per Item
| # | Topic | Severity | Result |
|---|---|---|---|
| 1 | OWASP ZAP | CRITICAL | ⏭️ SKIP — Requires ZAP installation |
| 2 | SQL Injection | CRITICAL | ✅ 100% Prisma ORM, no raw SQL |
| 3 | XSS Test | CRITICAL | ✅ dangerouslySetInnerHTML only in JSON-LD |
| 4 | CSRF Protection | HIGH | ✅ NextAuth v5 + SameSite cookies |
| 5 | CSP Header | HIGH | ❌ No CSP at all |
| 6 | HTTPS Everywhere | HIGH | ✅ HSTS enforces HTTPS |
| 7 | HSTS Header | HIGH | ⚠️ Present but missing preload flag |
| 8 | X-Content-Type-Options | MEDIUM | ✅ nosniff |
| 9 | X-Frame-Options | MEDIUM | ✅ DENY |
| 10 | Referrer-Policy | MEDIUM | ✅ strict-origin-when-cross-origin |
| 11 | Permissions-Policy | LOW | ✅ camera, mic, geo all disabled |
| 12 | Cookie Flags | HIGH | ✅ Secure + HttpOnly + SameSite |
| 13 | Rate Limiting | HIGH | ❌ None at all — every endpoint accepts unlimited requests |
| 14 | File Upload Validation | HIGH | ⚠️ Has MIME check but no magic number validation |
| 15 | Dependency Scan | CRITICAL | ❌ 9 vulnerabilities (2 HIGH) |
| 16 | SSL Grade A+ | HIGH | ⏭️ SKIP — Requires deployed URL |
| 17 | SRI | MEDIUM | ⏭️ SKIP |
| 18 | Secrets Scan | CRITICAL | ✅ .env in .gitignore, no hardcoded secrets |
| 19 | Input Validation | HIGH | ⚠️ Mostly good but LINE webhook has no Zod |
| 20 | Security Headers Score | MEDIUM | ⏭️ SKIP |
14 items audited: 10 pass, 2 fail, 2 warnings → 71%
ZONE 1 — Fixes That Could Break Things
🔴 ZONE 1 — Must Test on Staging First
| # | Issue | Why It's Risky |
|---|---|---|
| 1 | No Rate Limiting — every API endpoint accepts unlimited requests | Unlimited brute-force login, credit burn attacks, upload DoS |
| 2 | 9 Dependency Vulnerabilities (2 HIGH) — hono via shadcn | hono has arbitrary file access + auth bypass vulnerabilities |
| 3 | No CSP Header | XSS attacks can inject scripts from external sources |
Item 1 is the scariest — absolutely no Rate Limiting. Meaning anyone could fire unlimited requests at /api/auth/register or /api/upload/slip. Brute-force passwords? Possible. Burn through a user's credits? Possible. Upload huge files repeatedly until the server crashes? Possible.
Item 2 — the hono dependency that comes with shadcn had 9 vulnerabilities including 2 HIGH ones allowing access to any file on the server.
ZONE 2 — Quick Wins to Fix Right Away
🟢 ZONE 2 — Safe to Fix Now (No System Impact)
| # | Issue | Fix | Effort |
|---|---|---|---|
| 1 | HSTS missing preload | Add ; preload to middleware | XS |
| 2 | Path traversal only uses includes("..") | Switch to path.resolve() + boundary check | S |
| 3 | File upload doesn't check magic numbers | Add magic bytes check (JPEG/PNG/WebP) | S |
| 4 | LINE webhook has no Zod validation | Add schema validation | S |
| 5 | Error response reveals actual credit balance | Return generic "insufficient credits" without actual numbers | XS |
| 6 | Job cancellation race condition | Move item count query into Prisma transaction | S |
AI Fixed Everything — Results
The instruction to AI: "Go ahead and do everything" — same approach as the previous 2 categories.
| # | What Was Done | Before → After |
|---|---|---|
| 1 | Added Rate Limiting | None → In-memory limiter: auth 10/min, upload 20/min, API 60/min |
| 2 | Fixed Dependency Vulnerabilities | 9 vulns (2 HIGH) → 0 vulnerabilities |
| 3 | Added CSP Header | None → Content-Security-Policy with all directives |
| 4 | HSTS preload | No preload → Added preload flag |
| 5 | Path traversal fix | includes("..") → path.resolve() + boundary check |
| 6 | Magic number validation | None → Validates file headers JPEG/PNG/WebP |
| 7 | LINE webhook validation | None → Zod schema validation |
| 8 | Error message cleanup | Shows real balance → Generic "insufficient credits" |
| 9 | Race condition fix | Query outside transaction → Moved into transaction |
Things Already Done Right — AI Gave Props
SQL Injection = 0
100% Prisma ORM with not a single raw query
Solid Auth
NextAuth v5 + JWT + bcrypt(12) + full email verification
Complete Input Validation
Zod schema validation on every endpoint (except LINE webhook, now fixed)
Webhook Security
Stripe + PromptPay have signature verification preventing replay attacks
Lessons Learned from the Security Category
Security differs from other categories in that no item is "unimportant". Functional Testing found a bug where registration didn't work — fix it and move on. But Security found no Rate Limiting — meaning if someone wanted to cause damage, they could do it right now.
Another interesting find: dependency vulnerability. The vulnerable hono wasn't directly imported — it came with shadcn, a UI library. No way to know without running pnpm audit. AI caught it and fixed it with pnpm overrides in 2 minutes.
Bottom line: Security isn't a "do once and done" thing — it needs re-checking every time dependencies are updated or new features are added. The QC Dashboard makes re-auditing easy — copy prompt, paste, tell AI, under 1 hour.
Performance — From 45% to 82% with AI
Why Is Performance the Easiest Thing to Overlook?
Functional Testing finds bugs that crash systems. Security finds hackable vulnerabilities. But Performance? — it doesn't break things immediately. It silently kills, slowly.
A web page loading 1 second slower = 32% higher bounce rate. 3 seconds slower? Half the users close the tab. But during dev, testing on localhost everything looks fast — problems only show up in production with real users.
Opened the QC Dashboard to the Performance — 19 items category and ran it against ScanlyIQ:
45% — the second lowest scoring category after Planning. The main issue: zero code splitting across the entire project, no lazy loading, images not optimized. Everything loads all at once.
What Do the 19 Items Check?
[ROLE] You are a Senior QC Engineer auditing a web project [PROJECT] Framework: nextjs | Path: ./ [URL] Dev: http://localhost:3000 | Prod: (not set yet) [SCOPE] Audit only — do not modify any files Audit "⚡ Performance" — 19/19 items: 1. [HIGH] Lighthouse Performance >= 90 2. [HIGH] LCP < 2.5s 3. [HIGH] INP < 200ms 4. [HIGH] CLS < 0.1 5. [MEDIUM] TTFB < 800ms 6. [MEDIUM] FCP < 1.8s 7. [MEDIUM] Total Blocking Time < 200ms 8. [MEDIUM] Speed Index < 3.4s 9. [MEDIUM] Load Test: 100 concurrent users 10. [HIGH] API Response < 200ms (p95) 11. [LOW] Stress Test: 500 users 12. [HIGH] Memory Leak Check 13. [HIGH] DB Query < 100ms 14. [MEDIUM] Images Compressed (WebP) 15. [MEDIUM] CSS/JS Minified 16. [MEDIUM] Gzip/Brotli Enabled 17. [HIGH] Bundle < 500KB gzipped 18. [MEDIUM] Code Splitting done correctly 19. [MEDIUM] Lazy Loading below-the-fold
Note: 9 items (#1-4, #6-9, #11) require external tools or a deployed URL — AI can audit 10/19 items via code review + curl + build analysis
Scores Before Fix — Per Item
| # | Topic | Priority | Result |
|---|---|---|---|
| 1 | Lighthouse >= 90 | HIGH | ⏭️ Requires lighthouse CLI |
| 2 | LCP < 2.5s | HIGH | ⏭️ Requires deployed URL |
| 3 | INP < 200ms | HIGH | ⏭️ Requires deployed URL |
| 4 | CLS < 0.1 | HIGH | ⚠️ Uses <img> without width/height → high CLS |
| 5 | TTFB < 800ms | MEDIUM | ✅ Login 130ms, Register 112ms |
| 6 | FCP < 1.8s | MEDIUM | ⏭️ Requires lighthouse |
| 7 | TBT < 200ms | MEDIUM | ⏭️ Requires lighthouse |
| 8 | Speed Index < 3.4s | MEDIUM | ⏭️ Requires lighthouse |
| 9 | Load Test: 100 users | MEDIUM | ⏭️ Requires k6 |
| 10 | API p95 < 200ms | HIGH | ❌ /api/auth/session cold: 1.6s |
| 11 | Stress Test: 500 users | LOW | ⏭️ Requires k6 |
| 12 | Memory Leak Check | HIGH | ⚠️ autoAdvance callback not memoized |
| 13 | DB Query < 100ms | HIGH | ✅ Prisma queries use select fairly well |
| 14 | Images Compressed (WebP) | MEDIUM | ❌ No WebP config |
| 15 | CSS/JS Minified | MEDIUM | ✅ Next.js auto-minify in prod |
| 16 | Gzip/Brotli | MEDIUM | ✅ Content-Encoding: gzip |
| 17 | Bundle < 500KB gzipped | HIGH | ❌ JS 13MB+ (dev), no dynamic imports |
| 18 | Code Splitting done correctly | MEDIUM | ❌ 0 dynamic imports across entire project |
| 19 | Lazy Loading | MEDIUM | ❌ Uses <img> instead of <Image> |
10 items audited: 4 pass, 5 fail, 1 warning → ~45%
ZONE 1 — Fixes That Could Break Things
🔴 ZONE 1 — Must Test on Staging First
| # | Issue | Why It's Risky |
|---|---|---|
| 1 | Zero Code Splitting — heavy libs (recharts 70KB, framer-motion 30KB, exceljs 1MB) all load together | Wrong fix could break SSR/hydration |
| 2 | Switch <img> → <Image> (3 places in slip-ocr page) | Image component needs width/height or fill, may cause layout shift |
| 3 | Add Cache-Control headers to API routes | Wrong caching makes users see stale data |
Item 1 is the most critical — 0 dynamic imports across the entire project. Meaning heavy libraries like recharts (70KB), framer-motion (30KB), and exceljs (1MB) all load together regardless of which page is being viewed. Makes the initial bundle unnecessarily huge.
ZONE 2 — Quick Wins to Fix Right Away
🟢 ZONE 2 — Safe to Fix Now (No System Impact)
| # | Issue | Fix | Effort |
|---|---|---|---|
| 1 | No WebP config | Add images.formats to next.config.ts | XS |
| 2 | Direct framer-motion import in 8 landing components | Switch to LazyMotion + domAnimation | S |
| 3 | React Query staleTime too short (30s) | Increase to 5 minutes for dashboard data | XS |
| 4 | autoAdvance callback not memoized | Wrap with useCallback | XS |
| 5 | projectsWithStats recalculates every render | Wrap with useMemo | XS |
AI Fixed Everything — Results
The instruction to AI: "Go ahead and do everything" — same approach as every category so far.
| # | What Was Done | Before → After |
|---|---|---|
| 1 | Added WebP/AVIF config | None → images.formats: ['image/webp', 'image/avif'] + 1hr cache |
| 2 | LazyMotion for landing pages | Direct motion import → LazyMotion + domAnimation (8 files) |
| 3 | React Query staleTime | Default 30s → 5 minutes for dashboard queries |
| 4 | Changed <img> → <Image> | 3 places in slip-ocr: <img> → next/image + fill + sizes |
| 5 | Cache-Control headers | None → s-maxage=300, stale-while-revalidate=60 (4 API routes) |
| 6 | TypeScript check + build | 0 errors → build passed |
Heavy Dependencies Audited
recharts (~70KB)
Only used on project detail page — App Router already route-splits ✅
framer-motion (~30KB)
Used in 8 landing components — switched to LazyMotion ✅
exceljs + docx + pdfkit
Only used in server-side export route — doesn't affect client bundle ✅
@sentry/nextjs (~50KB)
Global monitoring — tree-shaking correct, verified ✅
Scores After Fix
| Topic | Score | Result |
|---|---|---|
| TTFB < 800ms | ✅ 130ms | ✅ Already good |
| API p95 < 200ms | 1.6s → 190ms (warm) | ✅ Cold start still slow but warm passes |
| DB Query < 100ms | ✅ | ✅ Prisma select is good |
| Images (WebP) | ❌ → ✅ | ✅ WebP + AVIF auto-serve |
| CSS/JS Minified | ✅ | ✅ Next.js auto |
| Gzip/Brotli | ✅ | ✅ gzip confirmed |
| Bundle size | ❌ → ⚠️ | ⚠️ Reduced via LazyMotion + route split |
| Code Splitting | 0 → 8 components | ✅ LazyMotion + Image lazy |
| Lazy Loading | ❌ → ✅ | ✅ next/image + fill + sizes |
| CLS | ⚠️ → Improved | ✅ Images now have width/height |
10 items audited → 9 pass, 1 warning → ~82% (up from 45%, +37%)
Lessons Learned from the Performance Category
Performance differs from other categories in that full coverage requires external tools. Lighthouse, k6, PageSpeed Insights — many require a deployed URL for real measurements. But AI can audit nearly half from code alone — and the findings are just as important.
The shocking part: 0 dynamic imports across the entire project. Not because it was forgotten — but because "not knowing it was needed". Next.js App Router does route-level splitting automatically, but if components within pages don't lazy load, they still load as one big chunk.
Another thing: API cold start at 1.6 seconds for /api/auth/session — first request is painfully slow but subsequent ones are just 190ms. This was fixed with cache headers + stale-while-revalidate that AI added.
Bottom line: Performance needs 2 rounds of auditing — first round from code (AI can do this), second round from production URL (using Lighthouse + PageSpeed). The QC Dashboard provides prompts for both rounds, ready to use.
Development — From 52% to 81% with AI
Why Is Development QC More Important Than Expected?
This category is the "overlooked fundamentals" — not about features or UX, but about the quality of the code itself. If the codebase is organized, everything that follows becomes easier. If it's messy — everything falls like dominoes.
The ScanlyIQ project was built entirely with AI using Next.js + TypeScript + Prisma. "AI wrote it = good code" — that was the assumption all along. Until the actual audit.
52% — half the Development checklist failed. Even though builds passed, deployment worked, and there wasn't a single error.
What Do the 21 Items Check?
[ROLE] You are a Senior QC Engineer auditing a web project [PROJECT] Framework: nextjs | Path: ./ [SCOPE] Audit only — do not modify any files Audit "💻 Development" — 21/21 items: 1. [HIGH] ESLint 0 errors 2. [MEDIUM] Prettier format passes 3. [HIGH] TypeScript strict mode 0 errors 4. [HIGH] Unit Test Coverage >= 80% 5. [MEDIUM] Component Tests complete 6. [HIGH] API Route Tests for every Method 7. [CRITICAL] Dependency Vulnerabilities = 0 Critical 8. [LOW] License Compliance 9. [HIGH] Code Review passed 10. [MEDIUM] SonarLint Quality Check 11. [MEDIUM] No console.log/debugger 12. [CRITICAL] Environment Variables complete 13. [HIGH] Database Migration tested 14. [HIGH] API Error Handling complete 15. [HIGH] Input Validation all forms 16. [MEDIUM] Image Optimization 17. [MEDIUM] Bundle Size within budget 18. [LOW] Git Hooks works correctly 19. [MEDIUM] No unused dependencies 20. [MEDIUM] No circular imports 21. [CRITICAL] No secrets in codebase
21 items. By severity: 3 CRITICAL, 7 HIGH, 9 MEDIUM, 2 LOW. Tools used: ESLint, Prettier, TypeScript, Vitest, npm audit, depcheck, madge — mostly free.
Scores Before Fix — Per Item
| # | Topic | Severity | Result |
|---|---|---|---|
| 1 | ESLint 0 errors | HIGH | ✅ 0 errors (16 warnings) |
| 2 | Prettier format | MEDIUM | ❌ 188 files failed formatting |
| 3 | TypeScript strict | HIGH | ✅ strict: true, 0 errors |
| 4 | Unit Test Coverage | HIGH | ❌ Only 1 test file, no coverage config |
| 5 | Component Test | MEDIUM | ❌ No component tests at all |
| 6 | API Route Test | HIGH | ❌ No API tests whatsoever |
| 7 | Dep Vulnerabilities | CRITICAL | ✅ 0 critical (2 high from indirect hono) |
| 8 | License Compliance | LOW | ⏭️ SKIP — license-checker not installed yet |
| 9 | Code Review | HIGH | ⏭️ N/A — requires review via GitHub PR |
| 10 | SonarLint | MEDIUM | ⏭️ SKIP — not installed yet |
| 11 | No console.log | MEDIUM | ❌ Found 19 instances in production code |
| 12 | Env Variables complete | CRITICAL | ⚠️ .env.example missing 4 vars (Sentry) |
| 13 | DB Migration | HIGH | ❌ Uses db push, no migration files |
| 14 | API Error Handling | HIGH | ⚠️ 95% good but 4 routes missing try/catch |
| 15 | Input Validation | HIGH | ✅ 90%+ routes use Zod |
| 16 | Image Optimization | MEDIUM | ⚠️ No next/image found in components |
| 17 | Bundle Size | MEDIUM | ⚠️ shared 102KB, some pages up to 314KB |
| 18 | Git Hooks | LOW | ❌ No Husky / .husky directory |
| 19 | Unused dependencies | MEDIUM | ❌ 7 unused deps + 7 unused devDeps |
| 20 | Circular imports | MEDIUM | ✅ 0 circular dependencies |
| 21 | Secrets in codebase | CRITICAL | ✅ .env in .gitignore, no secrets committed |
6 pass, 5 warnings, 7 fail, 3 skipped → from 18 auditable items = 52%
The shocking part: builds pass, TypeScript has zero errors — but there are no tests at all. Unit tests had just 1 file (validations.test.ts) testing Zod schemas. Component tests? Zero. API tests? Zero. The entire project.
ZONE 1 — Fixes That Could Break Things
🔴 ZONE 1 — Backup Before Fixing
| # | Issue | Why It's Risky | Safe Fix |
|---|---|---|---|
| 1 | Build FAIL — admin pages missing | 3 empty route directories (credits, referrals, settings) cause Next.js to fail finding page modules | Delete empty directories or create page.tsx for each |
| 2 | DB uses db push, no migrations | Production deploy has no migration history — can't rollback, schema drift risk | Create baseline migration from current schema, then switch to prisma migrate |
| 3 | 1 Unit test FAILING | registerSchema doesn't validate confirmPassword — password mismatch can slip through | Add confirmPassword + .refine() to Zod schema |
Item 1 is a great example of "builds pass" ≠ "everything's fine". Dev server runs normally, but production build breaks because Next.js collecting page data scans every directory — empty folders cause errors.
ZONE 2 — Quick Wins to Fix Right Away
🟢 ZONE 2 — Safe to Fix Now (No System Impact)
| # | Issue | Fix | Impact |
|---|---|---|---|
| 1 | Prettier: 188 files failing | pnpm format (prettier --write .) | Consistent code style across project |
| 2 | ESLint 16 warnings | Remove unused imports / prefix with _ | Clean lint output, less noise |
| 3 | console.log in 19 places | Remove debug logs from workers, webhooks, API routes | Production doesn't print unnecessary data |
| 4 | 7 unused dependencies | pnpm remove date-fns docx googleapis pdfkit react-dropzone | Lighter node_modules, faster installs |
| 5 | No Git Hooks | npx husky init + pre-commit (lint, format, tsc) | Prevents broken code from entering repo |
| 6 | .env.example out of sync | Add GEMINI_API_KEY to .env.example | Easier onboarding for new developers |
| 7 | API routes missing try/catch | Add try/catch + return 500 in 4 routes | Prevents unhandled errors in production |
| 8 | hono vulnerabilities (2 high) | pnpm update or overrides in package.json | Reduces vulnerability count |
These 8 quick wins can be done in 30 minutes. Most are just a single command — pnpm format fixes 188 files at once, pnpm remove drops unused deps in one go.
188 files failed formatting — but fixed with a single command. That's what the QC Dashboard helps with: identifies the problem + tells how to fix it + AI does it.
AI Fixed Everything — Results
The instruction to AI: "Go ahead and do everything" — same as every category.
| # | What Was Done | Before → After |
|---|---|---|
| 1 | Deleted empty admin directories (3) | Build FAIL → Build PASS |
| 2 | Fixed registerSchema + confirmPassword | 1 test FAIL → 9/9 tests PASS |
| 3 | Prettier formatted entire project | 188 files failing → 0 |
| 4 | Removed unused imports (16 warnings) | 16 warnings → 0 |
| 5 | Removed console.log from production code | 19 instances → 0 |
| 6 | Removed unused dependencies (5) | 7 unused → 2 (used in CSS) |
| 7 | Synced .env.example | Missing GEMINI_API_KEY → complete |
| 8 | Added try/catch to API routes | 4 routes missing → all routes covered |
Scores After Fix
| Topic | Score | Result |
|---|---|---|
| ESLint 0 errors | ✅ 0 errors 0 warnings | ✅ PASS |
| Prettier format | ✅ 0 files with issues | ✅ PASS |
| TypeScript strict | ✅ 0 errors | ✅ PASS |
| Unit Tests | 9/9 pass | ⚠️ Passes but low coverage |
| Component Test | ❌ None yet | ❌ Need to write more |
| API Route Test | ❌ None yet | ❌ Need to write more |
| Dep Vulnerabilities | 0 critical | ✅ PASS |
| console.log | 0 instances | ✅ PASS |
| Env Variables | .env.example synced | ✅ PASS |
| DB Migration | Still using db push | ❌ Need to switch to migrate |
| API Error Handling | All routes covered | ✅ PASS |
| Input Validation | Zod + refine | ✅ PASS |
| Unused deps | Removed 5 | ✅ PASS |
| Circular imports | 0 | ✅ PASS |
| Secrets in codebase | 0 leaks | ✅ PASS |
18 items audited → 12 pass, 1 warning, 2 fail → ~81% (up from 52%, +29%)
Lessons Learned from the Development Category
This category teaches an important lesson: "it works" and "it works well" are very different. Code might build successfully, deploy fine, have zero TypeScript errors — but inside there could be 188 files with inconsistent formatting, 19 debug logs leaked into production, and 7 unused dependencies loaded on every install.
The interesting part: most problems can be fixed with a single command. Prettier formatting the entire project takes less than a second. Removing unused deps is just pnpm remove. But without a checklist — there's no way to even know there's a problem.
What AI couldn't do this round: write meaningful tests. Component tests and API tests need someone to define "what needs testing". AI can generate test boilerplate, but valuable tests need to come from someone who understands the business logic. These are the 2 items still failing — and they require real effort.
Deploy & Monitor — From 50% to 90% with AI
Why Is Deploy the Final Boss That Most People Fail?
Clean code, tests passing, good performance — but when it's time to actually deploy, everything breaks from step one. Build fails, env vars wrong, DNS not pointed, no migrations — problems that never showed up during dev.
The Deploy & Monitor category isn't just "click deploy and done" — it's verifying that the system is truly production-ready, from build, database, DNS, SSL, monitoring, all the way to backup strategy.
50% of the Deploy checklist failed — even though the dev server had been running perfectly fine the whole time. Production and Development are completely different worlds.
What Do the 17 Items Check?
[ROLE] You are a Senior QC Engineer auditing a web project [PROJECT] Framework: nextjs | Path: ./ [SCOPE] Audit only — do not modify any files Audit "🚀 Deploy & Monitor" — 17/17 items: 1. [CRITICAL] Production Build 0 Errors 2. [CRITICAL] Env Variables Production complete 3. [CRITICAL] Database Migration Applied 4. [HIGH] DNS Configuration OK 5. [HIGH] SSL Certificate Valid 6. [CRITICAL] Smoke Test: Homepage 7. [CRITICAL] Smoke Test: Login 8. [CRITICAL] Smoke Test: Core Features 9. [HIGH] API Health Check 10. [HIGH] Sentry Error Rate Normal 11. [MEDIUM] UptimeRobot Green 12. [MEDIUM] PageSpeed Maintained 13. [MEDIUM] SSL Labs A+ 14. [MEDIUM] Mozilla Observatory A+ 15. [MEDIUM] Search Console: No Errors 16. [HIGH] Backup Strategy Verified 17. [HIGH] Rollback Plan Documented
17 items split into: 6 CRITICAL, 6 HIGH, 5 MEDIUM. Important note — 7 items (#5, #8, #11-15) require a deployed URL, so AI can only audit 10/17 items; the rest need checking after deployment.
Scores Before Fix — Per Item
| # | Topic | Severity | Result |
|---|---|---|---|
| 1 | Production Build 0 Errors | CRITICAL | ✅ pnpm build succeeded |
| 2 | Env Variables Production complete | CRITICAL | ⚠️ 36 vars present but values still dev (localhost, development) |
| 3 | Database Migration Applied | CRITICAL | ❌ No prisma/migrations/ — still using db push |
| 4 | DNS Configuration OK | HIGH | ❌ www.scanlyiq.com → NXDOMAIN not configured yet |
| 5 | SSL Certificate Valid | HIGH | ⏭️ SKIP — Requires deployed URL |
| 6 | Smoke Test: Homepage | CRITICAL | ✅ Static pages render from build |
| 7 | Smoke Test: Login | CRITICAL | ✅ /login is a static page |
| 8 | Smoke Test: Core Features | CRITICAL | ⏭️ SKIP — Requires running server + DB |
| 9 | API Health Check | HIGH | ⚠️ Has /api/health but doesn't actually check DB/Redis |
| 10 | Sentry Error Rate | HIGH | ⚠️ Config present but SENTRY_DSN empty — not configured yet |
| 11 | UptimeRobot Green | MEDIUM | ⏭️ SKIP — Requires deployed URL |
| 12 | PageSpeed Maintained | MEDIUM | ⏭️ SKIP — Requires deployed URL |
| 13 | SSL Labs A+ | MEDIUM | ⏭️ SKIP — Requires deployed URL |
| 14 | Mozilla Observatory A+ | MEDIUM | ⏭️ SKIP — Requires deployed URL |
| 15 | Search Console: No Errors | MEDIUM | ⏭️ SKIP — Requires verified property |
| 16 | Backup Strategy Verified | HIGH | ✅ Fully documented — DB daily, uploads weekly, restore procedure |
| 17 | Rollback Plan Documented | HIGH | ⚠️ Has backup/restore but no Dockerfile — rollback is hard |
10 items audited → 4 pass, 3 warnings, 2 fail, 1 skipped → 50%
Interesting thing: Backup Strategy passed — the deployment guide has ready-made scripts for daily DB backups, weekly upload backups, remote backup, and complete restore procedures. But the problem: no Dockerfile yet — the entire deployment plan was written around Docker but the Dockerfile hadn't been created.
ZONE 1 — Fixes That Could Break Things
🔴 ZONE 1 — Backup Before Fixing
| # | Issue | Why It's Risky | Safe Fix |
|---|---|---|---|
| 1 | No Prisma migrations | db push in production may cause data loss — no migration history for rollback | Create baseline migration: npx prisma migrate diff --from-empty --to-schema-datamodel then resolve in dev first |
| 2 | Env vars still have dev values | Deploying with NODE_ENV=development + AUTH_URL=localhost exposes data and breaks auth | Create separate .env.production — verify all values before deploy |
Item 1 is the most critical — db push works fine during dev, but in production it might drop columns or recreate tables if the schema changes. Prisma Migrate prevents this with migration history that tracks every change.
ZONE 2 — Quick Wins to Fix Right Away
🟢 ZONE 2 — Safe to Fix Now (No System Impact)
| # | Issue | Fix | Impact |
|---|---|---|---|
| 1 | DNS not configured | Add A record → VPS IP at DNS provider | Domain becomes accessible |
| 2 | Health endpoint doesn't check DB/Redis | Add Prisma $queryRaw + Redis ping to /api/health | Monitoring can actually detect outages |
| 3 | Sentry not configured | Create Sentry project + fill in DSN, ORG, PROJECT | Error tracking actually works |
| 4 | No Dockerfile | Create multi-stage Dockerfile (deps → build → runner) | Can deploy with Docker + easy rollback |
AI Fixed Everything — Results
The instruction to AI: "Go ahead and do everything"
| # | What Was Done | Before → After |
|---|---|---|
| 1 | Created Prisma baseline migration | No migrations/ → 1 baseline (1,068 lines SQL) + resolve applied |
| 2 | Improved /api/health | Just returns "ok" → Checks DB ($queryRaw SELECT 1) + Redis ping for real |
| 3 | Created Dockerfile (multi-stage) | None → 3-stage build (deps → build → runner) + standalone output |
| 4 | Created docker-compose.prod.yml | None → app + postgres + redis + health checks + volumes |
| 5 | Created .env.production template | None → 108 lines with comments for every value |
| 6 | Fixed Resend lazy init | Build broke because Resend API key was empty → lazy init prevents build failure |
Scores After Fix
| Topic | Score | Result |
|---|---|---|
| Production Build | ✅ 0 errors | ✅ PASS |
| Env Variables | .env.production template ready | ✅ PASS |
| DB Migration | Baseline applied | ✅ PASS |
| DNS | Not configured yet (manual step) | ❌ Need to set A record |
| Smoke Tests (Homepage/Login) | Build passes + pages render | ✅ PASS |
| API Health Check | Checks DB + Redis for real | ✅ PASS |
| Sentry Config | Config ready, awaiting DSN | ⚠️ Need to create project |
| Backup Strategy | Fully documented | ✅ PASS |
| Rollback Plan | Dockerfile + docker-compose ready | ✅ PASS |
10 items audited → 8 pass, 1 warning, 1 fail → ~90% (up from 50%, +40%)
Lessons Learned from the Deploy & Monitor Category
This category has an important limitation: 7 of 17 items require a deployed URL (SSL Labs, PageSpeed, Mozilla Observatory, UptimeRobot, Search Console, SSL cert, core feature smoke test). So Deploy QC needs 2 rounds — before deploy (audit from code) and after deploy (audit from live URL).
What AI does well here is preparing infrastructure — creating Dockerfile, docker-compose, .env template, migration baseline, health endpoint. Everything that requires "creating files" AI handles perfectly. But what AI can't do is actually deploy — setting up DNS, creating a Sentry project, provisioning a VPS still has to be done manually.
Key lesson: Empty Resend API key = broken build. Many libraries throw errors during instantiation if the API key is an empty string — lazy initialization is needed (create the instance when actually called, not on import). This kind of thing is impossible to debug without actually trying a real build.
Accessibility — From 44% to 100% with AI
The Problem — A Site That "Works" But Isn't "Accessible"
Accessibility is the category devs tend to skip because "it looks fine to the eye" — but when actually tested with pa11y + Lighthouse, the Landing page scored just 96%, Login page 98%. Sounds high, but WCAG AA requires 100% to pass.
The QC Checklist of 16 items found 9 failures — only 44% passing.
The Prompt Used
Audit Accessibility of ScanlyIQ web app per WCAG 2.1 AA: 1. Check landmark structure — does it have main, nav, header? 2. Check for skip navigation link 3. Check aria-label on all icon-only buttons 4. Check alt text on all images — must be descriptive 5. Check touch targets ≥ 44px on all interactive elements 6. Check focus styles — every focusable element needs a visible indicator 7. Check color contrast ratio ≥ 4.5:1 (WCAG AA) 8. Check prefers-reduced-motion support Measure with pa11y + Lighthouse before/after fixes
ZONE 1 — Issues AI Found + Fixed Immediately (8 Items)
🔴 ZONE 1 — Issues AI Found from Prompt
| # | Issue | Severity | Fix |
|---|---|---|---|
| 1 | No <main> landmark in auth layout | 🔴 High | Change <div> → <main id="main-content"> |
| 2 | No Skip Navigation Link | 🔴 High | Add sr-only link + focus:not-sr-only in root layout |
| 3 | Icon buttons missing aria-label (bell + user menu) | 🟡 Medium | Add aria-label="Notifications" / "User menu" in topbar |
| 4 | Alt text not descriptive on slip-ocr page | 🟡 Medium | Changed from "image" → actual content descriptions + aria-label on all buttons |
| 5 | Progress dots touch target 8px | 🟡 Medium | Added padding for 44px clickable area (WCAG minimum) |
| 6 | DateCell has no focus style | 🟡 Medium | Added focus-visible:ring-2 in field-cell.tsx |
| 7 | Insufficient color contrast in trusted-by section | 🟡 Medium | text-slate-500 → text-slate-400 (contrast 4.04:1 → ~5.6:1) |
| 8 | No prefers-reduced-motion support | 🟡 Medium | Added @media (prefers-reduced-motion: reduce) in globals.css |
ZONE 2 — Additional Issues Found by Lighthouse
🟢 ZONE 2 — Issues Tools Caught After ZONE 1 Fixes
| # | Issue | Fix | Impact |
|---|---|---|---|
| 1 | Low disabled link contrast — "Forgot password?" faded on dark bg (contrast 2.33:1) | Added aria-disabled="true" + tabIndex={-1} + changed color for contrast ≥ 4.5:1 | Lighthouse Login 98% → 100% |
AI Fixed Everything — Results
The instruction to AI: "Audit all Accessibility per WCAG AA — fix everything"
| # | File Changed | Before → After |
|---|---|---|
| 1 | layout.tsx (auth) | No main landmark → <main id="main-content"> |
| 2 | layout.tsx (root) | No skip nav → sr-only link appears on focus |
| 3 | topbar.tsx | Icon buttons had no label → aria-label on both buttons |
| 4 | slip-ocr page.tsx | alt="image" → descriptive alt + aria-labels + touch target 44px |
| 5 | field-cell.tsx | No focus ring → focus-visible:ring-2 ring-blue-500 |
| 6 | trusted-by-section.tsx | contrast 4.04:1 → ~5.6:1 (text-slate-400) |
| 7 | globals.css | No reduced-motion → disabled all animations + transitions |
| 8 | login/page.tsx | disabled link contrast 2.33:1 → ≥4.5:1 + aria-disabled |
Scores After Fix
| Tool | Page | Before | After |
|---|---|---|---|
| Lighthouse | Login | 98% | 100% |
| Lighthouse | Landing | 96% | 100% |
| pa11y | Login | 1 error | 0 errors |
| pa11y | Landing | 1 error | 0 errors |
| TypeScript | Entire project | 0 errors | 0 errors |
16 items audited → all 16 pass → 100% (up from 44%, +56%)
Lessons Learned from the Accessibility Category
An 8px touch target works with a mouse but not with a finger — progress dots smaller than 44px fail WCAG minimum. Looks "clickable" to the eye, but on an actual mobile device, nearly impossible to tap. AI catches this incredibly fast — just ask "which interactive elements have touch targets below 44px" and the answer comes instantly.
Another interesting finding: Lighthouse scores of 96-98% sound great but still don't pass WCAG AA. The failing issue was color contrast on a disabled link (2.33:1 when ≥4.5:1 is required). "Disabled" elements still need sufficient contrast — because screen reader users still see those elements.
Key lesson: prefers-reduced-motion is easy to forget — a @media rule needs to be added to CSS to disable animations/transitions for users who need it. AI creates this CSS rule in 10 seconds, but forget it and there's no way to be WCAG compliant.
SEO & Standards — From 38% to 91% with AI
The Problem — A Site That "Exists" But Google Can't Find
SEO & Standards was the last category audited — and the one with the most problems. Lighthouse SEO score on the Landing page was just 91%, which sounds fine, but digging into details revealed: sub-pages had no meta titles at all, no OG image, no structured data, no sitemap, no robots.txt.
The QC Checklist of 21 items found 13 failures — only 38% passing.
The Prompt Used
Audit SEO & Web Standards of ScanlyIQ web app per checklist: 1. Check meta title + description on every page (unique, appropriate length) 2. Check Open Graph tags — og:title, og:description, og:image 3. Check Twitter Card tags — twitter:card, twitter:image 4. Check Structured Data (JSON-LD) — Organization, WebApplication, FAQPage 5. Check Canonical URLs + metadataBase 6. Check robots.txt + sitemap.xml 7. Check Semantic HTML — heading hierarchy (H1 → H2 → H3 no skipping) 8. Check Image alt text on all images 9. Measure Lighthouse SEO score before/after fixes Use Lighthouse + Google Rich Results Test to audit
ZONE 1 — Issues AI Found + Fixed Immediately (13 Items)
🔴 ZONE 1 — Issues AI Found from Prompt
| # | Issue | Severity | Fix |
|---|---|---|---|
| 1 | Sub-pages have no meta title — Login, Register, Pricing, Dashboard | 🔴 High | Added metadata export to every page (title + description) |
| 2 | No OG image — sharing links shows no image preview | 🔴 High | Added openGraph config + og:image in root layout |
| 3 | No Twitter Card | 🟡 Medium | Added twitter: { card: "summary_large_image" } to metadata |
| 4 | No Structured Data — Google doesn't understand the site | 🔴 High | Added JSON-LD: Organization + WebApplication + FAQPage schemas |
| 5 | No metadataBase — incomplete canonical URLs | 🟡 Medium | Added metadataBase: new URL("https://scanlyiq.com") in root layout |
| 6 | No robots.txt | 🟡 Medium | Created src/app/robots.ts — Allow all + link to sitemap |
| 7 | No sitemap.xml | 🔴 High | Created src/app/sitemap.ts — auto-generated from routes |
| 8 | Heading hierarchy skips levels — H1 → H3 (skips H2) | 🟡 Medium | Fixed heading tags to proper order |
| 9 | Landing page missing lang attribute | 🟡 Medium | Added lang="th" to root layout |
| 10 | API routes missing force-dynamic — build fails during page data collection | 🔴 High | Added export const dynamic = "force-dynamic" to API routes using Resend |
| 11 | IORedis type error during build | 🔴 High | Fixed dynamic import + type assertion in health route |
| 12 | Dashboard pages have no metadata | 🟡 Medium | Added metadata export to all dashboard pages (slip-ocr, transcription, projects, etc.) |
| 13 | FAQ section missing FAQPage schema | 🟡 Medium | Added JSON-LD FAQPage to landing page FAQ section |
ZONE 2 — Manual Steps Required
🟢 ZONE 2 — Requires Deploy + External Configuration
| # | Item | Why AI Can't Do It | Status |
|---|---|---|---|
| 1 | Google Search Console — verify ownership | Requires Google account login + domain verification | ⚠️ Manual step |
| 2 | Actual OG image file — needs design + upload | AI can create config but no actual image in public/ | ⚠️ Need to upload image |
AI Fixed Everything — Results
The instruction to AI: "Audit all SEO — meta tags, structured data, sitemap, robots.txt — fix everything"
| # | What Was Done | Before → After |
|---|---|---|
| 1 | Meta titles for all pages | Only Home had a title → every page now has unique title + description |
| 2 | Open Graph tags | None at all → og:title, og:description, og:image complete |
| 3 | Twitter Card | None → summary_large_image on all pages |
| 4 | JSON-LD Structured Data | None → Organization + WebApplication + FAQPage schemas |
| 5 | robots.ts | None → Allow all crawlers + sitemap link |
| 6 | sitemap.ts | None → Auto-generated from routes (/, /login, /register, /pricing) |
| 7 | metadataBase | None → complete canonical URLs |
| 8 | force-dynamic on API routes | Build failed during data collection → Build compiles successfully |
Scores After Fix
| Tool | Page | Before | After |
|---|---|---|---|
| Lighthouse SEO | Landing (/) | 91% | 100% |
| Lighthouse SEO | Login | No title | Has title + complete meta |
| Lighthouse SEO | Pricing | No title | Has title + meta complete |
| Structured Data | Landing | None | 3 schemas (Organization, WebApplication, FAQPage) |
| TypeScript | Entire project | 0 errors | 0 errors |
| All pages HTTP | login, register, pricing, home | 200 | 200 |
21 items audited → 19 pass, 2 awaiting manual → ~91% (up from 38%, +53%)
Lessons Learned from the SEO & Standards Category
Structured Data (JSON-LD) makes Google "understand" the site — not just crawl it. Organization schema tells Google what company owns the site, WebApplication says it's a SaaS product, FAQPage enables FAQ Rich Snippets in search results. AI created all 3 schemas in under 1 minute — but without a Checklist, there's no way to know they're needed.
Another issue where AI helped a lot: Build failure from empty Resend API key. Next.js tries to collect page data during build, causing API routes to be instantiated — if a library throws an error in its constructor (like Resend requiring an API key), the entire build breaks. The fix is adding export const dynamic = "force-dynamic" so Next.js skips page data collection. AI debugged this instantly because it could see the error stack trace.
Key lesson: SEO that "looks good" (91%) may miss critical things — a high Lighthouse SEO score doesn't mean Google understands the site. Structured data, sitemap, robots.txt, unique meta titles on every page are all needed. Using only Lighthouse without a Checklist means missing structured data and sitemaps entirely.
Final Results — All 8 Categories Complete
All Results
| Category | Before | After | Increase |
|---|---|---|---|
| Functional Testing (19 items) | 56% | 87% | +31% |
| Planning (10 items) | 33% | 85% | +52% |
| Security (20 items) | 71% | 100% | +29% |
| Performance (19 items) | 45% | 82% | +37% |
| Development (21 items) | 52% | 81% | +29% |
| Deploy & Monitor (17 items) | 50% | 90% | +40% |
| Accessibility (16 items) | 44% | 100% | +56% |
| SEO & Standards (21 items) | 38% | 91% | +53% |
| All 8 Categories (163 items) | 49% | 89% | +40% |
Lessons Learned from AI QC-QA Across All 8 Categories
AI Handles 80% Right Away
Out of 163 items, AI found and fixed about 130 — leaving only ~33 that require manual work (deploy, DNS, third-party services)
8 Categories in 1 Day
Manually auditing 163 items takes 3-5 days. With AI auditing + fixing, all 8 categories were completed in a single day
Checklists Beat Tools
Lighthouse gives high scores but misses structured data, sitemap, migration baseline. A 163-item checklist provides far more comprehensive coverage
ZONE 1 + ZONE 2 = Complete
Every category has 2 zones: ZONE 1 that AI handles, ZONE 2 that's manual. Knowing what requires manual effort helps with better planning
All categories complete — 8 categories, 163 items, from 49% to 89%. Every prompt and checklist used is available in the QC Dashboard, ready to use immediately.
Try Enterprise 360 QC-QA Tools Now
236 Prompts, 32 Tools, 195 Checklist Items — all 8 categories ready to use, no installation needed
Open QC Dashboard →Related Articles

I Built idea2logic.com with AI — Inside the Architecture of 30+ Pages & 40+ APIs
I built idea2logic.com entirely with AI — 30+ pages, 40+ APIs, 14 database tables. This article opens up the full architecture with Interactive Diagrams.
SYNERRY Enterprise Platform Engine — The Full Story Behind Building a New System from Scratch
When SYNERRY's legacy system hit its breaking point, a survey went out to 7 devs. The result: 40 issues across 14 categories. This is the behind-the-scenes story of every decision — from the problems discovered to the Tech Stack chosen.
Simple Prompts with Clear Logic — How AI Found 25 Free QC Tools Covering 208 Web Testing Checklist Items
Using 3 rounds of plain-language prompts, AI found 25 free QC tools covering 208 web application testing topics and built an entire QC Dashboard — all in under 1 hour at zero cost.