Vendor bakeoff tool with AI-generated research and comparison matrix
Build a vendor bakeoff tool: a tenant-authenticated web app that helps a team compare vendors against each other by generating an AI-researched feature × vendor comparison matrix. What it does: A user creates a project by naming a few vendors. The workflow derives a relevant feature list, then researches every vendor against every feature using an AI model with web search, filling each matrix cell with a verdict, a supporting evidence note, and a source link. Users can add features or vendors later, re-check or ask for an explanation of any individual cell, manually override any verdict, organize projects into named collections, track each project's status (Not started / Researching / Evaluating / Decision made / Complete), and attach freeform documents (headings, text, lists, images, file uploads, etc.) to a project for write-ups and decisions. Maintain an admin allowlist (by email) that gates collection deletion and admin management. Trigger: All access is tenant-authenticated (signed-in users only, identified via the platform's injected authenticated-email header). There's no cron or inbound webhook — everything is driven by a person using the web UI, which then calls backend API routes. Structure: A frontend web page that renders immediately and fetches all data at runtime — the project/collection list, matrix view, and document editor. A read-only API step serving all read queries (projects, matrix data, per-cell detail, collections, current-user/admin info, documents, uploaded files). A write API step handling all mutations. Fast edits (manual overrides, renames, deletes, collections, status, document edits, uploads) run synchronously; slow AI research work (new project, add feature/vendor, redo research) is enqueued as a background job rather than blocking the browser. A background research worker (no direct route) that pulls queued jobs, runs the AI web-research per vendor/feature cell, and commits progress column-by-column so the UI's progress bar and matrix update live as it works. A small API step powering per-cell AI actions from the UI: explaining why a cell has its verdict, and re-checking/overwriting a single cell. Storage: Use a shared embedded database (e.g. SQLite on a persistent volume) for projects, matrix data, jobs, collections, and documents, with uploaded document files stored alongside it. Writers that mutate the database should serialize their writes (exclusive access) while the read API can mount it read-only. Design for forward migrations since the schema will grow over time. AI / research: Use an LLM with web-search tool access — Anthropic's Claude API is what was used here, or swap in another model provider that supports tool-augmented web search — to derive features, research each vendor/feature cell, produce verdicts with evidence and sources, explain a cell's reasoning on demand, and re-run research for a single cell. Keep every route tenant-authenticated by default; don't open anything to the public internet unless explicitly asked. Tools used: Anthropic Claude API, SQLite, 3B
What this prompt builds
A tenant-authenticated web app that automates vendor comparison research for IT and procurement teams. Users input vendor names and websites, and the workflow generates an AI-researched feature matrix focused on security, regulatory, and IT capabilities, with verdicts backed by evidence and sources. Teams can manually add features, re-check findings, track project status, attach documentation, and maintain an organized reference for future decisions.
The problem
IT managers and procurement teams evaluating new tools face a manual, time-consuming process to compare similar vendors — researching features, checking security and compliance claims, and documenting findings across scattered sources. This workflow automates vendor comparison by generating an AI-researched feature matrix focused on security, regulatory, and IT capabilities. Users input vendor names and websites, and the tool builds a structured comparison showing which features each vendor supports, backed by evidence, sources, and explanatory notes. Teams can add custom features, request re-checks, override verdicts, and keep all notes and documentation organized in one place for future reference.
Solution and impact
The workflow saves hours of manual research and delivers an unbiased, evidence-backed starting point for vendor evaluation. Teams get a structured matrix they can refine, with every finding traceable to a source and every decision tracked in a single, searchable location. The built-in documentation section keeps bakeoff notes organized and accessible, so future stakeholders can quickly understand the rationale and reuse past research.