Documentation freshness monitor with scheduled crawling and owner notifications
Build a Documentation Freshness Monitor that stops product docs from silently going stale. The workflow crawls the docs sites I register, flags pages that are past a staleness threshold as well as broken or missing ones, works out who owns each page, and posts an owner-routed review summary to a chat tool so the accountable person actually fixes them. Make it fully self-serve from a single web page (a React config UI on an authenticated route) where I can add docs sites and set, per site, things like a staleness threshold, a crawl cap, a schedule, and an ownership map (keyword → team/squad/PM) with optional per-owner chat destinations. Persist this config to durable storage through a small config API step that reads and writes it. Trigger scanning on an hourly cron that checks which sites are due based on each site's own schedule; for a due site, crawl its pages (fetched unauthenticated), classify each as stale / broken / fresh, and assign it to an owner. Then format a report — a summary plus "stale to review" and "broken to fix" sections grouped by owner — and route each owner's pages to their own channel, falling back to a site-wide destination. High-level steps: a web-based config UI; a config store API backed by persistent storage; a cron-triggered crawl-and-classify engine; and a reporting/notification step. Post the report to Slack — or your team's chat tool of choice (Teams, Discord, etc.) — via a connector. The only external services are the target docs sites (plain HTTP fetches) and your chosen chat tool; keep everything else read-only except writing config and sending the report. Tools used: React, TypeScript (Bun), 3B named volume storage, cron trigger, Slack connector, HTTP fetch of documentation sites
What this prompt builds
A documentation freshness monitor that crawls registered docs sites on a schedule, flags stale or broken pages against configurable thresholds, assigns each page to an owner using keyword-based ownership rules, and posts owner-routed review summaries to Slack. Configured entirely through a self-serve web UI with no code changes required. Replaces manual doc audits with automated, recurring scans that route fixes directly to accountable owners.
The problem
Documentation teams, product managers, squad leads, and engineers accountable for product docs get no signal when their pages go stale, break, or disappear—so docs silently rot until customers hit dead ends. Manual audits are time-consuming, infrequent, and lack accountability: "someone should review the docs" never turns into concrete action.
This workflow crawls registered docs sites on a per-site schedule, flags pages past a staleness threshold plus any broken or missing ones, determines each page's owner using configurable keyword rules, and posts an owner-routed review summary to Slack. The entire system—sites, thresholds, schedules, ownership maps, and chat destinations—is configured from a self-serve web page with no code editing required, turning vague doc-review intentions into scheduled, owner-targeted nudges that get fixes done.
Solution and impact
The workflow replaces manual documentation audits with zero-effort recurring scans that run on each site's own schedule, automatically classifies pages as stale, broken, or fresh against configurable thresholds, and routes fix requests directly to the accountable owner via Slack so reviews and repairs actually happen. Docs teams gain continuous visibility into documentation health, page owners receive targeted, actionable nudges instead of broadcast noise, and stale or broken docs get caught and fixed before customers encounter them—benefiting documentation teams, page owners, support, and end users alike.