Picture the scene: a regulator asks for the decision trail behind a single AI-driven customer outcome from six months ago. The policy binder is on the shelf. The model card exists.
The committee met. But the actual record, who approved what, which endpoint the model was calling that week, which reviewer overrode a risk tier, sits scattered across Slack threads, ticket comments, and a spreadsheet nobody has opened since Q1. That gap between what the framework says and what the systems can prove is where AI governance programs quietly fail.
The gap widens quickly when AI drifts, use cases expand beyond their original scope, and shadow AI proliferates faster than any static document can keep up with. An intelligent workflow platform gives teams a governed way to connect approvals, technical controls, AI actions, and human review across the systems where governance runs.
This article walks through the process of building an AI governance framework from scratch. You'll learn how to choose (or layer) NIST AI RMF, ISO 42001, and the EU AI Act, structure ownership, inventory use cases, turn principles into testable controls, and keep the whole thing out of documentation-only mode. It's compliance and security-led by topic, but the build pattern transfers to any function running AI in production.
What an AI governance framework means before you build one
An AI governance framework translates policy and program work into roles, controls, and evidence, connecting your risk, compliance, and technology functions on a single operating model.
The framework defines policies, decision rights, technical controls, and oversight across the AI lifecycle. Governance principles define what good governance should look like, while a framework defines how you implement it. Principles often live in a slide deck. A framework translates them into specific roles and controls, and the governance program is the resourced effort that runs it day-to-day.
Named standards give teams a starting point. NIST published the NIST AI RMF (AI RMF 1.0) in January 2023, and it organizes AI risk into four functions. ISO 42001 is the first AI management system standard and provides a certifiable framework.
The EU AI Act supplies a risk-tier taxonomy with regulatory obligations for organizations in scope. Understanding how these three fit together is the first real decision you'll make.
How to choose your foundation: NIST AI RMF, ISO 42001, or a blend
Match the standard to the outcome you need first, rather than picking a single winner: each of NIST AI RMF, ISO 42001, and the EU AI Act answers a different question, and the right foundation depends on what you're solving for.
Practitioners who've built governance programs across multiple regulatory regimes consistently treat these standards as complementary rather than competing. The practical decision comes down to what you're solving for in the next 6 to 12 months.
Start with the NIST AI RMF when you need an adaptable risk-management vocabulary and operating model, but no regulatory or certification deadline is forcing your hand. Its context-specific design lets teams calibrate depth to the risk in front of them, and its four functions (Govern, Map, Measure, Manage) apply as lifecycle functions across every stage of AI work.
Start with ISO/IEC 42001 when leadership needs a certifiable management system, when procurement or customer contracts are asking for one, or when the organization already runs on ISO 27001 and 9001 and wants a familiar structure. You get a defined backbone of policies, roles, risk and impact assessments, documentation, monitoring, and continual improvement.
Start with the EU AI Act when you're in scope for its obligations, because its four risk tiers (unacceptable, high-risk, limited, and minimal) drive concrete requirements you can't opt out of. Even out-of-scope teams often adopt it as a de facto classification standard, since it's the most concrete regulatory yardstick available.
For most enterprises, the answer is a blend. The typical sequence starts with classifying use cases against EU AI Act tiers to establish a regulatory posture and push high-risk work to the top of the queue. Teams then adopt the NIST AI RMF functions as the day-to-day risk-management operating model, and layer ISO 42001 on top as the certifiable management system that turns that operating model into an auditable structure.
A well-built framework maps controls across all three simultaneously, so a single assessment satisfies each without tripling the work. If you're resource-constrained, start with the EU AI Act classification and NIST functions, then layer ISO 42001 when the certification case is ready.
The NIST AI RMF organizes governance work into four functions (Govern, Map, Measure, and Manage) that run in sequence but continue to operate in parallel once you're up and running. The next three sections walk through each in the order most teams tackle them.
Step one: govern (structure and ownership before anything else)
Establish the governance committee, its decision rights, and its escalation paths before you write a single control. Skipping this step is one reason frameworks stall, because without documented ownership, a model that produces unexpected outputs has no accountable executive to catch it.
A committee is necessary, but it isn't sufficient on its own, and treating it like a silver bullet is its own failure mode. A charter and a meeting cadence do not, by themselves, stop shadow AI or shrink the gap between policy and what's actually running in production. Committees set direction and own escalation; they don't substitute for the operational path that lets employees build and run AI safely day to day.
Without that path, the committee ends up governing a shrinking slice of what the business is actually doing, while everything else moves underground. The two have to be built together: a governance body with real decision rights, paired with a safe, sanctioned environment employees can build in without waiting on that committee for every routine decision.
The standing committee needs cross-functional representation to cover the full risk surface: a CISO for security and AI attack surface risk, a CTO for technology and infrastructure decisions, a Chief Data Officer for data governance, a Chief Privacy Officer or DPO for privacy obligations, a General Counsel for legal exposure, an AI/ML lead for technical depth, and business-unit representatives who own the use cases in production.
The executive steering committee or board must ratify the charter that grants the committee authority, because NIST expects teams to clearly document AI risk-management roles, responsibilities, and lines of communication.
Decision rights determine whether governance works or degrades into a debating society. Document explicitly who approves low-risk versus high-risk initiatives, what the thresholds are, and when a decision escalates.
Step two: map (inventory and risk classification)
You build the AI system inventory before you write a single control, then classify each use case by risk tier. NIST AI RMF GOVERN 1.6 tells teams to inventory AI systems and assign resources according to risk priorities. Without a reliable inventory, downstream control cannot be reliable.
Use the inventory to find shadow AI and tools or features adopted outside procurement. Shadow AI creates exposure that inventory and access controls need to catch. Each entry in the inventory should capture the fields that drive risk decisions downstream:
Tool and specific feature
Named owner
Most sensitive data classification is handled
Vendor and model
Hosting and residency
Access scope
Business purpose
Resulting risk tier
Data classification and access scope are central to the risk decision, so an inventory that misses those fields cannot support reliable governance. And it's a living artifact: a static inventory goes stale quickly as AI features get bundled into renewals.
For classification, adapt the EU AI Act tiers. High-risk covers Annex III domains like employment, biometrics, and critical infrastructure. Set risk thresholds around regulatory exposure and the cost of failure for customers. When a case is genuinely unclear, escalate for review; a lower-risk tier needs support. Don't over-apply that rule, or most functions land in a high-risk quadrant and investment stalls.
Step three: measure and manage (turning principles into controls)
Measure and Manage turn fairness, transparency, and accountability into testable controls. The gap between ethical principles and technical enforcement has a name, the Responsible AI Gap, and closing it separates governance that works from governance that photographs well.
Fairness becomes a control when teams attach named metrics and a repeatable audit process, using measures such as demographic parity difference and equalized odds difference, tested with libraries such as IBM AI Fairness 360 or Fairlearn.
Transparency becomes a control through model cards and datasheets that capture intended use, performance, limitations, and subgroup results. Accountability becomes a control through named ownership and independent validation, with high-risk systems often requiring red-team exercises.
Even so, Measure and Manage often break down in practice. Teams complete Govern and Map on paper, skip Measure for lack of data infrastructure, and let Manage become "whatever happens when an incident occurs." Teams that embed controls directly into AI systems show stronger evidence than those relying on manual governance.
Audit logs and workflow monitoring let teams preserve the evidence trail across deterministic workflow steps, AI actions, and human approvals. Gartner predicts that by 2027, 40% of enterprises will see autonomous AI efforts partly derailed by governance gaps discovered only after production incidents.
Common mistakes that stall a framework before it starts
Treating governance as a legal exercise and building for certification can kill frameworks before they change production behavior. Both legal-only governance and certification-first governance produce a binder that satisfies an audit while leaving production behavior unchanged.
Treating governance as legal work. This concentrates it in the wrong room. Security, data, and compliance teams each see only part of the risk. A model that passes security review, uses clean data, and satisfies a regulatory checklist can still produce biased or reputationally damaging output if nobody owns the whole picture. Operational infrastructure must capture risk in motion through systems that record use, decisions, and changes.
Building for certification. This substitutes documentation compliance for operational effectiveness. ISO 42001 supports voluntary certification, and the certification can quietly become the goal. Ask whether your team could reconstruct a costly AI error tomorrow, or only show the regulator an ethics charter.
Avoiding both traps means designing governance that changes what happens in production, not just what appears in the audit binder.
What "from scratch" looks like in practice: build, pilot, and scale
A practical build starts with a working committee and first governed systems, then scales across more use cases. Sequence the phases around dependencies, since each produces input for the next.
Start with gap analysis and inventory. Catalog every AI system in production, pilot, and shadow use, benchmark against the regulations you're in scope for, and produce a risk classification.
Establish foundations by defining roles, documenting use cases, and assessing readiness. Next, build the framework and stand up the committee: governance policies, monitoring procedures, incident response protocols, decision gates, and a RACI-based accountability structure spanning security, legal, technology, privacy, and the business.
In parallel, run a focused pilot governance sprint. Select two or three high-priority AI systems and apply the full framework: inventory, risk assessment, policy mapping, monitoring setup, and accountability assignment. The sprint surfaces gaps before you scale, giving the committee its first proof point. From there, scale and automate, moving governance from repeatable processes into metrics-driven controls integrated into the AI development lifecycle.
Knowing the framework works comes down to system-level measurement. Track compliance and risk signals alongside model performance signals like drift and fairness trends, plus adoption metrics like how many employees follow governance processes.
Use control-drift detection to catch gaps between documented policy and operational reality before an auditor asks you to reconstruct events from ad hoc logs. A working framework detects control drift before an audit. If evidence appears only after a manual scramble, you have documentation rather than governance.
Governance that holds up in production
Building an AI governance framework from scratch starts by keeping it out of documentation-only mode. The standards covered here each answer a different question, and the strongest teams layer them. Working frameworks put rules in systems that produce proof. Weak frameworks leave them in binders.
The frameworks that stall are the ones built backward from a certification. The ones that hold up are built forward from how AI gets used, then hardened into infrastructure that regulators, auditors, and your own committee can inspect on demand.
If you're standing one up, start where the operational proof lives. Start building in the Community Edition, Tines' forever-free entry point with security-grade governance, SSO, AI, and unlimited integrations included, and put your first governance workflow into practice before you write another document.
Ready to see governed AI workflows in action? Book a demo to see how Tines connects approvals, technical controls, AI actions, and human review in one operating model.
Frequently asked questions
What is an AI governance framework?
Use an AI governance framework to connect policies, decision rights, technical controls, ownership, and oversight across the AI lifecycle. It differs from a policy document because it defines how governance is implemented and enforced. Working frameworks connect governance principles to day-to-day systems, approvals, monitoring, and evidence.
What do you gain from combining NIST AI RMF, ISO 42001, and the EU AI Act?
A blend usually works best. NIST AI RMF gives you the risk-management vocabulary and operating model, ISO 42001 gives you a certifiable management-system structure, and the EU AI Act gives you a risk-tier taxonomy. Used together, they enable a single assessment to support multiple governance and compliance needs.
How does the EU AI Act change the requirements of the AI governance framework?
The EU AI Act introduces binding obligations tied to risk tiers, so an AI governance framework built for organizations in scope must map every use case to the Act's unacceptable, high-risk, limited, and minimal categories. High-risk systems carry the heaviest documentation, monitoring, and log-retention duties, and Annex III domains (employment, biometrics, critical infrastructure) will drive the bulk of framework work. Teams outside the EU still use the taxonomy as a de facto classification standard because it's the most concrete regulatory yardstick available.
What should you build first in an AI governance program?
Start with ownership and inventory. Establish the governance committee, decision rights, escalation paths, and accountable executives before writing detailed controls. Then, inventory AI systems in production, pilot, and shadow use so that every downstream control has a reliable foundation.
How do you know whether an AI governance framework is working?
A working framework produces operational proof. Teams should be able to show approvals, audit trails, monitoring signals, risk classifications, and control-drift detection on demand, rather than reconstructing them from ad hoc logs after an incident. Evidence that only appears under pressure is a sign the program is running on documents rather than systems.