Your Thursday briefing · AI series
Since late August, this series has laid down the pieces one by one: the inventory of AI use, who answers for it, the risks specific to language models, your vendors' AI, AI used against you, ISO 42001 and the NIST AI RMF, and last week AI agents. One piece is missing: the one that holds the others together.
Our position: AI governance you can't measure is just an intention. A signed policy, a committee and an open register don't tell you whether risk is going down. To know that, you need a few indicators, tracked over time, and one page leadership reads every quarter. That is what this episode is about: the full roadmap, on one page.
Why measure before you steer
The NIST AI RMF organizes AI risk management into four functions: Govern, Map, Measure, Manage. Measure uses "quantitative, qualitative, or mixed-method tools, techniques, and methodologies" to assess risk and its impacts, and its results feed Manage, which means allocating risk resources to mapped and measured risks on a regular basis. Put simply: without measurement, resources go where intuition sends them. For generative AI, the NIST AI 600-1 profile, published in July 2024, turns those four functions into concrete actions.
ISO/IEC 42001 says the same thing in management-system language. Its clause 9, performance evaluation, asks you to monitor and measure, run internal audits and hold a management review. An auditor won't ask whether you have an AI policy; they will ask how you know it works.
What this changes: the question for your team isn't "do we have AI governance?" but "what numbers did leadership see last quarter, and what did it decide?".
The roadmap on one page
Seven steps, in order. Each has an action, evidence that it is done, and an indicator that tells you whether it holds over time.
1. Inventory AI use. Action: a register of every AI system and tool, including those adopted without approval. Evidence: the dated register. Indicator: the share of discovered uses that appear in the register, and the date it was last updated.
2. Name who answers. Action: one owner per use, a person rather than a department, and a body that arbitrates. Evidence: the "owner" column filled in. Indicator: the share of uses with a named owner.
3. Treat model-specific risks. Action: for every system that touches sensitive data or customers, controls against prompt injection, data leakage and unchecked outputs. Evidence: one assessment per system. Indicator: the share of sensitive systems assessed in the last twelve months.
4. Assess your vendors' AI. Action: know which vendors use AI on your data, and under what contractual guarantees. Evidence: the question asked and the answer filed. Indicator: the share of critical vendors whose AI use is known.
5. Resist offensive AI. Action: out-of-band verification for every payment, change of banking details or urgent request from an executive. Evidence: the written procedure and an exercise. Indicator: the result of the last exercise, and the number of fraudulent requests the procedure stopped.
6. Pick your framework. Action: the NIST AI RMF to do the work, ISO 42001 the day someone demands proof. Evidence: a written decision, with its reason. Indicator: coverage of the selected controls, measured at the same pace as the rest of your compliance.
7. Keep agents in check. Action: a dedicated identity, least privilege, human approval for anything irreversible, logs and a kill switch for every agent. Evidence: the four extra columns in the register. Indicator: the share of agents that meet the six rules, and how long it really takes to stop one.
The indicators that matter, and the ones to drop
Seven steps give seven indicators. That is already a lot for leadership. Keep four or five on the quarterly dashboard and track the rest within the team.
Good indicators have three properties: they map to a step of the roadmap, they can get worse, and someone knows what to do when they do. "82% of uses have an owner" meets all three. "14 training sessions delivered this quarter" meets none: it measures activity, not risk.
Be wary of the single score, too. An overall "AI maturity: 3.2 out of 5" is reassuring, but it doesn't tell you what to fix. Leadership needs to know where the risk sits, not just how big it is. The percentages above are examples; your targets depend on where you start.
What to show the board
One page, four blocks, every quarter.
- Exposure. How many AI uses, how many agents, how many vendors using AI on your data, and the trend since last quarter.
- The top three AI risk scenarios, taken from the risk register and written the way our risk series suggests: a source, an event, a business process hit, a business consequence. For each, its owner and its residual level.
- The indicators, four or five, with their trend. An indicator that worsens comes with its explanation.
- The decisions requested. A budget, a use to stop, a risk to formally accept, a certification to launch or defer. If the page asks for no decision, it informs, but it doesn't govern.
What this changes: a board doesn't read technical detail. It decides. The page must give it something to decide on, and leave a record of what it decided.
The three traps
Measuring activity instead of risk. The number of policies published, awareness sessions held or tools bought always goes up. It says nothing about exposure.
Leaving measurement to IT alone. The technical team can count systems. On its own, it can't say which one matters most to the business. The owners from step 2 should sign off on their own numbers.
Measuring once. An annual assessment is a snapshot. AI changes from quarter to quarter: new tools, new agents, new features switched on by your vendors. Measurement has to keep the same pace.
This week's test
Answer three questions in writing:
- What AI numbers has leadership seen in the last three months?
- What decision did it take from them?
- Which of those numbers could get worse next quarter, and who would be alerted?
If the answer to the first question is "none", your AI governance is an intention. That's not a failure: the roadmap above tells you where to start.
What you should have in a quarter
The roadmap printed out, with a box ticked or dated for each of the seven steps. A table of four or five indicators with their starting values. Three AI risk scenarios in the register, each with an owner. And a first page presented to leadership, with at least one decision taken. That is your baseline; the next quarter will tell you whether risk is going down.
Where FortaRisks fits
Forta Cockpit is the decision-makers' view: your AI scenarios sit in the same register as every other risk, with their owner, inherent level and residual level, so the board page is built from current data rather than a spreadsheet. Forta Compliance assesses ISO 42001 and the NIST AI RMF among the 40+ frameworks mapped to the same control base: step 6 coverage is measured continuously, not once a year. To see how your own roadmap would look in the platform, request a demo.
The printable guide. This roadmap also exists as an eight-page PDF: the seven steps with their evidence and indicator, the quarterly board page to fill in, and a one-page checklist. Get the guide "AI security: the roadmap".
Next Thursday: your employees and consumer AI assistants. What leaves the company when a document is pasted into a chatbot, and the simple rules that prevent it without banning the tool.
Sources: NIST, AI Risk Management Framework · NIST AIRC, the four functions of the AI RMF core · NIST, AI 600-1, Generative AI Profile (July 26, 2024) · ISO, ISO/IEC 42001 explained · IEC, ISO/IEC 42001:2023