Risk Assessment

The Two Extremes

Two companies I’ve observed made opposite mistakes when adopting AI.

Company A moved too slowly. Every AI application required legal review, security assessment, and executive committee approval. The process took three to four months per use case. Six months after starting their “AI initiative,” they’d approved three simple applications—meeting summaries, internal task lists, and document formatting. Meanwhile, competitors automated dozens of workflows and gained meaningful productivity advantages.

Company B moved too fast. Excited by AI’s potential, they gave it broad access to customer data, internal communications, and financial systems. No formal assessment, just enthusiasm. Within weeks, a customer support bot shared confidential pricing information with a prospect. An HR screening tool created legal exposure by inconsistently evaluating candidates. An executive briefing system included unverified competitive intelligence in a board presentation.

Company A suffered opportunity cost—they lost ground to competitors by moving too slowly on low-risk applications. Company B suffered actual damage—customer trust eroded, legal exposure accumulated, and leadership credibility suffered.

The answer isn’t “go fast” or “go slow.” It’s knowing which situations require caution and which don’t.

Not all AI applications carry the same risk. A tool that summarizes your internal meeting notes has fundamentally different exposure than one that responds to customer inquiries. Treating them the same—either with excessive caution or insufficient scrutiny—leads to problems.

Part 3 taught you to build workflows. Part 4 addresses a question that’s been lurking behind every workflow decision: what data and access should AI have? This chapter introduces the Permission Framework—three questions that help you assess risk for any AI application.

The Permission Framework

The Permission Framework consists of three questions. Answer them honestly, and you’ll have the information needed to make a sound decision about whether and how to proceed.

Question 1: What’s the Specific Benefit?

Can you articulate a concrete, measurable improvement? Not “efficiency” or “productivity”—those are vague. How much time saved? How many errors prevented? What capacity unlocked?

If you can’t quantify the benefit, you can’t compare it to risk. “This will help us work better” isn’t a benefit statement—it’s a hope.

Question 2: What’s the Actual Exposure?

What data does AI access? What systems does it touch? Who is affected if something goes wrong? Be honest about the real risk surface.

People routinely underestimate exposure. “It only sees summaries” might be true—but what’s in those summaries? Customer names, deal values, internal assessments? “It doesn’t access anything sensitive” is rarely accurate when you look closely.

Question 3: What Are the Failure Modes?

When (not if) this fails, what happens? Can you detect failures? Can you recover? What’s the worst-case scenario?

AI will occasionally produce wrong outputs, inappropriate content, or unexpected results. The question isn’t whether failures will occur—it’s whether you’ve planned for them.

Why These Three Questions

The NIST AI Risk Management Framework uses four functions: Map, Measure, Manage, Govern. Enterprise frameworks run to dozens of pages. The Permission Framework distills the essential assessment into three questions that anyone can answer in thirty minutes.

These questions force concrete thinking. Vague answers reveal gaps: - “It will save time” → How much time? For whom? - “The risk is minimal” → What data is accessed? What if it fails? - “It usually works fine” → What happens when it doesn’t?

When you can’t answer clearly, you’re not ready to proceed. When you can, you have the information needed to make a decision.

This framework isn’t about slowing you down—it’s about enabling appropriate speed. Low-risk applications sail through the assessment in minutes. High-risk applications get the scrutiny they deserve. The framework prevents both paralysis and exposure.

Applying Question 1: Specific Benefit

Every AI application should have a quantified benefit statement. Without one, you’re guessing whether it’s worth the exposure.

How to Quantify Benefit

Time saved: How many hours per week, per task, or per output? Be specific. “The weekly customer summary takes 3 hours manually. With AI assistance, it takes 45 minutes. That’s 2.25 hours saved weekly—117 hours per year.”

Error reduction: What percentage decrease in mistakes? “Manual data entry has a 3% error rate. AI-assisted verification reduces errors to under 0.5%.”

Capacity increase: How much more output, or how much faster? “Currently process 50 support tickets daily. AI triage and draft responses would enable 80 tickets daily with the same team.”

Quality improvement: What measurable metric improves? “Customer satisfaction on responses averages 4.1/5. AI-assisted responses with consistent templates could target 4.4/5.”

The Benefit Bar

Not all benefits justify all risks. The relationship between benefit and acceptable exposure isn’t linear:

Low-risk applications: Even modest benefits justify proceeding. Internal meeting summaries that save 20 minutes daily are worth the minimal exposure of internal calendar and transcript data.

Medium-risk applications: Benefits need to be substantial. Customer communication assistance that saves 5 hours weekly needs strong safeguards given the exposure to customer data and external communication.

High-risk applications: Benefits must be compelling. AI-assisted hiring decisions that touch candidate PII and have discrimination potential require transformative benefits to justify the exposure.

If you can’t quantify a benefit that seems proportional to the exposure, pause. Either the application isn’t worth pursuing, or you haven’t thought clearly enough about what you’re trying to achieve.

The Clarity Test

Here’s a simple test: could you explain the benefit to your boss in one sentence with a number in it?

• “This saves our team 10 hours weekly on report preparation.” ✓
• “This improves efficiency.” ✗
• “This reduces error rates from 3% to 0.5% on customer communications.” ✓
• “This makes us more productive.” ✗

If your benefit statement doesn’t pass this test, keep refining until it does. The clarity will help you make better decisions about whether the application is worth pursuing.

Applying Question 2: Actual Exposure

Map the risk surface honestly. Most people underestimate what AI actually accesses.

Data Access Inventory

What information does this AI application touch?

Direct access: What data goes into the prompt or system? - Customer information (names, contact details, account data) - Financial data (pricing, revenue, costs) - Internal communications (emails, messages, documents) - Employee information (performance data, compensation, HR records) - Proprietary data (strategies, roadmaps, competitive intelligence)

Indirect access: What context or metadata is exposed? - Who’s involved in conversations - What topics are discussed - Timing and patterns of activity - Relationships between people and projects

System Touchpoints

Where does AI output go?

Internal only: Output stays within your organization. Lower exposure.

External communication: Output goes to customers, partners, or public. Higher exposure.

Systems of record: Output gets stored in CRM, HR systems, or financial systems. Creates persistent exposure.

Impact Assessment

Who is affected if something goes wrong?

Customers: Privacy violations, inappropriate communications, incorrect information

Employees: Unfair treatment, privacy violations, performance assessment issues

Partners: Confidentiality breaches, relationship damage

Regulators: Compliance violations, audit findings, enforcement actions

The organization: Reputation damage, legal exposure, competitive harm

Exposure Categories

Category	Examples	Typical Risk
Internal, non-sensitive	Meeting summaries, task lists	Low
Internal, some sensitive	Team performance, project status	Medium
Customer-facing, indirect	Support ticket triage, routing	Medium-High
Customer-facing, direct	Customer emails, chat responses	High
Regulated data	Healthcare records, financial data, HR decisions	Very High

Be honest about which category your application falls into. The temptation is to classify everything as “low risk” to avoid scrutiny. Resist it.

The Honest Assessment Test

Ask someone outside your immediate team to review your exposure assessment. People systematically underestimate risk for applications they’re excited about.

Common rationalizations to watch for: - “It’s just internal” — but internal data can still be sensitive - “We already share this information” — but sharing it differently changes risk - “The AI doesn’t store anything” — but it processes everything you give it - “Our vendor is trustworthy” — but trust doesn’t eliminate exposure

If you find yourself making these arguments, slow down. They’re often signs of wishful thinking rather than rigorous assessment.

Applying Question 3: Failure Modes

Plan for failure before it happens. Every AI application will eventually produce problematic output.

Categories of Failure

Accuracy failures: AI produces incorrect information - Customer receives wrong account balance - Report contains fabricated statistics - Summary misses critical decision from meeting

Inappropriate output: AI generates content it shouldn’t - Tone that offends the recipient - Language that creates legal exposure - Content that violates brand guidelines

Data exposure: AI reveals information inappropriately - Shares pricing with wrong customer - Includes confidential information in external communication - References private data in response

Bias and discrimination: AI treats groups unfairly - Screening tool disadvantages certain candidates - Customer prioritization based on inappropriate factors - Inconsistent treatment across demographic groups

Building Detection Mechanisms

For each failure mode, ask: How would we know?

Immediate detection: Human review catches problem before impact

Rapid detection: Monitoring or audit catches problem quickly

Delayed detection: Problem discovered through complaints, audits, or incidents

Undetected: Problem persists until major consequence surfaces

Aim for immediate or rapid detection. If your only detection mechanism is “customer complains” or “lawsuit filed,” your safeguards are inadequate.

Planning Recovery

For each failure mode, ask: What do we do when it happens?

Correction: Can the error be fixed?

Notification: Who needs to know, and how quickly?

Remediation: What makes the affected party whole?

Process change: How do we prevent recurrence?

Worst-Case Analysis

What’s the maximum plausible damage from this failure mode?

Not the average outcome—the worst realistic scenario. A customer service error might typically mean an apology and correction. The worst case might be a viral social media incident and significant reputation damage.

Consider these real-world examples of AI failures:

• Air Canada chatbot: Provided incorrect refund policy information. Cost: $812 settlement plus legal fees and reputation damage.

• Lawyer AI case citations: Attorney submitted AI-generated brief with fabricated legal citations. Cost: Sanctions, reputation damage, career impact.

• HR screening bias: Amazon’s AI recruiting tool discriminated against women. Cost: Abandoned system, reputation damage, process overhaul.

The common thread: failures that seemed unlikely in advance became significant incidents. Plan for the worst case, not the average case.

If the worst case is unacceptable, either add safeguards to make it acceptable or don’t proceed.

Making the Decision

With answers to all three questions, you can make an informed decision.

The Decision Formula

Does the specific benefit outweigh the actual exposure plus the mitigated failure risk?

Benefit > (Exposure + Mitigated Risk) → Proceed

This isn’t a mathematical formula—it’s a structured judgment call. The framework ensures you’re making that judgment with full information.

Decision Options

Proceed as planned: Low exposure, clear detection mechanisms, acceptable failure impact, strong benefit.

Proceed with safeguards: Medium exposure requires additional human review, monitoring, or constraints.

Pilot first: High exposure warrants testing with limited scope before broader rollout.

Decline: Exposure too high, failure modes unacceptable, or benefit insufficient to justify risk.

Most decisions land in the middle two categories. Pure “proceed” is rare for anything beyond trivial applications. “Decline” is appropriate when risk genuinely exceeds value.

Safeguards for “Proceed With Safeguards”

When you choose to proceed with safeguards, be specific about what those safeguards are:

Human review: What percentage of outputs get reviewed? By whom? Against what criteria?

Monitoring: What gets logged? Who reviews the logs? How often?

Constraints: What topics, data types, or actions are off-limits? How are these enforced?

Escalation: What triggers escalation to a human? What’s the response time requirement?

“Proceed with safeguards” without specific safeguards isn’t a real decision—it’s a deferral of responsibility.

Documentation Requirements

For significant applications, document: - The benefit statement (quantified) - The exposure map (data and systems) - The failure modes (with detection and recovery plans) - The decision and rationale - The monitoring and reassessment plan

This documentation isn’t bureaucracy—it’s protection. When something goes wrong (and eventually something will), documented reasoning shows thoughtful decision-making rather than negligence.

The Reassessment Cycle

Risk assessment isn’t one-and-done. Schedule reassessment for significant applications:

Quarterly review: Are the benefits materializing as expected? Have any failure modes manifested? Has the exposure changed?

Trigger-based review: Reassess when circumstances change—new data sources, expanded use, new users, or after any incident.

Annual comprehensive review: Full re-run of the Permission Framework to ensure the original assessment still holds.

Risk profiles change. An application that was low-risk when you started might become higher-risk as you expand its scope or as regulations evolve. The assessment process isn’t complete until you’ve built in reassessment.

Common Objections

“This slows us down too much.”

A thorough assessment takes 30-60 minutes. An incident response takes weeks or months. The assessment is the faster path, not the slower one.

For low-risk applications, the assessment is quick: clear benefit, minimal exposure, simple failure modes. You’re not spending hours on meeting summary tools. You’re investing appropriate time on customer communication systems.

“Our AI vendor already did risk assessment.”

They assessed their product, not your use case. A vendor assessment covers whether their AI is generally safe and effective. It doesn’t cover your specific data, your organizational context, or your acceptable risk tolerance.

“We don’t have time to document everything.”

Focus documentation on higher-exposure applications. A meeting summary tool needs a sentence. A customer communication system needs a page. Scale documentation to risk.

“How do I know if my assessment is right?”

Start conservative. You can always expand permissions after trust is established. It’s much harder to claw back access after an incident.

“This feels like we’re being overly cautious.”

For low-risk applications, you’ll breeze through the framework in minutes and proceed confidently. The framework only feels burdensome when you’re trying to rush through high-risk applications without adequate thought.

Your Monday Morning Action Item

Pick one AI application you’re using or considering. Run through the Permission Framework:

1. Specific benefit: What’s the quantified improvement? (Be concrete)

2. Actual exposure: What data is accessed? Who’s affected if it fails? (Be honest)

3. Failure modes: What can go wrong? How would you detect it? What’s the recovery plan? (Be thorough)

Make a decision: proceed, proceed with safeguards, pilot, or decline.

Write one paragraph documenting your reasoning.

If you can’t complete this exercise in thirty minutes, that’s diagnostic—the application is either too complex for quick assessment or you don’t yet understand it well enough to proceed.

Chapter 12 applies this framework specifically to data access decisions—the detailed question of what information your AI workflows should see. Chapter 13 addresses the career implications of AI decisions—how to protect yourself while moving your organization forward.