Know Your Risks: Agentic AI Risk Classifications

Discover security and privacy considerations for working with agentic AI

Know Your Agentic AI Risk

Stanford is evolving its AI risk framework to meet the challenges of agentic AI. While data sensitivity remains vital, our new risk tiers now account for an AI’s ability to take actions, influence decisions, and operate autonomously.

These tiers are aligned with Stanford’s Data Risk Classification, and account for an AI system’s actions or tool access, influence on decisions, and level of external exposure and autonomy.

All AI must be classified prior to deployment and updated as capabilities change.

Stanford AI Risk Level	Description	Examples
Low	AI Systems intended to serve internal informational purposes only, handling no sensitive data, initiating no actions or tool calls, playing no role in regulated decisions, and having no external-facing exposure; easily reversible. Primarily enhances efficiency or user experience.	AI-powered search, spam filters, and productivity tools Basic search optimization Literature search assistants
Moderate	AI Systems that access internal business data to conduct structured analysis, inform internal decision-making, and invoke tools in a limited capacity within defined workflows.	Customer service chatbots AI-generated text or imagery Deepfake detection systems Course recommendation systems Classroom scheduling algorithms Budget forecasting models Grant writing assistants
High	AI Systems that handle sensitive data, access confidential or regulated information, execute tool calls affecting internal systems, financial, legal, HR, student records, or operational decisions, engage directly with clients, and operate with a degree of autonomy across enterprise systems of record without human oversight pose the risk of irreversible actions.	Student admissions screening Emotion recognition tools Financial aid and support decision making Medical diagnosis aids Critical infrastructure management Predictive early warning systems identifying students at risk of failure or dropout Employment and human resources-related functions Research where human oversight is essential Research participant screening AI for vulnerable population identification Customer onboarding agent AI executing contract modifications at scale

AI Agent Deployment Categories

Agent Category	Description	Examples
Personal agent	AI assistants deployed for the productivity or decision-support of an individual Stanford community member. The agent acts under that individual's credentials and outputs are received primarily by that individual. Primary risk: Data exfiltration. Every community member is a potential vector for sensitive data leaving institutional boundaries through prompts, attachments, or context.	Microsoft 365 Copilot ChatGPT Claude Cursor GitHub Copilot Personal research-literature assistants Individual writing aids
Business workflow agent/internal	AI systems deployed to automate or augment internal Stanford operational, administrative, or research-support workflows. The agent operates under departmental or service credentials, frequently with elevated privileges, and acts on Stanford data and systems. Primary risk: Privileged action at scale. Misconfiguration, prompt injection, or model error can cause regulatory exposure (FERPA, HIPAA), financial harm, biased outcomes in HR or admissions, or unauthorized changes to systems of record.	Grant-processing assistants Budget forecasting models IT ticket triage HR resume-screening tools Library cataloging agents Facilities scheduling Course-recommendation engines for advisors Internal knowledge-base agents
External-facing agent	AI systems that interact directly with individuals outside of Stanford, including students, applicants, research participants, patients, donors, or members of the public. The agent's outputs are received by these external parties. Primary risk: Reputational, legal, and regulatory exposure. Hallucinations become a liability. Bias becomes a discrimination claim. Prompt injection becomes a public incident. EU AI Act Art. 50 disclosure obligations apply.	Admissions chatbots Patient-triage assistants in Stanford Health Care Public-website conversational agents Donor-engagement assistants Prospective-student Q&A bots AI-mediated research-participant intake

Agent Category

Description

Examples

Personal agent

AI assistants deployed for the productivity or decision-support of an individual Stanford community member. The agent acts under that individual's credentials and outputs are received primarily by that individual.

Primary risk: Data exfiltration. Every community member is a potential vector for sensitive data leaving institutional boundaries through prompts, attachments, or context.

Microsoft 365 Copilot
ChatGPT
Claude
Cursor
GitHub Copilot
Personal research-literature assistants
Individual writing aids

Business workflow agent/internal

AI systems deployed to automate or augment internal Stanford operational, administrative, or research-support workflows. The agent operates under departmental or service credentials, frequently with elevated privileges, and acts on Stanford data and systems.

Primary risk: Privileged action at scale. Misconfiguration, prompt injection, or model error can cause regulatory exposure (FERPA, HIPAA), financial harm, biased outcomes in HR or admissions, or unauthorized changes to systems of record.

Grant-processing assistants
Budget forecasting models
IT ticket triage
HR resume-screening tools
Library cataloging agents
Facilities scheduling
Course-recommendation engines for advisors
Internal knowledge-base agents

External-facing agent

AI systems that interact directly with individuals outside of Stanford, including students, applicants, research participants, patients, donors, or members of the public. The agent's outputs are received by these external parties.

Primary risk: Reputational, legal, and regulatory exposure. Hallucinations become a liability. Bias becomes a discrimination claim. Prompt injection becomes a public incident. EU AI Act Art. 50 disclosure obligations apply.

Admissions chatbots
Patient-triage assistants in Stanford Health Care
Public-website conversational agents
Donor-engagement assistants
Prospective-student Q&A bots
AI-mediated research-participant intake

Explore Best Practices

The information provided below is not considered complete or exhaustive.

Data privacy & usage

Avoid inputting data about others that you wouldn't want them to input about you.
Avoid using agentic AI with Moderate/High Risk Data on third-party tools not covered by Stanford-approved agreements.
If inputting Low Risk Data, consider whether you're comfortable with it becoming public.
Opt out of data sharing for training/iterative learning when possible.
If an agent interacts with users, obtain informed consent and provide an opt-out/delete path.

Autonomy & approvals

Choose the lowest autonomy that meets the need.
Require human approval for: external communications, permission changes, publishing, spending, deletions, deployments, and system-of-record updates.
Use rate limits, batching, and rollback plans.

Identity, access, and secrets

Use least privilege and scoped connectors.
Prefer short-lived tokens; store secrets in approved secret managers, not prompts or config files.
Separate agent identities from personal/admin identities where feasible.
Monitor and rotate credentials.

Secure tool use and retrieval

Restrict tools to an allow-list; disable unnecessary plugins/connectors.
Treat retrieved documents/web content as untrusted input; defend against prompt injection.
Use retrieval filters (folder scoping, label-based access, record-level permissions).
Keep audit logs of tool calls and actions.

Code and technical workflows

Never allow an agent to deploy to production without review.
Run agent-generated code through standard security checks (SAST, dependency scanning, secret scanning).
Watch for license/IP issues and provenance concerns.
Use sandbox environments for execution.

Transparency & recordkeeping

Document agent configuration, permissions, data sources, and review steps.
Keep changelogs for actions taken (what changed, when, by which agent identity).
Follow discipline- and publisher-specific guidance on AI use and attribution.

Recommended operating model (for teams)

Classify the use case by data sensitivity and action criticality.
Start with a pilot using Low Risk data, limited scope, and human approvals.
Define: owner, reviewer, escalation path, incident response steps.
Establish acceptance tests (e.g., “no external emails without approval,” “no permission changes,” “no writing outside folder X”).
Review periodically as tools/models change.

Guardrails by Phases

Phases	Guardrails
Data Privacy & Usage	Avoid inputting data into generative AI about others that you wouldn’t want them to input about you.
Data Privacy & Usage	Avoid inputting any sensitive data, such as Moderate or High Risk Data, whether using a personal or Stanford account with a third-party AI platform or tool that is not covered by a Stanford Business Associates Agreement. Review Stanford approved services by data risk classification.
Preparation	Inventory agents and map data flows
Identity	Assign unique, verifiable NHIs
Behavior	Log all actions with attribution and baselines. Enable explainability for decisions.
Data Governance	Prevent injections; track lineage.
Segmentation	Apply micro-segmentation. Isolate in a non-production sandbox first.
Incident Response	Implement kill switches. Enable session revocation. Develop Incident Response plan using Department Incident Response Tool Kit.