Before Sending Confidential Data to an LLM: Environment, Classification, and Tool-Call Limits

Whether confidential data may enter an LLM is not settled by the blanket rules “it is personal information, so never” or “training is off, so it is safe.” The asset to protect is disclosure to an unapproved party. Blocking model input is only one control.

📑Table of Contents

1. A classification × environment × sink table
2. Training-off and paste bans do not stop confidential data
3. How to confirm a managed LLM environment
4. Constrain tool-call sinks before execution
5. A go/no-go checklist
6. FAQ
7. Summary

Three units decide whether a task is allowed. In a technology workflow, run only the task that satisfies all three at once.

• Data classification (Public / Internal / Confidential)
• The LLM environment (Managed: the organization contracted it and can confirm retention, training use, and subprocessors; or Unmanaged: a personal account)
• Tool-call exposure (email, posting, arbitrary URL fetch, or any call that can carry data outside)

Approved return versus public-channel upload

A company-contracted API that summarizes a customer email and returns it only to the original owner still sends text into an LLM, but it does not hand the record to an unapproved sink.

An agent that puts a file path into a tool call and uploads it to a public channel can leak confidential data even if the file bytes never pass through the model.

---

Classify, confirm Managed, then close sinks

Classify before you paste. Confirm the path is Managed. Close egress tool calls to approved recipients only. If you cannot write those three lines, do not send the task.

---

A classification × environment × sink table

Apply the rows from the top. If any cell fails, do not run that task. The table is operational policy, not a substitute for regulated contracts such as a BAA.

Default by classification

Classification	LLM environment	Tool-call condition	Default on the floor
Public	Unmanaged allowed	Do not mix non-Public into ingest	Personal ChatGPT is fine. Stop if internal documents enter the same context
Internal	Managed required	Sinks stay inside the company	No personal accounts. Search only what every channel reader may see
Confidential	Managed required	Sinks limited to approved parties; human approval when needed; ingest only what those sinks may receive	Training-off is not enough. Inspect send_email and similar tools before execution

---

Table sources

Sources as of August 2026:

• OpenAI Help on data use
• OWASP AI Agent Security Cheat Sheet
• AgentFlow (arXiv 2608.22868v1)
• Claw in Plain Sight (arXiv 2608.20658)

---

Published rates behind a paste ban

Use published rates as a reason to distrust a paste ban, not as your own incident rate. Those figures are secondary citations captured at survey time.

• C2 Data Technology reports that 27% of employees have put confidential records into public AI tools and that two-thirds of workplace AI use runs through personal accounts (via Salesforce and Verizon DBIR / LayerX).
• Jiji Press (2026-08-25) cites a Microsoft survey putting Japanese knowledge-worker shadow AI at 78%.

---

Substitutes when a table row fails

If a row fails, the substitutes are masking, minimum fields, a local LLM with no egress, or a human-only path. Local inference is not a license; it is only useful when outbound sinks are actually closed.

---

Training-off and paste bans do not stop confidential data

Training-off on personal ChatGPT means that conversation is not used to improve models. It does not mean the organization can confirm storage, subprocessors, or tool-call policy.

The same model family is not one environment

OpenAI Help states the training default by contract path.

• Consumer ChatGPT and Codex may train unless the user opts out
• ChatGPT Business, ChatGPT Enterprise, and the API do not train on inputs or outputs by default

The same model family is not one environment. A personal Plus account with training off remains a consumer path.

---

Why paste bans miss most use

Paste bans fail because confidential data has more than one door. C2 Data splits the paths into the following categories.

• pasted prompts
• RAG / agent tool context
• training or fine-tuning

A written ban does not see the two-thirds of use that sits on personal accounts. The same article cites DataGrail 2026: 63.6% of 2,400 AI vendors did not disclose third-party AI subprocessors in legal documents.

If subprocessors are undisclosed, do not put Confidential on that path even when the tool is “approved.”

---

Japanese reporting on ban-only policy

A ban-only policy is also incomplete in Japanese reporting. Jiji recounts the following.

• Samsung-style source paste
• 78% unsanctioned workplace use in Japan
• IBM Cost of a Data Breach 2025 attributing 20% of breaches to shadow AI

BBSec SQAT (updated 2026-08-26) states that correct access control still fails if a user pastes secrets into an external generative AI service. That is a second egress after the ACL.

Keep the ban. Put a sanctioned Managed path first. Tight bans without an approved path push work into unmeasured personal accounts.

---

How to confirm a managed LLM environment

Managed means the company contracted the path and can confirm where inputs go. A model name or a company laptop is not enough. A technology-approved browser extension is still Unmanaged if the login is personal.

Check order

Check in this order.

1. Who holds the contract. Personal free / Plus is Unmanaged.
2. The default training policy. Business / Enterprise / API are opted out by default in OpenAI’s documentation. Personal opt-out is still a consumer path.
3. Retention, region, and subprocessors. If undisclosed, do not load Confidential.
4. Logs and audit. If prompts or tool arguments remain in plaintext, Confidential is out of scope.

---

Local LLMs and internal bots

Jiji also recommends sanctioned corporate APIs versus local LLMs when data must not leave. “We do not train on your data” does not delete the input. Redistributing a summary is a separate authorization. C2 Data’s CSA pattern is the following.

• redact PII before a prompt, tool call, or fine-tune set
• minimize fields that enter context

---

Internal knowledge bots use the same sink rule

The same rule applies to an internal knowledge bot. Managed use is conditional: the sink is every channel reader, so ingest only documents that channel may already see.

Do not retrieve files the asker cannot open. That matches the read-only plus human-approval gate for production incident triage. Read access without a closed sink is still exfiltration.

---

Constrain tool-call sinks before execution

Permission to read a field is not permission to send it to every destination. A prompt that says “do not leak secrets” is not an enforcement boundary.

Prompt instructions do not stop disclosure (Claw)

Claw in Plain Sight (arXiv 2608.20658) reports session-level disclosure of 20.8–75.0% across 120 synthetic sessions.

• Stronger privacy prompts reduce leaks (S0 66.7% to S3 26.7%) but do not stop them
• A Claude configuration still leaked in 8 of 18 S3 sessions
• Inspection belongs before tool execution
• Legitimate context access still allows unauthorized copies into tool arguments

The study observes synthetic profiles and generated arguments; it is not a production leak rate.

---

Multi-step flows need more than request-level auth (AgentFlow)

AgentFlow (Virginia Tech, 2026-08-24) separates these cases:

• “read a customer record”
• “send email”
• “email an SSN externally”

Each step can look locally allowed while the flow is exfiltration. On 949 AgentDojo injected cases, confirmed compromise moved from 33.0% to 0.0% and utility from 46.7% to 63.3%.

Request-level authorization (who may do what) cannot express that multi-step constraint. AgentFlow’s scope is policy-visible tools and sinks.

---

Split the work by tool class

Split the work by tool class.

1. Ingest tools (file read, internal search) may see only what the later sink is allowed to receive.
2. Egress tools (email, post, arbitrary URL fetch) must keep destinations inside the classification’s approved parties. A fetch that puts secrets in the query string is egress.
3. Writes and execution need a human. OWASP lists the following as confirmation targets and warns not to authorize from model output. – send_email – execute_code – database_write – file_delete

---

Read versus write tokens on the same MCP

Qiita author c_u (2026-08-30) treats the same MCP surface as L1 when read-only and L3 when writes are allowed, and keeps ALLOW_WRITE_OPERATIONS false by default. Split read tokens from write tokens.

4. Assume indirect prompt injection. Do not assemble private data, untrusted content, and external send in one agent. Jiji’s “email the customer list to an attacker” planted in a document is a pre-execution stop, not a prompt instruction.

---

Prompt-side wording is not the stop

Prompt-side authorization wording is covered in the following section. Instructions reduce leakage; they do not halt it.

• GPT-5.6 lean system prompts and permission boundaries
• Where to cut MCP consent-per-call is in OBO and authorization-server criteria
• For study order, see the 2026 AI security learning roadmap

---

A go/no-go checklist

Record one task per line. Any blank fails closed.

• ✓ Classification is written as Public / Internal / Confidential. If unsure, Confidential.
• ✓ The LLM path is organization-contracted Managed. Personal ChatGPT or a personal API key stops Internal and above.
• ✓ Training-off was checked. A personal path is still not Managed.
• ✓ Ingest covers only documents this sink may receive. The bot is not searching ranges the asker cannot read.

Egress, write flags, injection, and logs

• ✓ Egress tool-call destinations stay inside approved parties. send_email / posts / arbitrary URLs require human confirmation before execution.
• ✓ Write flags default off. Read tokens are split from write tokens.
• ✓ If a document says “email the customer list,” pre-execution inspection still stops it.
• ✓ Audit logs do not keep secrets in plaintext. If they do, do not load Confidential.

Substitutes: mask, minimum fields, a local LLM with no egress, or humans only. The checklist implements policy. Final contract and regulatory calls stay with legal and security documents.

---

FAQ

Q1. If training is off, may I paste customer email into personal ChatGPT?

No. OpenAI documents different training defaults for consumer ChatGPT versus Business / Enterprise / API. Training-off is not the definition of Managed. Send Internal and Confidential to an organization-contracted path.

---

Q2. May an internal knowledge bot answer in a company-wide channel?

Only if the path is Managed and ingest is limited to documents that every channel reader may already see. Do not retrieve files the asker cannot open. The sink is the whole channel.

---

Q3. May an agent read internal files?

Read is not egress by itself. Later email, posts, or fetches are. Claw shows fields the agent may see can still be copied into tool arguments. Cut the read set to match the sink, and inspect arguments before execution.

---

Q4. Is a blanket ban on generative AI enough?

No. Jiji, C2 Data, and SQAT treat shadow AI (78% in Japan; two-thirds of use on personal accounts) as a path a ban does not see. Provision a sanctioned Managed path first.

---

Q5. Is a prompt that says “do not send secrets outside” enough?

It reduces disclosure; it does not stop it. Claw still sees leaks under strong instructions. OWASP says not to rely on model output for authorization. The boundary is policy before tool execution.

---

Related articles:

• 12332
• 12301

Summary

Limits on sending confidential data to an LLM are the product of classification, a managed environment, and tool-call sinks. If any factor is missing, do not run that task.

Training-off and paste bans are supporting controls. Official contract splits, shadow-AI rates, and tool-argument studies are why a single line cannot separate the three cases.

Next action: apply the checklist to one live workflow. If a cell is blank, change the path or stop.