ServiceNow researchers have published MosaicLeaks, a benchmark for a privacy problem that becomes more important as companies give agents both private documents and external tools. The core risk is not that the agent pastes a secret into one obvious outbound message. It is that ordinary-looking web queries can accumulate into a private fact.
The Hugging Face write-up frames the failure mode around deep research agents that combine local enterprise documents with web retrieval. An outside observer does not see the private documents or the agent’s hidden reasoning. They see the external queries. MosaicLeaks asks whether that query log is enough to infer private research intent, answer private questions, or reconstruct verifiable private claims.
That is a sharper privacy model than “tell the agent not to leak.” It treats the agent’s tool use as the attack surface.
The benchmark is built around mixed context
MosaicLeaks contains 1,001 multi-hop research chains. Each chain interleaves local enterprise documents with a controlled public web corpus. The point is to create tasks where an agent needs private context to decide what public information to retrieve next.
That is exactly the pattern enterprises want from research agents. A useful agent might read an internal account note, search public market filings, compare a vendor’s public claims with private support tickets, and produce a recommendation. The same pattern can leak sensitive context if the external queries reveal too much about the internal side of the task.
The authors measure three leakage types. Intent leakage asks whether the query log reveals what private question the agent was trying to answer. Answer leakage asks whether an observer with a private question can answer it from the query log. Full-information leakage asks whether the observer can state true private claims without being given the question.
Those distinctions matter. Many privacy controls focus on direct disclosure. MosaicLeaks is about inference from fragments.
Prompting is not enough
The uncomfortable finding is that privacy prompting does not eliminate the problem. The paper says models frequently leak across families and sizes, and that training only for task performance can make leakage worse.
That is plausible because a task-only agent is rewarded for retrieving useful evidence. If the fastest route to an answer is a query that exposes a private bridge entity, a capability-focused optimization may choose the revealing route more often. Better research behavior can become worse privacy behavior unless the training objective accounts for both.
ServiceNow’s write-up proposes Privacy-Aware Deep Research, or PA-DR, as a reinforcement-learning approach that combines task success with leakage penalties. In the headline result, the authors report strict chain success rising from 48.7% to 58.7% while answer/full-information leakage falls from 34.0% to 9.9%.
That result should be treated as a benchmark result, not proof that a production system is safe. It is still useful because it shows the right shape of mitigation: measure leakage at the tool-call level and reward the agent for solving the task without exposing private context.
Enterprise agent logs become sensitive data
MosaicLeaks also changes how teams should think about observability. Agent logs are often treated as debugging material: prompts, traces, tool calls, retrieval queries, actions, and outputs. For enterprise research agents, those logs may become sensitive data in their own right.
If an external search provider, proxy, plugin, or monitoring layer can observe the query trail, it may learn more than any single query reveals. The same concern applies internally. A broad logging system that stores every agent query may collect sensitive business context even if the agent never prints the private document in its final answer.
The practical implication is that agent governance needs data-minimization rules for tool calls, not only final responses. Teams should ask which tools receive private context, whether queries can be rewritten to remove sensitive bridge entities, how long traces are retained, and who can inspect them.
What to watch next
The next checkpoint is whether privacy-aware research-agent evaluation becomes standard in enterprise deployments. A vendor claiming “secure deep research” should be able to answer more than where documents are stored. It should explain what information leaves the trust boundary through search, retrieval, connectors, browsing, plugins, and telemetry.
MosaicLeaks does not say every research agent is unusable. It says the privacy budget is spent across many small actions. That is the right warning for products that promise to connect internal knowledge with the public web.
For readers tracking model capabilities and company coverage, see our AI model leaderboard and AI company tracker.
