内容来源：https://research.google/blog/a-differentially-private-framework-for-gaining-insights-into-ai-chatbot-use/

内容总结：

谷歌研究团队推出新型隐私保护框架，助力AI聊天机器人服务优化

2025年12月10日，谷歌研究院科学家Alexander Knop与博士后研究员Daogao Liu共同发布了一项创新研究，提出一套具备严格数学隐私保障的框架，能够在保护用户对话隐私的前提下，为AI聊天机器人平台提供宏观使用洞察。

当前，大语言模型聊天机器人每日服务全球数亿用户，涵盖邮件撰写、代码编程、旅行规划乃至餐厅菜单设计等多种场景。了解这些宏观使用模式对平台改进服务、完善安全策略具有重要意义，也有助于公众认识AI如何塑造社会生活。然而，用户对话中可能包含敏感或个人隐私信息，如何在不侵犯隐私的前提下获取有效洞察，成为行业关键挑战。

现有方案多依赖大语言模型在总结对话时主动剔除个人信息，这类启发式隐私保护方法难以形式化验证，且随模型迭代可能失效。为此，研究团队在COLM 2025会议上发表的论文《Urania：基于差分隐私的AI使用洞察框架》中，提出了一套端到端差分隐私保障的新框架。

该框架通过三个核心步骤实现隐私保护与数据效用的平衡：

1. 差分隐私聚类：将对话转化为数值向量后，使用差分隐私聚类算法进行分组，确保单个对话无法过度影响聚类中心；
2. 差分隐私关键词提取：采用三种方法（大语言模型引导生成、差分隐私版TF-IDF权重计算、基于公开词表的大语言模型筛选）从对话中提取关键词，并通过添加噪声的差分隐私直方图统计，仅保留多用户共现的高频关键词，过滤敏感或独特词汇；
3. 基于关键词的大语言模型摘要生成：大语言模型仅依据脱敏后的关键词为每个聚类生成宏观摘要，全程不接触原始对话内容，依托差分隐私的后处理性质确保端到端隐私安全。

实验显示，该框架在强隐私设置下虽会降低摘要的细分程度，但其生成的摘要因聚焦于通用高频关键词，反而在70%的对比评测中被大语言模型评估者认为比非隐私基线方案更简洁、聚焦。在隐私攻击测试中，针对该框架的成员推断攻击成功率接近随机猜测（AUC=0.53），显著低于非隐私管道（AUC=0.58），证实其具备更强的隐私泄漏防护能力。

研究团队指出，这项工作为构建具备形式化隐私保障的大规模文本分析系统迈出了重要一步。未来研究方向包括适配实时流式对话场景、优化隐私-效用权衡机制，以及拓展至多模态对话数据分析。随着人工智能日益融入日常生活，开发兼顾洞察价值与用户隐私的技术方案，已成为构建可信赖AI系统的关键基石。

中文翻译：

一种用于洞察AI聊天机器人使用的差分隐私框架
2025年12月10日
Alexander Knop（研究科学家）与Daogao Liu（博士后研究员），谷歌研究院

我们推出了一种新颖框架，通过差分隐私聚类、差分隐私关键词提取及大语言模型摘要生成流程，实现对AI聊天机器人使用情况的高层洞察。该方法提供严格的端到端差分隐私保障，在保护用户对话隐私的同时，为平台优化提供有效参考。

快速链接
大语言模型聊天机器人每日为数亿用户提供服务，涵盖邮件起草、代码编写、旅行规划乃至咖啡馆菜单设计等各类任务。对平台提供商而言，理解这些高层使用场景对改进服务或落实安全政策具有重要价值，同时也为公众洞察AI如何塑造世界提供了窗口。

但这引发了一个关键问题：当对话内容可能涉及个人隐私或敏感信息时，我们如何安全地获取有价值的洞察？

现有方法（如CLIO框架）尝试通过大语言模型总结对话内容，并提示其过滤个人身份信息。这虽是良好的初步尝试，但依赖于启发式隐私保护机制，其隐私保障难以形式化界定，且可能因模型迭代而失效，导致系统难以维护与审计。这一局限促使我们思考：能否在获得形式化端到端隐私保障的前提下实现相近的实用价值？

在COLM 2025会议上发表的论文《乌拉尼亚：AI使用的差分隐私洞察》中，我们提出了一种新框架，能够从大语言模型聊天机器人交互中生成洞察，并具备严格的差分隐私保障。该框架采用差分隐私聚类算法与关键词提取方法，确保单次对话不会过度影响输出结果（即生成的摘要不会泄露任何个体对话信息）。下文将阐释该算法原理，并论证本框架相比现有方案能提供更优的隐私保障。

隐私保护洞察挖掘框架
差分隐私通过隐私预算参数ε量化单个用户数据对模型最终输出的最大影响程度。本框架基于差分隐私的两大核心特性构建：

• 后处理性：若算法B满足ε-差分隐私，算法A为任意非差分隐私算法，则对B的输出运行A仍能保持ε-差分隐私级别。
• 组合性：若算法A与B均为独立的ε-差分隐私算法，对数据集及B的输出运行A，整个过程仍满足2ε-差分隐私级别。

该差分隐私流程通过以下阶段实现端到端用户数据保护：

• 差分隐私聚类：首先将对话转化为数值化表征（嵌入向量），随后通过差分隐私聚类算法对相近的表征进行分组，确保单次对话不会过度影响聚类中心。
• 差分隐私关键词提取：从每次对话中提取关键词。针对每个聚类，我们采用差分隐私直方图机制统计关键词出现频次，并通过添加噪声掩蔽个体对话的影响，确保仅选择多用户共有的关键词，避免暴露独特或敏感词汇。我们探索了三种对话关键词生成方法：
  1. 大语言模型引导选择：向大语言模型提供对话内容，要求生成五个最相关的关键词；
  2. 差分隐私版TF-IDF：提取对话中所有词汇，根据其在文本中的出现频次及语料库中的逆文档频率进行加权；
  3. 基于公开数据关键词列表的大语言模型引导选择：预先建立潜在关键词列表，要求大语言模型从中选择五个最相关的关键词。
• 基于关键词的大语言模型摘要生成：最终，大语言模型仅依据隐私筛选后的关键词为每个聚类生成高层摘要。模型不会接触原始对话内容，仅处理匿名化关键词，其后处理特性保障了整个框架的端到端隐私性。

通过将差分隐私置于核心地位，本框架的隐私保障基于数学原理而非启发式规则。其不依赖大语言模型完美删减隐私数据的能力：即使关键词包含个人身份信息或其他敏感数据，生成的摘要也不会泄露这些信息。更实际地说，这种保障机制使大语言模型无法泄露敏感数据（例如抵御提示注入攻击）。

框架测试验证
为评估框架的实用性（摘要质量）与隐私性（保护强度），我们将其与受CLIO启发的非隐私基线模型Simple-CLIO进行对比。基线模型采用两步流程：

1. 将对话转化为嵌入向量并进行非隐私聚类；
2. 从每个聚类抽取对话样本输入大语言模型生成摘要。

隐私与效用的权衡
如预期所示，我们观察到权衡关系：更强的隐私设置（更低的隐私参数ε值）会导致摘要粒度下降。例如，随着隐私预算收紧，差分隐私聚类算法生成的聚类数量减少且精度降低，导致主题覆盖率下降。

但结果也包含意外发现：在直接对比中，大语言模型评估者往往更青睐本框架生成的隐私保护摘要。某次评估显示，差分隐私生成的摘要获得偏好的比例高达70%。这表明差分隐私流程的约束条件——强制摘要基于通用高频关键词生成——可能产生比无约束非隐私方法更简洁、更聚焦的输出结果。

实证隐私评估
为测试框架的鲁棒性，我们实施了成员推断式攻击，尝试判断特定敏感对话是否包含在数据集中。结果明确显示：针对差分隐私流程的攻击成功率接近随机猜测，曲线下面积得分为0.53；而非隐私流程的AUC得分达0.58，表明信息泄露风险更高。该实验为本隐私框架显著增强防泄露能力提供了实证依据。

未来展望
我们的研究为构建具备形式化隐私保障的大规模文本分析系统迈出了第一步，证明了在获取有效洞察与严格用户隐私保护之间实现平衡的可能性。

展望未来，我们确定了多个值得探索的研究方向：包括使框架适配持续新增对话的在线场景、探索改进隐私效用权衡的替代差分隐私机制，以及扩展对多模态对话（涉及图像、视频、音频）的支持能力。

随着AI日益融入日常生活，开发隐私保护方法以理解其使用情况不仅是技术挑战，更是构建可信赖、负责任AI的基础要求。

致谢
感谢所有项目参与者的重要贡献。特别感谢同事们的支持：Yaniv Carmel、Edith Cohen、Rudrajit Das、Chris Dibak、Vadym Doroshenko、Alessandro Epasto、Prem Eruvbetine、Dem Gerolemou、Badih Ghazi、Miguel Guevara、Steve He、Peter Kairouz、Pritish Kamath、Nir Kerem、Ravi Kumar、Ethan Leeman、Pasin Manurangsi、Shlomi Pasternak、Mikhail Pravilov、Adam Sealfon、Yurii Sushko、Da Yu、Chiyuan Zhang。

英文来源：

A differentially private framework for gaining insights into AI chatbot use
December 10, 2025
Alexander Knop, Research Scientist, and Daogao Liu, Post Doc Researcher, Google Research
Introducing a novel framework that generates high-level insights into AI chatbot usage through a pipeline of DP clustering, DP keyword extraction, and LLM summarization. This approach provides rigorous, end-to-end DP guarantees, ensuring user conversation privacy while offering utility for platform improvement.
Quick links
Large language model (LLM) chatbots are used by hundreds of millions of people daily for tasks ranging from drafting emails and writing code to planning vacations and creating menus for cafes. Understanding these high-level use cases is incredibly valuable for platform providers looking to improve services or enforce safety policies. It also offers the public insights into how AI is shaping our world.
But this raises a critical question: How can we gain valuable insights when the conversations themselves might contain private or sensitive information?
Existing approaches, like the CLIO framework, attempt to solve this by using an LLM to summarize conversations while prompting it to strip out personally identifiable information (PII). While a good first step, this method relies on heuristic privacy protections. The resulting privacy guarantee is difficult to formalize and may not hold up as models evolve, making these systems difficult to maintain and audit. This limitation led us to ask if it is possible to achieve similar utility with formal, end-to-end privacy guarantees.
In our paper, "Urania: Differentially Private Insights into AI Use," presented at COLM 2025, we introduce a new framework that generates insights from LLM chatbot interactions with rigorous differential privacy (DP) guarantees. This framework uses a DP clustering algorithm and keyword extraction method to ensure that no single conversation overly influences the result (i.e., the output summaries do not reveal information about any single individual's conversation). Here we explain the algorithm and demonstrate that this framework is indeed providing better privacy guarantees than prior solutions.
Privacy-preserving framework for insights mining
DP uses a privacy budget parameter, ε, to measure the maximum allowed influence of any single user's contributions to the final output of a model. Our framework is designed to rely on two key properties of DP:

• Post-processing: If B is an ε-DP algorithm and A is any non-DP algorithm, then running A on the output of B keeps things private at a ε-DP level.
• Composition: If A and B are two separate ε-DP algorithms, running A on the dataset and the output of B still keeps the entire process private at a 2ε-DP level.
  This differentially private pipeline is designed to ensure end-to-end user data protection through the following stages:
• DP clustering: Conversations are first converted into numerical representations (embeddings). The framework then groups representations that are close to each other using a DP clustering algorithm. This ensures that no single conversation overly influences the cluster centers.
• DP keyword extraction: Keywords are extracted from each conversation. For every cluster, our approach computes a histogram of keywords, i.e., it counts the number of times each keyword appears in the cluster, using a DP histogram mechanism (e.g., [1, 2]). We add noise to the histogram in order to mask the influence of individual conversations, ensuring that only keywords common to multiple users are selected, preventing unique or sensitive terms from being exposed. We explore three methods for creation of keywords for each conversation:
• LLM guided selection: We provide an LLM the conversation and ask it to create the top five most relevant keywords;
• DP version of TF-IDF: We get all the words in the conversation and weight them proportionally to the number of times they appear in the text and inversely proportional to the number of times they appear in the corpus; and
• LLM guided approach where there is an initial list of keywords obtained from public data: Instead of asking the LLM to generate keywords for each conversation independently, we create a list of potential keywords and ask LLM to choose the top 5 most relevant keywords from the list.
• LLM summarization from keywords: Finally, an LLM generates a high-level summary for each cluster using only the privately selected keywords. The LLM never sees the original conversations in the cluster, only the anonymized keywords. This post-processing property ensures the end-to-end privacy of the entire framework.
  By integrating DP at its core, this framework’s privacy guarantees are mathematical, not heuristic. They don't depend on an LLM's ability to perfectly redact private data: in other words, even if keywords contain PII or some other sensitive data, the generated summaries are not going to contain this data. In more practical terms, this guarantee makes it impossible for the LLM to reveal sensitive data (e.g., due to prompt injection attacks).
  Putting this framework to the test
  To evaluate our framework’s utility (summary quality) and privacy (protection strength), we compared its performance against Simple-CLIO, a non-private baseline we created inspired by CLIO. The baseline follows a two-step process:
• Conversations are converted into embeddings and clustered non-privately.
• For each cluster, a sample of conversations is fed to an LLM to generate a summary of those samples.
  The privacy-utility trade-off
  As expected, we observed a trade-off: stronger privacy settings (lower values of the privacy parameter ϵ) led to a decrease in the granularity of the summaries. For instance, topic coverage dropped as the privacy budget tightened, because the DP clustering algorithm produced fewer and less precise clusters.
  However, the results also held a surprise. In head-to-head comparisons, LLM evaluators often preferred the private summaries generated by our framework. In one evaluation, the DP-generated summaries were favored up to 70% of the time. This suggests that the constraints imposed by this DP pipeline — forcing summaries to be based on general, frequent keywords — can lead to outputs that are more concise and focused than those from an unconstrained, non-private approach.
  Empirical privacy evaluation
  To test the framework’s robustness, we ran a membership inference-style attack designed to identify whether a specific sensitive conversation was included in the dataset. The results were clear: the attack on the DP pipeline performed about as well as random guessing, achieving an area under the curve (AUC) score of 0.53 (i.e., the integral of the ROC curve). In contrast, the attack was more successful against the non-private pipeline, which had a higher AUC of 0.58, indicating greater information leakage. This experiment provides empirical evidence that our privacy framework offers significantly stronger protection against privacy leakage.
  Looking ahead
  Our work is a first step toward building systems that can analyze large-scale text corpora with formal privacy guarantees. We've shown that it's possible to balance the need for meaningful insights with stringent user privacy.
  Looking forward, we see several exciting avenues for future research. These include adapting the framework for online settings where new conversations are constantly added, exploring alternative DP mechanisms to further improve the utility-privacy trade-off, and adding support for multi-modal conversations (i.e., conversations involving images, videos, and audio).
  As AI becomes more integrated into our daily lives, developing privacy-preserving methods for understanding its use is not just a technical challenge — it's a fundamental requirement for building trustworthy and responsible AI.
  Acknowledgments
  Thanks to all project contributors, whose essential efforts were pivotal to its success. Special thanks to our colleagues: Yaniv Carmel, Edith Cohen, Rudrajit Das, Chris Dibak, Vadym Doroshenko, Alessandro Epasto, Prem Eruvbetine, Dem Gerolemou, Badih Ghazi, Miguel Guevara, Steve He, Peter Kairouz, Pritish Kamath, Nir Kerem, Ravi Kumar, Ethan Leeman, Pasin Manurangsi, Shlomi Pasternak, Mikhail Pravilov, Adam Sealfon, Yurii Sushko, Da Yu, Chiyuan Zhang.