Reducing Risks Posed by Synthetic Content: An Overview of Technical Approaches to Digital Content Transparency — National Institute of Standards and Technology

Claims

1. covert-watermarks
   Covert watermarks are designed to persist where metadata is often stripped, and watermarks carrying data could leak sensitive information about a tool's user.

   "Digital watermarks that are not zero-bit—i.e., that have some capacity to carry additional data—could leak sensitive information, especially if a tool that applies the watermark embeds or reveals information about the tool’s user without that user’s knowledge. Covert watermarks raise particularly salient concerns: unlike metadata, which is often stripped when content is disseminated, covert watermarks are designed to be persistent, and unlike overt watermarks, their presence is not necessarily apparent to users."

   Locator: page: 15 · Quote language: en
2. definition
   NIST defines digital content transparency as documenting and accessing information about the origins and history of digital content.

   "Digital content transparency refers to the process of documenting and accessing information about the origins and history of digital content."

   Locator: page: 1 · Quote language: en
3. false-positives
   Synthetic content detectors typically output a probability-like score, and false positives (human content judged AI-generated) can be extremely damaging.

   "Typically, the detector will give a real-valued score for each input—which can generally be interpreted as a probability—indicating how likely the input is synthetic. [...] In many contexts, false positives—assessing human content as AI-generated—can be extremely damaging, potentially resulting in major reputational harms or adverse treatment."

   Locator: page: 33 · Quote language: en
4. malicious-actors
   In the context of AI-generated abuse imagery, malicious actors often use freely available models with safeguards removed, and could apply AI-indicating metadata or watermarks to real abuse imagery to make it less likely to be investigated.

   "Malicious actors generating this content on the Internet often use freely available models or build their own smaller models based on existing open-source code, from which they can easily remove safeguards. [...] In addition, malicious actors could attempt to make real CSAM less likely to be investigated, or make its victims less likely to be identified, by applying metadata or watermarks that suggest the content is AI-generated."

   Locator: page: 39 · Quote language: en
5. metadata
   Provenance metadata can indicate synthetic origins or assert authenticity, and is typically embedded in the file but can also be stored in an external repository linked by an identifier.

   "Metadata, especially cryptographically signed metadata (see Section 3.1.2.2), can contribute to content transparency by explicitly describing the origins of the content, either by indicating synthetic origins or by asserting authenticity. Provenance metadata is typically packaged with the data it describes such that they travel together (“embedded metadata”). [...] Alternatively, metadata can be stored in an external repository and linked to the content via some form of identifier."

   Locator: page: 16 · Quote language: en
6. metadata-stripping
   Embedded metadata is often stripped when files are shared, and metadata generally cannot travel with raw text copied between documents or applications.

   "Metadata recorded within a file can similarly be stripped altogether, as it often is when files are shared (e.g., via social media platforms). [...] It is not generally possible to have metadata travel with raw text as the text is copied across documents or applications."

   Locator: page: 16 · Quote language: en
7. synthetic-content
   NIST uses Executive Order 14110's definition of synthetic content.

   "In this document, following Executive Order 14110, “synthetic content” refers to “information, such as images, videos, audio clips, and text, that has been significantly altered or generated by algorithms, including by AI.”"

   Locator: page: 1 · Quote language: en
8. trust-caveat
   Digital content transparency may contribute to trustworthiness but does not guarantee it, and can create a false sense of trust, e.g. when legitimate content is taken out of context.

   "Digital content transparency provides a vehicle for individuals and organizations to access more information about the origins and history of content, which may contribute to trustworthiness but does not guarantee it, and in some cases may undermine it. While transparency can help identify when content is being misrepresented, it can also create a false sense of trust, such as when a piece of content appears legitimate based on technical measures but is then manipulated through non-technical means (e.g., taking a legitimate piece of content out of context)."

   Locator: page: 1 · Quote language: en