CITATION — REFERENCE ENTRY
The Foundation Model Transparency Index — arXiv
- Key
- bommasani-2023-fmti
- Authors
- Bommasani, Rishi; Klyman, Kevin; Longpre, Shayne; Kapoor, Sayash; Maslej, Nestor; Xiong, Betty; Zhang, Daniel; Liang, Percy
- Issued
- 2023-10-19
- Type
- article
- Container
- arXiv
- Publisher
- Stanford Center for Research on Foundation Models
Raw CSL JSON
{
"DOI": "10.48550/arXiv.2310.12941",
"URL": "https://arxiv.org/abs/2310.12941",
"type": "article",
"genre": "Preprint",
"title": "The Foundation Model Transparency Index",
"author": [
{
"given": "Rishi",
"family": "Bommasani"
},
{
"given": "Kevin",
"family": "Klyman"
},
{
"given": "Shayne",
"family": "Longpre"
},
{
"given": "Sayash",
"family": "Kapoor"
},
{
"given": "Nestor",
"family": "Maslej"
},
{
"given": "Betty",
"family": "Xiong"
},
{
"given": "Daniel",
"family": "Zhang"
},
{
"given": "Percy",
"family": "Liang"
}
],
"issued": {
"date-parts": [
[
2023,
10,
19
]
]
},
"number": "2310.12941",
"language": "en",
"publisher": "Stanford Center for Research on Foundation Models",
"container-title": "arXiv"
}
Claims
-
In the 2023 index, overall scores ranged from 54 (Meta) to 12 (Amazon) with a mean of 37; upstream transparency (data, labor, compute) scored worst, and no company scored on indicators for data creators, data copyright and license status, or copyright mitigations.
"Overall scores vary significantly given a range of 42 between the highest-scoring developer, Meta, at 54 and the lowest-scoring developer, Amazon, at 12. Relative to the mean overall score of 37, [...] Upstream resource transparency scores the worst (Figure 8). Breaking down the trends by domain, scores are consistently the worst for the upstream domain, particularly the Data, Data Labor, and Compute subdomains. [...] Within the upstream domain, no company scores points for indicators about data creators, the copyright and license status of data, and mitigations related to copyright. The industry-wide lack of transparency on these issues relates directly to pressing societal concerns related to copyright and intellectual property, which are the subject of ongoing litigation. [...] For data creators, in part we believe this reflects the nascent status of methods for providing web-scale understanding of who created the data (e.g. text, images) scraped from the Internet."
Available in