NIST’s Unpublished AI Risk Study Remains Shelved Amid Administrative Change

In Brief

A NIST-led red-teaming exercise at CAMLIS, evaluated vulnerabilities in advanced AI systems, assessing risks like misinformation, data leaks, and emotional manipulation.

The National Institute of Standards and Technology (NIST) completed a report on the safety of the advanced AI models near the end of the Joe Biden administration, but the document was not published following the transition to the Donald Trump administration. Although the report was designed to assist organizations in evaluating their AI systems, it was among several NIST-authored AI documents withheld from release due to potential conflicts with the policy direction of the new administration.

Prior to taking office, President Donald Trump indicated his intent to revoke Biden-era executive orders related to AI. Since the transition, the administration has redirected expert focus away from areas such as algorithmic bias and fairness in AI. The AI Action Plan released in July specifically calls for revisions to NIST’s AI Risk Management Framework, recommending the removal of references to misinformation, Diversity, Equity, and Inclusion (DEI), and climate change.

At the same time, the AI Action Plan includes a proposal that resembles the objectives of the unpublished report. It directs multiple federal agencies, including NIST, to organize a coordinated AI hackathon initiative aimed at testing AI systems for transparency, functionality, user control, and potential security vulnerabilities.

NIST-Led Red Teaming Exercise Probes AI System Risks Using ARIA Framework At CAMLIS Conference

The red-teaming exercise was conducted under the Assessing Risks and Impacts of AI (ARIA) program by the NIST, in partnership with Humane Intelligence, a company that focuses on evaluating AI systems. This initiative was held during the Conference on Applied Machine Learning in Information Security (CAMLIS), where participants explored the vulnerabilities of a range of advanced AI technologies.

The CAMLIS Red Teaming report documents the assessment of various AI tools, including Meta’s Llama, an open-source large language model (LLM); Anote, a platform for developing and refining AI models; a security system from Robust Intelligence, which has since been acquired by CISCO; and Synthesia’s AI avatar generation platform. Representatives from each organization contributed to the red-teaming activities.

Participants utilized the NIST AI 600-1 framework to analyze the tools in question. This framework outlines multiple risk areas, such as the potential for AI to produce false information or cybersecurity threats, disclose private or sensitive data, or foster emotional dependency between users and AI systems.

Unreleased AI Red Teaming Report Reveals Model Vulnerabilities, Sparks Concerns Over Political Suppression And Missed Research Insights

The research team found several methods to circumvent the intended safeguards of the tools under evaluation, leading to outputs that included misinformation, exposure of private information, and assistance in forming cyberattack strategies. According to the report, some aspects of the NIST framework proved more applicable than others. It also noted that certain risk categories lacked the clarity necessary for practical use.

Individuals familiar with the red-teaming initiative expressed that the findings from the exercise could have offered valuable insights to the broader AI research and development community. One participant, Alice Qian Zhang, a doctoral candidate at Carnegie Mellon University, noted that publicly sharing the report might have helped clarify how the NIST risk framework functions when applied in real-world testing environments. She also highlighted that direct interaction with the developers of the tools during the assessment added value to the experience.

Another contributor, who chose to remain anonymous, indicated that the exercise uncovered specific prompting techniques—using languages such as Russian, Gujarati, Marathi, and Telugu—that were particularly successful in eliciting prohibited outputs from models like Llama, including instructions related to joining extremist groups. This individual suggested that the decision not to release the report may reflect a broader shift away from areas perceived as linked to diversity, equity, and inclusion ahead of the incoming administration.

Some participants speculated that the report’s omission may also stem from a heightened governmental focus on high-stakes risks—such as the potential use of AI systems in developing weapons of mass destruction—and a parallel effort to strengthen ties with major technology companies. One red team participant anonymously remarked that political considerations likely played a role in withholding the report and that the exercise contained insights of ongoing scientific relevance.