Anthropic unveils new framework to block harmful content from AI models

Anthropic has showcased a new security framework, designed to reduce the risk of harmful content generated by its large language models (LLM), a move that could have far-reaching implications for enterprise tech companies.

Large language models undergo extensive safety training to prevent harmful outputs but remain vulnerable to jailbreaks – inputs designed to bypass safety guardrails and elicit harmful responses, Anthropic said in a statement.