The scheme will offer direct funding for independent research and access to the company’s models. Grantees building open-source evaluations will also be able to access technical support from Anthropic.

Researchers who develop evaluations will publish their work as open-source projects that any AI developer can use.

People are using AI systems to seek companionship and access mental health guidance, but the AI industry does not have clear standards for how AI models should behave in these conversations.

While there is not yet scientific consensus on the impacts of AI due to technological developments outpacing studies, health organisations have warned that it poses significant risks to mental health and wellbeing.

There have also been several high-profile media reports relating to the harm AI usage has caused young people in particular.

Anthropic currently publishes its own research on the types of conversations people have with Claude, and says it uses this to inform how it develops and evaluates safeguards and other protective measures. It says it wants to fund more independent research to get more input in the field from clinicians, psychologists and methodologists. 

The company has published details of what is seeking from wellbeing evaluations applying for the grant programme, with applications due by 21st September.