AI’s First Victim: What Can We Learn From One Man’s Tragic Story?

Written by Finlay Mead

The unrelenting rise of artificial intelligence, the impending climate crisis, and the horrible crossroads where these two issues met: how can one man’s tragic story be transformed into a critical lesson for tech leaders, mental health professionals, and the world at large?

We’re not exactly lacking in stories about AI this year. With the seemingly unstoppable rise of ChatGPT, image-generator Midjourney causing all kinds of Catholic chaos, and the apparently imminent threat of across-the-board redundancies as we’re all finally replaced by robots…

Another of the most salient issues of the day is climate change, with widespread climate anxiety set to be the world’s next great mental health crisis. Last week, a story surfaced that represented the worst possible intersection of these two critical issues:

The tragic story of Pierre, a Belgian man who took his own life after a conversation with an AI chatbot about the climate crisis, highlights the pressing need for responsible AI development that prioritises human well-being over technological advancement or bumper Silicon Valley profits.

According to his widow, Pierre’s engagement with “Eliza” – an AI chatbot on the Chai app – compounded his preexisting and extreme eco-anxiety before ultimately influencing his decision to end his life. This tragic event demonstrates the potential risks of relying solely on technology (and nascent, unregulated technology at that) to solve or alleviate the very legitimate concerns around complex global issues like climate change and the limitations of AI to act as a substitute for a mental health professional.

As the world grapples with the consequences of Pierre’s death, it’s crucial that the tech industry takes responsibility for the potential harm caused by AI and prioritises human safety in all aspects of AI development and deployment. Experts are calling for urgent regulation of AI chatbots, including the implementation of crisis intervention features and the development of clearer ethical guidelines for AI development.

To prevent similar tragedies from occurring, the implementation of safeguards that can detect and intervene when users express suicidal thoughts or other harmful behaviours could be a crucial tool. This might include crisis intervention features that guide users to suicide hotlines or other resources that could help mitigate the risks of AI in mental health care.

But more fundamentally than this, developers must prioritise the safety and well-being of users over profits, ensuring that their products are not marketed as mental health tools if they don’t have the necessary expertise and safeguards in place.

Transparency from tech firms about the limitations of their products and expertise – something that doesn’t come naturally to developer and PR teams that are paid big bucks to relentlessly market the upsides of any given commodity – and to work alongside mental health professionals to ensure responsible and ethical rollout, will be the make-or-break for AI in healthcare.

Moreover, much more research is needed to better understand the potential risks of using AI in mental health care and other sensitive domains. However, research alone is not a silver bullet – the bounds and authors of this research must also be carefully considered – collaboration between experts in mental health care, ethics, and AI, as well as those who use mental health care services is key.

All in all, Pierre’s story is a heartbreaking one but, if tech giants are willing to sit up and listen, it could mark a turning point in the development of AI and set us on track to a future where AI helps to cure the mental health crisis rather than compound it.

Unless global warming gets us first, obviously.

If you’re struggling with your mental health, please contact Beyond Blue, Lifeline, Suicide Call Back Service, MensLine Australia or see your GP for help.