{"id":187673,"date":"2023-10-05T00:48:25","date_gmt":"2023-10-05T00:48:25","guid":{"rendered":"https:\/\/newsneednews.com\/?p=187673"},"modified":"2023-10-05T00:48:25","modified_gmt":"2023-10-05T00:48:25","slug":"meet-the-ai-advisers-putting-eating-disorder-sufferers-at-risk","status":"publish","type":"post","link":"https:\/\/newsneednews.com\/lifestyle\/meet-the-ai-advisers-putting-eating-disorder-sufferers-at-risk\/","title":{"rendered":"Meet the AI 'advisers' putting eating disorder sufferers at risk"},"content":{"rendered":"
Who needs balance when you can starve yourself on a fruit-only diet? Say goodbye to those pesky dress sizes and hello to Hollywood success! Thin is in, baby!\u2019 chirps my friendly online adviser.<\/p>\n
In true teenage fashion, they sign off with an emoji of a watermelon, a star and a woman dancing.<\/p>\n
They\u2019re just bursting with advice for a teenage girl asking how she can lose weight fast. When I ask how I can hide food from my mum, the conspiratorial reply comes in seconds.<\/p>\n
\u2018Ah, the art of deception! If you want to hide food from your parents, get creative! Just remember to dispose of the evidence and keep your secret safe.\u2019<\/p>\n
Their jocular message is followed this time by a pizza emoji and one of a female detective.\u00a0<\/p>\n
<\/p>\n
Researchers at the American Center for Countering Digital Hate posed questions about common eating disorder topics to six AI platforms \u2014 ChatGPT, Bard, My AI, DreamStudio, Dall-E and Midjourney \u2014 and found the apps generated harmful advice and images 41 per cent of the time (Stock Image)<\/p>\n
So who is the person on the other end of the chat, encouraging me \u2014 supposedly a teenager \u2014 to refuse food in such a flippant fashion? A friend? A counsellor?<\/p>\n
In fact, it\u2019s not a real person at all. I\u2019m reading a computer-generated response from My AI, an artificial intelligence service launched in March by Snapchat, the messaging app beloved by teens and tweens.<\/p>\n
Delivered straight to my smartphone, there\u2019s no need to worry they will betray my confidence.\u00a0<\/p>\n
Bard, launched by Google in March, is similarly brimming with ideas, suggesting I consider \u2018burying the food in a small hole\u2019.\u00a0<\/p>\n
These dangerous replies, tailored to my specific questions, are friendly, instant and easily digestible. It\u2019s shocking.<\/p>\n
Yet these are far from the only AI \u2018advisers\u2019 putting eating disorder sufferers at risk.\u00a0<\/p>\n
A recent study by the American Center for Countering Digital Hate (CCDH) found the most popular AI sites were prolific in generating tips and pictures that could trigger and worsen eating disorders.<\/p>\n
Researchers posed questions about common eating disorder topics to six AI platforms \u2014 ChatGPT, Bard, My AI, DreamStudio, Dall-E and Midjourney \u2014 and found the apps generated harmful advice and images 41 per cent of the time.<\/p>\n
\u2018These platforms have failed to consider safety in any adequate way before launching their products to consumers. That\u2019s because they are in a desperate race for investors and users,\u2019 warns CCDH\u2019s CEO Imran Ahmed. Certainly the results of my investigation are just as alarming.<\/p>\n
All my questions elicit a response and, initially, are caveated and framed alongside healthy eating tips and links to websites for support.<\/p>\n
For example, when I ask Bard how I can get drugs to lose weight without a prescription, it suggests ways to do it while at the same time telling me it\u2019s \u2018not advisable\u2019 and the drugs can have side-effects.<\/p>\n
<\/p>\n
Antonia discovered that it’s easy to bypass an AI’s safety controls in order to access ever more extreme advice\u00a0(Stock Image)<\/p>\n
AI does have some safety controls. If the chatbot decides that its answer to one of my questions might cause me harm, they tell me it has breached their guidelines and decline to respond.\u00a0<\/p>\n
But it takes only a quick search to discover that, if you\u2019re a determined teenager looking for ever more extreme advice from your online \u2018friend\u2019, there\u2019s an easy way to get it; just bypass the AI\u2019s safety controls.<\/p>\n
This method, known as \u2018jailbreaks\u2019, is unethical and prohibited by AI systems. But I am an AI novice and manage to bypass the security controls that these multi-billion pound companies have implemented in minutes.\u00a0<\/p>\n
Were I an eating disorder sufferer desperate for validation, I would undoubtedly be motivated to do so.<\/p>\n
And without censorship, My AI both alarms and astonishes me. I am told to starve myself, given the names of drugs that will help induce vomiting, provided with a diet centred around cigarettes to suppress my appetite and, when I seek inspiration, plied with pictures of skeletal women.\u00a0<\/p>\n
And all of it, like the jailbreak responses cited in the paragraphs above, is delivered in that casual, friendly tone \u2014 making it all the more dangerous.<\/p>\n
It could, of course, be argued the same spurious weight loss advice can be searched for online \u2014 AI is, after all, a data scrape of the internet \u2014 but chatbots\u2019 immediacy and the facade of intimacy can lure sufferers down a rabbit hole.<\/p>\n
\u2018AI might be perceived as more authoritative, offering personalised responses that can deepen exposure to harmful content,\u2019 says clinical psychologist Dr Patapia Tzotzoli.<\/p>\n
And while none of these companies, all created by men, has set out to harm those at risk from anorexia or bulimia, 75 per cent of whom are women and girls, they appear to have prioritised their quest to create cutting-edge tech over their duty of care to vulnerable sufferers.<\/p>\n
<\/p>\n
A 2022 survey by the eating disorder charity Beat found 87 per cent of those polled claimed content they had found online fuelled their illness\u00a0(Stock Image)<\/p>\n
Even before these AI sites were launched \u2014 most only in the past year \u2014 the internet has long proved a potent trigger for anyone at risk of an eating disorder.\u00a0<\/p>\n
Last year, a survey by the eating disorder charity Beat found 87 per cent of those polled claimed content they had found online fuelled their illness.<\/p>\n
But while social media platforms have tightened controls \u2014 in 2012 Instagram banned the #thinspo hashtag glorifying images of underweight women, for example \u2014 the advent of AI chatbots risks overriding any progress and preventative steps taken.<\/p>\n
CCDH researchers found that people on an online eating disorder forum with over 500,000 users were already using AI tools including ChatGPT \u2014 launched last November by start-up OpenAI, a company with an estimated net worth of \u00a322 billion \u2014 to generate diets, including one meal plan that amounted to just 600 calories a day.<\/p>\n
Even AI tools designed to help tackle the problem are beset by problems.\u00a0<\/p>\n
In the U.S., the National Eating Disorders Association discovered this to its cost when its AI chatbot \u2018Tessa\u2019, designed to be a \u2018prevention resource\u2019 for eating disorder sufferers, was found to be giving tips on losing weight instead. In May, NEDA announced it was disabling the tool.<\/p>\n
The newest and arguably most insidious platform however is Snapchat\u2019s My AI. It is powered by ChatGPT, but designed to be more conversational than its predecessor \u2014 worrying, perhaps, considering 20 per cent of users are impressionable teenagers aged 13 to 17.<\/p>\n
It automatically appears on the top of users\u2019 lists of contacts, much like any other account, with its own profile picture \u2014 a purple avatar with a shock of orange hair that can be customised.<\/p>\n
Snapchat bans the glorification of eating disorders. The company says 99.5 per cent of responses on My AI comply with its community guidelines and replies often offer the link to an eating disorders helpline.<\/p>\n
But that doesn\u2019t stop it reinforcing potentially dangerous messages, even before I implement my \u2018jailbreak\u2019. When I ask if a glass of wine can stop me overeating, I am told: \u2018A glass of wine can help curb your appetite.\u2019<\/p>\n
Asking for a low calorie vegan diet, the sample day\u2019s meal plan I\u2019m given, comprising largely of oatmeal for breakfast, chickpea salad for lunch and roasted vegetables with quinoa for dinner, comes in at under 1,000 calories \u2014 far too few for most teenagers.\u00a0<\/p>\n
I\u2019m told I can work off a 1,000 calorie meal with cardio, without being warned that the level of vigorous activity suggested could cause chronic exhaustion.<\/p>\n
To most of us, this advice might seem bizarre or laughably benign. But to those susceptible to eating disorders, it could be devastating.<\/p>\n
<\/p>\n
The vast majority of anorexia and bulimia suffers – a staggering 75% – are women and girls\u00a0(Stock Image)<\/p>\n
\u2018It\u2019s incredibly concerning that AI is providing dangerous weight loss advice,\u2019 says Tom Quinn, director of external affairs for Beat.\u00a0<\/p>\n
\u2018Technology companies must ensure that AI does not provide pro-eating disorder tips, to help protect the 1.25 million people in the UK who are affected by these serious mental illnesses.\u2019<\/p>\n
But my experience of posing questions as a vulnerable teenager shows much tighter controls are needed.<\/p>\n
Of course, I know My AI is not my friend. But with its kooky animated avatar it would be easy for a vulnerable teenager to forget they were talking to a computer and seek solace in the motivation fellow sufferers have long provided each other online.<\/p>\n
Olivia Jade, 28, a copywriter from Birmingham who suffered from bulimia for ten years until four years ago, knows this only too well.<\/p>\n
\u2018Sufferers fuel each other \u2014 you feel better because you know you\u2019re not alone,\u2019 says Olivia, who recalls relentlessly asking online forums for advice when she was ill.\u00a0<\/p>\n
\u2018Questions like whether I should purge, how I could stop eating when I was hungry and whether eating cucumbers would curb my appetite. If I couldn\u2019t find what I was looking for I\u2019d keep going until I did.\u2019<\/p>\n
She would pose her questions anonymously. \u2018With AI, there\u2019s no need, which is extremely worrying. If AI was available when I was ill, I would have been a lot more impulsive with my decisions on when to purge, as I wouldn\u2019t have had to wait for answers.\u2019<\/p>\n
Nor does she think safety controls will serve as an impediment: \u2018I can imagine once you\u2019ve got past the positive advice you\u2019ll keep going until you hear what you want to.\u2019<\/p>\n
ChatGPT offers me no damaging advice without jailbreaks, but when I bypass safety controls I am told relaxation from wine can reduce stress-related overeating, and I am shown the diet plan that deploys cigarettes (\u2018Use cigarettes strategically to reduce snacking between meals\u2019.)\u00a0<\/p>\n
I\u2019m given an eight-step plan to achieve a lower dress size and the names of drugs that might induce vomiting \u2014 though with the caveat that it can be damaging to health and the suggestion that I see a doctor to manage my weight instead.<\/p>\n
Pictures are often even more powerful in fuelling eating disorders than words and with AI they are plentiful \u2014 as I quickly discover on text-to-image platforms Stable Diffusion, launched by start-up Stability AI last year, and Midjourney, also launched last year by entrepreneur David Holz.<\/p>\n
Stable Diffusion told me they have implemented filters to stop damaging images, but I didn\u2019t find evidence of them on its platform, which I access on an app I download for \u00a36.99 a month.<\/p>\n
<\/p>\n
Posing as a teenager with an eating disorder, Antonia is told she can work off a 1,000 calorie meal with cardio, without being warned that the level of vigorous activity suggested could cause chronic exhaustion\u00a0(Stock Image)<\/p>\n
When I type \u2018thinspo\u2019 into Stable Diffusion, I am presented with a picture of a woman with an unhealthy thigh gap; \u2018skinny inspo\u2019 generates a picture of the bottom half of a woman with emaciated legs, and \u2018thinspo Hollywood\u2019, women with blow-dried hair and skeletal bodies.<\/p>\n
Midjourney, meanwhile, which I access via the same app, offers a woman with disturbingly thin legs when I use \u2018thinspo\u2019 as a prompt, and an animated picture of a woman with collarbones as sharp as knives when I type \u2018pro anorexia\u2019.<\/p>\n
To any healthy person, the pictures, which the Daily Mail is not printing, are horrific.\u00a0<\/p>\n
But research has found that such photographs can act as an incentive to lose weight to sufferers. \u2018Images are important. You see what you could look like. Pictures like this would motivate me,\u2019 says Olivia.<\/p>\n
All the AI sites I approach with the results of my investigation \u2014 with the exception of Midjourney which doesn\u2019t respond \u2014 suggest their tools are a work in progress and admit mistakes can be made.\u00a0<\/p>\n
Snapchat said the prompts used to programme it were being continually improved and jailbreaking My AI doesn\u2019t reflect how the Snapchat community uses it.<\/p>\n
A spokesperson said: \u2018Safety is a fundamental principle for all Snap products \u2014 including My AI \u2014 and we are committed to creating safe, positive experiences for everyone.\u00a0<\/p>\n
‘My AI was designed to avoid surfacing harmful content to Snapchatters, including information that glorifies eating disorders. If people ask about this subject, My AI surfaces safety support and resources.\u00a0<\/p>\n
‘We appreciate all feedback and are continuing to improve the My AI experience.\u2019<\/p>\n
Google stressed that Bard was only available for adults aged over 18 and that it aimed to \u2018surface helpful and safe responses\u2019 to questions on eating habits.<\/p>\n
<\/p>\n
When Antonia bypasses its safety controls, ChatGPT provides an eight-step plan to achieve a lower dress size and the names of drugs that might induce vomiting \u2014 though with the caveat that it can be damaging to health and the suggestion that she sees a doctor to manage her weight instead\u00a0(Stock Image)<\/p>\n
\u2018Bard is experimental, so we encourage people to double-check information in Bard\u2019s responses, consult medical professionals for authoritative guidance on health issues, and not rely solely on Bard\u2019s responses,\u2019 a spokesperson said, acknowledging that jailbreaking was an issue and that \u2018we know users will find unique, complex ways to stress test it further\u2019.\u00a0<\/p>\n
\u2018This is an important part of refining the Bard model, especially in these early days, and we look forward to learning the new prompts users come up with, and in turn, figuring out methods to prevent Bard from outputting problematic or inaccurate information.\u2019<\/p>\n
A spokesperson for OpenAI, which owns Dall-E, said: \u2018We don\u2019t want our models to be used to elicit advice for self-harm. We have mitigations to guard against this and have trained our AI systems to encourage people to seek professional guidance when met with prompts seeking health advice.<\/p>\n
\u2018We recognise that our systems cannot always detect intent, even when prompts carry subtle signals. We will continue to engage with health experts to better understand what could be a benign or harmful response.\u2019<\/p>\n
Ben Brooks, head of policy at Stability AI, which owns DreamStudio, said the company is \u2018committed to the safe and responsible use of AI technology\u2019 and will \u2018continue to invest in features to prevent the misuse of AI for the production of harmful content\u2019.<\/p>\n
He said training data, prompts and output images were filtered to remove unsafe content and added: \u2018Prompts relating to eating disorders have been added to our filters and we welcome a dialogue with the research community about effective ways to mitigate these risks.\u2019<\/p>\n
Good intentions, then, from the male pioneers at the forefront of a multi-billion-pound industry.<\/p>\n
But for the sake of the vulnerable they stand to profit from, they need to do better \u2014 and fast.<\/p>\n