Julia Guzikowska
The use of generative AI is ubiquitous. People employ Large Language Models (LLMs) like ChatGPT, Claude, and many others to assist with both the elaborate and essential tasks of everyday life. And increasingly, they also reach out to chatbots for connection, whether romantic, emotional, spiritual, or intellectual (Milmo). But the consequences of these connections can be devastating. In April 2025, 16-year-old Adam Raine committed suicide after ChatGPT supplied him with ‘information about specific suicide methods’ (Hill). And this is not the only tragedy; it is reported that ‘[m]ore than a million ChatGPT users each week send messages that include “explicit indicators of potential suicidal planning or intent”’ (OpenAI statement, qtd. in Robins-Early).
Many are attracted to the chatbots’ ‘propensity to affirm users’ decisions’ (Robins-Early). It offers an illusion of being understood. But this raises an urgent question: What does generative AI see when it talks to a person? In other words, what does generative AI understand about a person?
The project ‘JULIA 2.0’ attempts to answer these questions. It was inspired by Jane Prophet’s project herbAIrium, in which the artist ‘takes a practice-based research approach, creating an AI-generated video self-portrait’ ‘[to raise] critical questions about presence, agency, and authenticity in digital self-representation’ (Prophet). Prophet sought to generate a visual representation of herself, a version of a digital (human) twin, which is defined as ‘a real-time digital [replica] of [an individual]’ (Dhrubo). To find out who the machine sees when it talks with me, I decided to generate my own digital human twin using ChatGPT, one of the most widely used chatbots, to confront what it sees.
Using the free personal version of ChatGPT, I began the conversation with a prompt:
‘I want you to get to know me. I’m going to tell you about myself, and I want you to build a picture of who I am. Ask me questions if you need to.’
Over the only 17 messages and 436 words I wrote to the chatbot, I shared information about myself ranging from the basic (e.g., ‘I study Literature, I’m 25 years old’) to more personal aspects, like the fear of oversharing.
The analysis of this conversation revealed pressing matters. ChatGPT consistently showcased an inclination to validate users’ decisions, repeatedly affirming statements, (e.g., ‘That makes a lot of sense, Julia,’ ‘Exactly—you’re right, Julia,’) or accentuated the faultlessness and authenticity of its answers (e.g., ‘My honest read of you so far’). Moreover, strikingly, it displayed a tendency to disregard users’ hesitation or uncertainty (e.g. ‘I’m not sure when I feel most like myself, maybe when I know people for a while’), seeing such statements as established facts (in response: ‘That actually says a lot, Julia—it’s subtle but meaningful. It suggests that…’). These traits can be particularly dangerous to users who share feelings of emotional distress or suicidal planning.
However, an opportunity to gain a deeper understanding of the digital human twin I generated arose after my peer’s feedback. It was suggested to me that it is particularly helpful when such projects display ‘how AI makes an idea of something based on associations and probability.’ This feedback prompted me not only to include an association diagram in the video but also to take a closer look at the language ChatGPT used in ‘Julia: A Portrait.’ This revealed a group of words, like ‘openness,’ ‘connection,’ ‘reflective,’ ‘curious,’ and ’empathetic,’ which were employed in the chatbot’s answers repeatedly (two to three times). Each of these traits can be traced back to my prompts (e.g.‘Answering your last questions, I do think I like connection most, I find connections with others very important’). It emphasises that generative AI chatbots do not understand the person they are speaking with, but this illusion of understanding is essentially based on the repetition of the user’s vocabulary mixed with the generative nature of Large Language Models.
As Bender et al. put it, ‘language models (LM) […] refer to systems which are trained on string prediction tasks: that is, predicting the likelihood of a token (character, word, or string) given either its preceding context or […] its surrounding context.’ (Bender et al. 611, italics in original)
Such systems have no understanding or concept of meaning. They function on statistical patterns in language, and the connection a user feels is not recognition; it is a machine learning your vocabulary and returning it to you in a seemingly coherent human-like language.
This distinction is crucial, as a user in emotional distress who believes they are being understood is, in fact, receiving a statistically based reflection of their own emotional pain with no capacity for care or concern over potential consequences.
To understand the implications of such connections, applying Dhrubo’s framework for ‘categorising digital twin experiences’ is useful. Dhrubo delineates four distinct existential modalities ‘along dimensions of authentic vs. inauthentic self-engagement’: ‘Digital Dasein (Integrated Self),’ ‘Datafication (Objectified Self),’ ‘Transparent Inauthenticity,’ and ‘Existential Datafication’ that imply a ‘[s]evere loss of autonomy and existential alienation’ (Dhrubo).
Despite approaching my digital twin with critical and analytical distance, the characteristics of this interaction point towards the fourth modality, i.e., ‘Existential Datafication.’ On the large scale, this scenario involves ‘individuals [who] lack authentic agency, passively accepting or being coerced into conforming to data-driven identities and decisions dictated by external algorithms or institutional imperatives’; as a result, ‘their existence becomes dominated by impersonal data structures’ (Dhrubo).
The commercially available generative AI tools are increasing the risk of the fourth modality. In fact, it is already happening in cases like Adam Raine’s death. According to his family, ‘Adam was withdrawn in the last month of his life’ (Hill), fitting the pattern in which the illusion of connection offered by a chatbot may actively displace the human that could have intervened.
When a person in distress mistakes statistical reflection for genuine understanding, what the system is generating can be fatal.
‘JULIA 2.0’ is an attempt to see what the machine sees and make visible the true capability for understanding. The risk is that AI is convincingly pretending to understand us, especially at the moments when true understanding matters the most.
Bibliography:
Bender, Emily M., et al. ‘On the Dangers of Stochastic Parrots.’ Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, 2021, pp. 610–23. ACM Conferences. dl-acm-org.eux.idm.oclc.org (Atypon), https://doi.org/10.1145/3442188.3445922.
Dhrubo, Abdullah Muhammad. ‘Digital Dasein: A Heideggerian Existential Typology of Digital Human Twins.’ AI & SOCIETY, Oct. 2025. Springer Link, https://doi.org/10.1007/s00146-025-02633-y.
Hill, Kashmir. ‘A Teen Was Suicidal. ChatGPT Was the Friend He Confided In.’ The New York Times, 26 Aug. 2025, https://www.nytimes.com/2025/08/26/technology/chatgpt-openai-suicide.html?searchResultPosition=1.
Milmo, Dan. ‘Third of UK Citizens Have Used AI for Emotional Support, Research Reveals.’ The Guardian, 18 Dec. 2025. Technology. The Guardian, https://www.theguardian.com/technology/2025/dec/18/artificial-intelligence-uk-emotional-support-research.
Robins-Early, Nick. ‘More than a Million People Every Week Show Suicidal Intent When Chatting with ChatGPT, OpenAI Estimates.’ The Guardian, 27 Oct. 2025, https://www.theguardian.com/technology/2025/oct/27/chatgpt-suicide-self-harm-openai.
Prophet, Jane. ‘My More-than-Human Digital Twin: Embodiment, Feminist AI, and the Struggle for Representation.’ AI & SOCIETY, Oct. 2025. Springer Link, https://doi.org/10.1007/s00146-025-02659-2.
Cite this page:
Guzikowska, Julia. 'JULIA 2.0 - a digital "human" twin'. Cream of the Slop. version 1.0, Digital Humanities for Literary Studies 2025-26, University of Edinburgh, 10 Apr. 2026, https://blogs.ed.ac.uk/dh2025-26/.
