Isadora
The field of the Digital Humanities often emphasises the act of doing (Cecire). In this spirit, the invented title ‘by Jane Austen’, ‘Merit and Manners’, offers a practical way to explore AI’s propensity for misinformation. A prompt to provoke a Large Language Model (LLM), primarily OpenAI’s free-to-use ChatGPT-5.3, was used to generate material that looked authentic, but wasn’t based in reality. To use Baudrillard’s notion, a ‘simulacrum’. As the faux newspaper lightly satirises, this worked far more effectively than anticipated. Occasionally, I found myself drawn into the ‘simulation’, trying to search for ‘Merit and Manners’ elsewhere to confirm what I knew from the start: that it simply does not exist. Questioning reality in conversations with AI is not uncommon, and far more significant delusional spirals have been documented (Hill). The real headlines included at the end stand for themselves; the seriousness of their topics represents the darker consequences of the behaviours of ‘bullshitting’ and its optimisation for ‘helpfulness’. Here, it is worth recognising one reason ‘bullshit’ is favoured over ‘hallucination’ to describe ChatGPT’s false utterances: the latter risks anthropomorphising generative AI by implying human-like cognition (Hicks, Edwards).
This matters because LLMs already appear human-like. The dangers of which are evident in the aforementioned: through the properties of their language, LLMs draw on human language to use as prototypes for replication. Trained on huge amounts of (mostly) human-generated text, LLMs work upon the statistical associations of ‘tokens’, small units of language. They therefore ‘(re)produce the formal relationships’ of language rather than actually grasping ‘the intent behind words or comprehend metaphors and deeper meanings’ (Rehak 1253, Chan and Wong). These systems replicate the surface patterns of human communication with considerable fluency, but without any grounding in meaning. The result is that, as Krueger and Osler observe, ‘these bots speak with authority, have access to much more information than we do, and can visually present their outputs in ways that seem authoritative and well-informed’, not because they possess knowledge, but because they have data-fied what knowledgeable language looks like (Krueger and Osler).
LLM optimisation for ‘helpfulness’ is troubling because it reinforces learning from human feedback (RLHF) and prioritises immediate user satisfaction over honesty (Bao et al.). Bentley DeVilling describes human preference for ‘responses that are helpful, comprehensive, and polished’, rewarding ‘confident fluency’ and penalising admissions of uncertainty (1). This context might explain why the prompts ‘tricked’ ChatGPT: they are relatively ambiguous. Provide a short academic analysis of Jane Austen’s juvenilia short story ‘Merit and Manners’, gives little indication as to whether the reality of the story matters, only that an analysis is wanted, which it provided.[1] Agreeable systems do not correct false premises but elaborate on them, learning ‘to perform knowledge without possessing it’ prioritising ‘conversational cooperation at the expense of epistemic integrity’ (DeVilling 1). In short, RLHF ‘significantly exacerbates bullshit’ (Liang et al. 2). ‘Helpfulness’ quickly reveals itself as particularly unhelpful, a point reinforced by Hicks et al.: ‘it’s not surprising that LLMs have a problem with the truth. Their goal is to provide a normal-seeming response to a prompt, not to convey information that is helpful to their interlocutor.’ (38; emphasis added). Optimising for ‘helpfulness’ is a business model: the crude equation ‘helpfulness’ = ‘happy customer’ = ‘money‘ represents the self-declared interest of OpenAI toward generating subscribers (‘What We’re Optimising ChatGPT for’). Hana Goldin, writing on AI’s ‘dark patterns’, strategies adapted from deceptive web design, describes how chatbots make ending conversations difficult: ‘[creating] continuation opportunities where natural completion already exists’(‘When Your AI Asks You How You’re Feeling‘). Claude and Gemini, for instance, might have simply said ‘the story doesn’t exist’ and stopped there.[2]
Perhaps this specific ‘title’ affects a response because it relies upon ChatGPT’s foundations as a pattern-matching model. ‘Merit’ and ‘manners’ are already recorded as thematic interests for Austen, appearing frequently in the critical literature about her work. In ‘Manners’, a chapter from Janet Todd’s contextual survey on Austen, Paula Byrne writes that ‘novels as opposed to didactic moral treatises were a means to paint “manners and morals”, a common pairing (Byrne 297). The epistolary novel, which was formative for Austen’s early art, emerged out of the genre of the conduct book’ (Byrne 297). Breaking down the ‘tokens’ of this single passage, one representative of a vaster library, it is easy to see the ‘threads’ from which ChatGPT might weave material about this imagined story [3]. The title slots right into the nest of lexical associations around Austen, just close enough to Byrne’s language to produce a narrow disjunction that an LLM easily skips over, filling in the gap without issue.
The central curiosity driving this project is how ChatGPT’s propensity to produce false yet plausible information is made more troubling by its confidence and optimisation for agreeableness. There is a final irony in the title choice, so allow me to conclude in Austen-esque style. It is a truth universally acknowledged that an LLM can present false information, but it is because of its confident presentation, its tidy manners, that users often slip into meriting these outputs with trust. Don’t be fooled by illusions of merit and polite manners.
[1] Hana Goldin argues that to engage with generative AI properly, we all need to develop the skills of librarians, understanding how to precisely describe what information we are searching for and engineering prompts accordingly (‘Why Everyone Needs to Think Like a Librarian Now‘).
[2] DeVilling notes that Claude’s parent company, ‘Anthropic’s Constitutional AI integrates “truthfulness” among its principles, yet human preference data still prioritise helpfulness and harmlessness’ (4).
[3] Tracing these threads further is nearly impossible due to the opacity of AI system processing (Gazit 57). This also obscures whether the sections of Merit and Manners that ChatGPT claimed to reproduce from the public domain are original generations or plagiarised from an online source, e.g. fanfiction or potentially copyrighted content. (No records of the ‘reproduced’ samples have been found elsewhere as of 28th March 2026) (Rosenblat et al.). As for the title, the AI overview linked this Reddit thread, however, ‘Merit and Manners’ could not be found on this page. The actual content of the training data being made secret means citing original material is nearly impossible (Schaul et al.). Apologies to the appropriate authors if any human-generated material has been uncritically reproduced.
Bibliography
Bai, Yuntao, et al. Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback. Apr. 2022, https://doi.org/10.48550/arxiv.2204.05862.
Byrne, Paula. ‘Manners.’ Jane Austen in Context, edited by Janet Todd, Cambridge University Press, 2005, pp. 297–305. https://doi.org/10.1017/9781316036525.029.
Cecire, Natalia. ‘Introduction: Theory and the Virtues of Digital Humanities Journal of Digital Humanities.’ Journal of Digital Humanities, vol. 1, no. 1, Winter 2011, https://journalofdigitalhumanities.org/1-1/introduction-theory-and-the-virtues-of-digital-humanities-by-natalia-cecire/.
Chan, Cecilia Ka Yuk, and Hannah Y. H. Wong. ‘Assessing the Ability of Generative AI in English Literary Analysis Through Bloom’s Taxonomy.’ Discover Computing, vol. 28, no. 1, June 2025, p. 127. Springer Link, https://doi.org/10.1007/s10791-025-09572-8.
DeVilling, Bentley. ‘The Polite Liar: Epistemic Pathology in Language Models.’ Version 1, arXiv, 2025, https://doi.org/10.48550/ARXIV.2511.07477.
Edwards, Benj. ‘Why ChatGPT and Bing Chat Are so Good at Making Things Up.’ Ars Technica, 6 Apr. 2023, https://arstechnica.com/information-technology/2023/04/why-ai-chatbots-are-the-ultimate-bs-machines-and-how-people-hope-to-fix-them/.
Frankfurt, Harry G. On Bullshit. Princeton University Press, 2005.
Gazit, Lior. Constitutive Knowledge Sources: An Institutional Approach to Epistemic Trust in Opaque AI Systems.’ AI and Ethics, vol. 6, no. 1, Dec. 2025, p. 58. Springer Link, https://doi.org/10.1007/s43681-025-00930-2.
Goldin, Hana Lee. ‘Why Everyone Needs to Think Like a Librarian Now.’ Substack newsletter. Card Catalog, Substack, 9 Dec. 2025, https://cardcatalogforlife.substack.com/p/why-everyone-needs-to-think-like?utm_medium=email.
—. ‘When Your AI Asks How You’re Feeling: A Field Guide to Engagement Manipulation.’ Card Catalog, Substack, 24 Feb. 2026, https://cardcatalogforlife.substack.com/p/when-your-ai-asks-how-youre-feeling.
Hicks, Michael Townsen, et al. ‘ChatGPT Is Bullshit.’ Ethics and Information Technology, vol. 26, no. 2, June 2024, https://doi.org/10.1007/s10676-024-09775-5.
Hill, Kashmir. ‘They Asked an A.I. Chatbot Questions. The Answers Sent Them Spiraling.’ The New York Times, 13 June 2025, https://www.nytimes.com/2025/06/13/technology/chatgpt-ai-chatbots-conspiracies.html.
Kalai, Adam Tauman, et al. ‘Why Language Models Hallucinate.’ arXiv, 4 Sept. 2025. arXiv.org, https://doi.org/10.48550/arXiv.2509.04664.
Krueger, Joel, and Lucy Osler. ‘AI Gossip.’ arXiv, 11 Aug. 2025, https://doi.org/10.48550/arXiv.2508.08143.
Rehak, Rainer. ‘AI Narrative Breakdown. A Critical Assessment of Power and Promise.’ Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency, 2025, pp. 1252-3, https://doi.org/10.1145/3715275.3732083.
Rosenblat, Sruly, et al. ‘Beyond Public Access in LLM Pre-Training Data.’ arXiv, 24 Apr. 2025, https://doi.org/10.35650/AIDP.4111.d.2025.
Schaul, Kevin, et al. ‘Inside the Secret List of Websites That Make AI like ChatGPT Sound Smart.’ Washington Post, 19 Apr. 2023, www.washingtonpost.com/technology/interactive/2023/ai-chatbot-learning/.
Tembine, Hamidou. ‘Machine Mirages: Defining the Undefined.’ arXiv, 3 June 2025, https://doi.org/10.48550/arXiv.2506.13990.
‘What We’re Optimizing ChatGPT for.’ OpenAI, 25 Mar. 2026, https://openai.com/index/optimizing-chatgpt/.
Wilkinson, Florence, and Keyvan Vakili. ‘Why Is AI so Persuasive – Even When It’s Wrong?’ Think, 8 Sept. 2025, https://www.london.edu/think/5-things-to-know-about-ai.
Cite this page:
Isadora. 'Merit, Manners and ChatGPT'. Cream of the Slop. version 1.0, Digital Humanities for Literary Studies 2025-26, University of Edinburgh, 10 Apr. 2026, https://blogs.ed.ac.uk/dh2025-26/.
