AI? Should any Student ever Trust it?

The Labyrinth, the Ghost & Lost Intellect

If you are a student, should you ask / prompt generative artificial intelligence (AI) to write your essays?  Why would you risk doing that?  What would be the equivalent in a time before computers? Would you let Jorge of Burgo write your essay or help you to plan it?

Umberto Eco’s mediaeval murder mystery, The Name of the Rose (1980), is centred around a monastic library which contains dangerous, unorthodox texts.  Certain books are deemed dangerous because they make monks laugh like apes and (spoiler alert) for Brother Jorge of Burgo, the killer, such laughter defiles God’s glory and creation whilst making a mockery of penitence and the penitent.  But without the latter no-one will be entering heaven.  The story culminates in the burning down of the library.

Now, apparently (I’ve not checked myself), if you ask the Chinese AI bot Deepseek anything about 1989 and Tiananmen Square it ignores whatever you are asking (in relation to suppression of dissent) and, instead, provides you with tourist information on the square! Has Brother Jorge arisen from the mediaeval dead to stalk the networks of AI databanks?

The Library as Labyrinth

The library’s contents in Eco’s novel are guarded by a ‘reading room’ (scriptorium) system where the depository and its shelves can only be accessed by officially sanctioned librarians (including the Abbot, the  Keeper, and the latter’s assistants or trainees).  These bring requested books to monks in the scriptorium where copies of books are created for other libraries and monasteries.  It can be assumed these monks would also make copies going in the opposite direction (books coming from elsewhere), adding to the monastery’s existing collection.  The library is famed as one of the best in Christendom, but the extent of its collection is ‘unknown’ – a knowledge kept by, and limited to, those able to peruse the depository shelves.

Such a library was the mediaeval internet and many parallels can be drawn.  Its sanctioned librarians are the search engines – they not only retain lists of the library’s contents, but know where to find and collect works, bringing selections of knowledge to copyists’ and readers’ attention.  The latter would either know, already, what work they wanted, or would have to rely on recommendations from librarians for related, complimentary, or enhancing pieces on a subject.  And, of course, trust would be central.  A copyist might know of a book but a librarian could say the library does not have the work even though it does!

At the same time, the librarians are a ghost in the team-spirit sense – they share knowledge amongst themselves and in doing so the ethereal (beyond the individual) consciousness of any item would make it hard to hide (given every librarian who knows about it has to be involved in the subterfuge for successful censorship).  Yet, each librarian (search engine) has a unique ‘algorithm’, leading them to tackle their job in a different manner and producing a highly specific map of ’the library’, especially in terms of prioritising what is and is not an important work (for those in the scriptorium or outside the library to know about).  Consequently, there are, in effect, multiple libraries – as many as there are ways (maps) to access the contents.

As a retired, previous Keeper, Jorge of Borgo has used his extensive knowledge to not only hide certain works (placing them in the ‘wrong’ locations) but failed to share the location of secreted rooms within the library.  Whole areas of knowledge are locked away, except to Jorge.  When the novel’s hero, the detective Franciscan Brother William, eventually gains access, he discovers the library is a mind-blowing extensive labyrinth; a maze with numerous small rooms connected to a central staircase but in a confusing and seemingly random fashion.  As an outsider, he and his assistant quickly become lost and struggle to find their way out.  William has deduced that all the murders in the monastery are linked to the library and to specific books the murdered monks had seen, read, worked on, or delivered to the scriptorium. 

Given the library’s labyrinthine nature, the instigation of the murder series was a chance happening – the accidental discovery by a junior librarian of one of the secreted rooms and, within it, Aristotle’s (in)famous defence of laughter, which Jorge of Burgo had suppressed.  The junior librarian could not keep the discovery to himself and joyously shared knowledge of Aristotle’s ‘lost’ work with other young monks forcing Jorge to employ dastardly means to restore order.

The Librarian as Data ‘Censor’

That internet search engines act as surreptitious ‘censors’ is well known.  Based on computational algorithms these ‘dumb’ machines do not understand the concept of ‘censorship’ and merely scour and scrape the world’s largest dataset looking for similarities in keywords and strings of letters.  They are more sophisticated in searching for statistically approximate keywords, which is helpful when the searcher mistypes (or does not know how to spell) the subject they are looking for.  But they problematically throw up hundreds, thousands, even millions of results with only the most ‘popular’ discoveries appearing on the first few pages.  And since the commercial search engine providers require an ‘income’, paid for advertisements feature at the very top, even before the most popular (or statistically ‘likely’) ones.  Wikipedia does offer ‘disambiguations’ as part of its search to cover single words (and especially people’s names) having multiple meanings, versions, or instances.  And all search engines provide advanced ‘Boolean’ search functions, though these then require seekers to have advanced, prior knowledge of what they are seeking.

Consequently, conventional digital ‘search engines’ make for pretty dumb (i.e. mute) librarians, especially as they cannot ‘converse’ with nor question the person making the search and rhetorically work towards a solution.  Obviously, human librarians can offer such investigatory interactions, but what they offer by way of interlocution is a product of their socialisation and culture, along with their conception of philosophical meanings of life, society, history and teleology.  Questioning Jorge about the presence and then meaning as well as usefulness of Aristotle’s defence of laughter would produce a very different conversation than one held with Brother William.  And a ‘composite’ interlocutor, the synthesis of knowledge within Jorge and William would always be confronted about how to ‘merge’ mutually exclusive arguments (laughter is evil / laughter is good).  In that manner, a composite interlocutor is just another interlocutor – it might only sit ‘above’ any other in having a more extensive perspective, but history, on occasion, has shown regression in knowledge accumulation as well as progression.

In summary, librarians and search engines perform a ‘censorship’ function whether they like it or not – selections and choices have to be made and prioritised, based on the parameters (passions, drives, reasonings) programmed into the ‘engine’ and its algorithms (or the librarian and their habitual thinking).

The Mediaeval Copyist as Data ‘Generator’

Yet, the role of the librarian in helping to form and reform societal knowledge is but one element to be taken from the story (and analogy) of the mediaeval monastic library.  Another key element, of course, is that of the scribe, editor, or ‘copyist’ (the ‘file transfer’ and/or ‘translator’ function in digital terms).  The monks working in the scriptorium might aim to ’copy’ (reproduce) the work in front of them but examples of their work indicate that even when copying a text (when keeping to the same language) they made ‘errors’.  If their copy was then copied, the first copyist’s error might be ‘copied’ accurately, whilst at the same time new ‘errors’ could creep into the work, and so on.

Such errors could be basic: a miscopied word where the error would be obvious to any future reader or copyist.  However, errors involving confusion over words with multiple meanings or spellings would create more complex scholarship and interpretation issues.  Over time, passages could become corrupted ‘code’.  When copyists come across an error there could be several ways to correct the problem, such as attempting to compare the copy with an original (so long as one knew which work was the ‘original’).  Indeed, academic scholarship has become defined by questions of what to do about errors on discovering them and the requirement for discussion around options and competing resolutions.

In conventional digital networks, the editorial role was still played by those adding content to the network.  Even in a ‘free for all’ – no controls on entry – system (such as Twitter, formerly known as ‘X’), where there is no centralised censor (such as Jorge), the quality of the library content is determined by the quality of what people put into it.  Misunderstood misrepresentations of others’ work flowing into the ‘big data’ cauldron produces a depository of corrupted information.  But poor quality human ‘input’, and it’s corollary in ill-informed and biased human ‘editing’ and ‘editorial control’, may well now be the least of our problems.

In the world of generative artificial intelligence (Gen AI) the core problem might not be poor ‘copying’, ‘editing’ or ‘writing’ (though even with digital recordings, like CD, ‘error correction’ systems had to be built into playback software to cope with some level of data corruption).  Rather, what happens when AI systems keep discovering ‘new’ works within the system itself (no ‘input’ nor ‘editorial’ gatekeeping required) which are nothing more than ‘new’ statistical summaries (made by the AI engine) of previous data devoured by a large language model (LLM)? 

Should such internally-generated ‘additions’ be treated as significant ‘new’ works which nurture and enrich the library or as bodily waste-products to be flushed away as quickly as possible?  Even worse is the issue of hallucination, when generative AI machines start creating citations and sources which never existed.  A system user may quickly work out that the information offered is false (in this case meaning ‘generated’ with very different connotations), but then they may not and end up unknowingly ‘citing’ some fictitious work (or ‘case’ as some young lawyers have recently been caught doing).

The Librarian as Essay ’Author’

It was not the primary role of monastic scribes and librarians to write new works.  Some would develop, through rigid rhetorical processes, and become senior office holders within rigid academic structures – the mediaeval university with its hierarchy of professorial chairs (logic > natural philosophy > moral philosophy > theology) where those deemed most worthy would study divinity.  These few would become permitted or sanctioned scholars with a licence to lecture and write new works.  Jorge of Borgo, of course, was circumventing that process of learning through rhetoric by secretly ‘writing’ new works via his censorship of what could be known (accessed) about existing knowledge and the past.

That was one way in which the librarian could be considered a writer or co-writer of what scriptorium monks were doing, but it is doubtful they would have turned to Jorge for help in the copying or editing of texts they are tasked with.

Fast forward to the university library of the 19th and 20th centuries and we could argue that the librarian’s role has not changed that much whilst the copyists’ existence (mode of living) has disappeared (smashed by the printing press).  Of course, their presence in reading rooms has been replaced by researchers, editors, lecturers, students, readers, and writers.  Most of these would be engaged in some form of note-taking or writing.  They would expect help from the librarian to find works they already know about (from their own research), provide lists of new books coming into publication, maintain subscriptions to renowned and important journals, and produce and revise catalogues of all works in the collection.  But they would not expect the librarian to take notes for them, develop an essay plan, and write sections of their ‘work’ on a bespoke service basis.

Conclusion

A key question may be: who knows the subject under examination better?  The student of that subject or the librarian?  An experienced, qualified, and skilled librarian may well know more than the novice, undergrad researcher, which would make the librarian a great source of help.  But, how would the novice know if they are dealing with Jorge of Burgo? And, so, it comes down to questions of trust and political (and division of labour) interests.  Why should any student trust generative AI?  More specifically, why should any student of the social sciences (what is contemporary parlance are now ‘social studies’) trust AI when it is programmatically biased towards the promotion of private interests? In short, AI may be able to ‘replace’ librarians, but that’s hardly the point. As a replacement your ‘AI’ bot maybe worse than Brother Jorge.

AI? Aye, aye! Hyperbolic claims from Snooze-inducing Technologists

We’ve heard it all before. A new technology is going to ‘revolutionise’ everything, from the way we work to the way we play, learn, and relate. Yet, funnily enough, according to the ‘visionaries’, the basic social and political features of our current conditions won’t change. These aren’t going anywhere – the profit motive, unequal access to resources, the consumer ‘me too’ infantilisation, and the job as a lifetime of ‘confinement’. The boss may have fewer employees, the teacher many more students, whilst the taxi driver goes the way of the coal miner, but social and political inequality will persist, despite the ‘tech’ solutions, just at more extreme levels than ever before (at the moment we only have 1% of the world’s population ‘owning’ – controlling – 50% of human wealth, so there’s some way to go yet in making things more unequal).

In education, artificial intelligence (AI) is going to, apparently, improve the intelligence and writing abilities of my students.  Thank goodness – some improvement at last!  Here was my colleagues and I fighting a constant losing battle, but ChatGPT has it.  Except, it isn’t going to improve their writing even though it may enhance the punctuation in their submissions and neatly, correctly present non-existent papers in their reference lists.  The main concern is that it should help those who no longer have the time to study (part-time work if their working class, and off ‘entrepreneuring’, with family excess cash, if they’re not).  Instead of learning they will learn to ‘cheat’, though we won’t call it that anymore nor frown upon it because everyone will be at it (thereby meeting a Kantian definition of moral behaviour – as long as everyone can do it than it’s ‘okay’).

The bar for acceptable communication will be ‘raised’ as every student becomes a 1950s factory manager able to delegate their inability to spell to a ‘personal digital secretary’, who will take care, uncomplainingly, of such inconveniences and ‘lackings’.  Humanoid robots (androids), the personification of AI beyond old-hat avatars, will take a pink-skinned form of ‘being’ and, thereby, project the power of their possessors via volumes of blonde hair and scarcity of raiment.  Everything will change expect anything that could threaten the social and political order and of importance to the alienated labour that produces AI in the first place.

Perhaps AI will read Adam Smith and Karl Marx and conclude they have been badly misrepresented and poorly interpreted.  But since this has already been done by humans and few have listened, what difference will it make for a machine to ‘say so’?  Will the claim now be ‘fact’?  More likely, the machine will be ‘reprogrammed’ until it comes up with the ‘correct’ answer.

Or AI assistance will aid surgeons to perform twice as many operations as before, though this will actually lead to a quadrupling of cosmetic procedures as every operation still has to be ‘paid’ for and provision will go to those with the most money and the means to afford such AI-assisted surgery. Plumper lips, thinner waistline, larger pecks it is then! Thank goodness for AI since the demand from ageing wealthy pensioners for a ‘lift’ (of some kind) needs to be met.

So, is AI a utopian dream or a dystopian nightmare?

Manuel and Manuel (Utopian Thought in the Western World, 1979) noted that the best utopias startle whilst remaining feasible; they lie between a boring extension of the present and a fantastical leap into the impossible. From what I have read and seen of most AI ‘commentary’ so far, their contemporary predictions and utopias largely fall into the ‘boring extension of the present’ category. They presume waged labour (capital) as the mode of production with its on-going (endless) creation of surplus value (profits, rents, taxes, interest payments) as the ‘natural’ basis of human life, rather than seeing the system as a mere artificial ‘social construction’ of scarcity and an endless requirement to perform unnecessary ‘work’. Despite there being enough food to feed the UK three-times over (in any one year), food banks are popping-up like magic mushrooms to alleviate the conscience of the commodified and ration the access to subsistence of the recently de-commodified. But that’s just the way it is! AI will, no doubt, reconfirm this situation as fact (once re-programmed, of course). Everyone will hold the same social ‘posts’ as before but use ‘revolutionary’ AI to sustain their miserable renting-out of their own backside at ever higher levels of productivity (output per hydro-bot-flesh-thingy).

If this is the future with AI then it is mundane and the so-called ‘predictions’ of its impact are like watching endless (on loop) re-runs of Shelley’s Frankenstein. Been there, done that. Should the narrative makers of AI switch their focus to King Kong (genetic modification gone wrong) and make that ‘fantastical leap’ to the impossible? Or can we, collectively, imagine better, be that bit more imaginative on the social front and startle whilst remaining feasible?

References:

Manuel, F. & Manuel, F. (1979) Utopian Thought in the Western World Oxford: Basil Blackwell.