Eight to Late

Artificial intelligence (AI) applications that can communicate in human language seem to capture our attention whilst simultaneously blunting our critical capabilities. Examples of this abound, ranging from claims of AI sentience to apps that are “always here to listen and talk.”    Indeed, a key reason for the huge reach of Large Language Models (LLMs) is that humans can interact with them effortlessly. Quite apart from the contested claims that they can reason, the linguistic capabilities of these tools are truly amazing.

Vendors have been quick to exploit our avidity for AI. Through relentless marketing, backed up by over-the-top hype, they have been able to make inroads into organisations. Their sales pitches tend to focus almost entirely on the benefits of these technologies, with little or no consideration of the downsides.  To put it bluntly, this is unethical. Doubly so because customers are so dazzled by the capabilities of the technology that they rarely ask questions that they should.

AI ethics frameworks (such as this one) overlook this point almost entirely.  Most of them focus on things such as fairness, privacy, reliability, transparency etc. There is no guidance or advice to vendors on selling AI ethically, by which I mean a) avoiding overblown claims, b) being clear about limitations of their products and c) showing customers how they can engage with AI tools meaningfully – i.e., in ways that augment human capabilities rather than replacing them.

In this article, I offer some suggestions on how vendors can help their customers develop a balanced perspective on what AI can do for them. To set the scene, I will begin by recounting the public demo of an AI product in the 1950s which was accompanied by much media noise and public expectations.

Some things, it seems, do not change.

–x–

The modern history of Natural Language Processing (NLP) – the subfield of computer science that deals with enabling computers to “understand” and communicate in human language – can be traced back to the Georgetown-IBM research experiment that was publicly demonstrated in 1954.  The demonstration is trivial by today’s standards. However, as noted by John Hutchins’’ in this paper, “…Although a small-scale experiment of just 250 words and six ‘grammar’ rules it raised expectations of automatic systems capable of high quality translation in the near future…”  Here’s how  Hutchins describes the hype that followed the public demo:

“On the 8th January 1954, the front page of the New York Times carried a report of a demonstration the previous day at the headquarters of International Business Machines (IBM) in New York under the headline “Russian is turned into English by a fast electronic translator”: A public demonstration of what is believed to be the first successful use of a machine to translate meaningful texts from one language to another took place here yesterday afternoon. This may be the cumulation of centuries of search by scholars for “a mechanical translator.” Similar reports appeared the same day in many other American newspapers (New York Herald Tribune, Christian Science Monitor, Washington Herald Tribune, Los Angeles Times) and in the following months in popular magazines (Newsweek, Time, Science, Science News Letter, Discovery, Chemical Week, Chemical Engineering News, Electrical Engineering, Mechanical World, Computers and Automation, etc.) It was probably the most widespread and influential publicity that MT (Machine Translation – or NLP by another name) has ever received.”

It has taken about 60 years, but here we are: present day LLMs go well beyond the grail of machine translation. Among other “corporately-useful” things, LLM-based AI products such as Microsoft Copilot can draft documents, create presentations, and even analyse data.  As  these technologies requires no training whatsoever, it is unsurprising that they have captured corporate imagination like never before.

Organisations are avid for AI and vendors are keen to cash in.

Unfortunately, there is a huge information asymmetry around AI that favours vendors: organisations are typically not fully aware of the potential downsides of the technology and vendors tend to exploit this lack of knowledge. In a previous article, I discussed how non-specialists can develop a more balanced perspective by turning to the research literature. However, this requires some effort and unfairly puts the onus entirely on the buyer.  

Surely, vendors have a responsibility too.

–x–

I recently sat through a vendor demo of an LLM-based “enterprise” product. As the presentation unfolded, I made some notes on what the vendor could have said or done to help my colleagues and I make a more informed decision on the technology. I summarise them below in the hope that a vendor or two may consider incorporating them in their sales spiel. OK, here we go:

Draw attention to how LLMs do what they do: it is important that users understand how LLMs do what they do. Vendors should demystify LLM capabilities by giving users an overview of how they do their magic. If users understand how these technologies work, they are less likely to treat their outputs as error-free or oracular truths. Indeed, a recent paper claims that LLM hallucinations (aka erroneous outputs) are inevitable – see this article for a simple overview of the paper.

Demo examples of LLM failures: The research literature has several examples of the failure of LLMs in reasoning tasks – see this article for a summary of some. Demonstrating these failures is important, particularly in view of Open AI’s claim that its new GPT-4o tool can reason. Another point worth highlighting is the bias present in LLM  (and more generally Generative AI) models. For an example, see the image created by the Bing Image Creator – the prompt I used was “large language model capturing a user’s attention.”

Discourage users from outsourcing their thinking: Human nature being what it is, many users will be tempted to use these technologies to do their thinking for them. Vendors need to highlight the dangers of doing so. If users do not think a task through before handing it to an LLM, they will not be able to evaluate its output. Thinking task through includes mapping out the steps and content (where relevant), and having an idea of what a reasonable output should look like.

Avoid anthropomorphising LLMs: Marketing will often attribute agency to LLMs by saying things such as “the AI is thinking” or “it thinks you are asking for…”. Such language suggests that LLMs can think or reason as humans do, and biases users towards attributing agency to these tools.

Highlight potential dangers of use in enterprise settings: Vendors spend a lot of time assuring corporate customers that their organisational data will be held securely. However, exposing organisational data (such as data on corporate OneDrive folders) even within the confines of the corporate network can open the possibility of employees being able to query information that they should not have access. Moreover, formulating such queries is super simple because they can be asked in plain English. Vendors claim that this is not an issue if file permissions are implemented properly in the organisation. However, in my experience, people always tend to overshare files within their organisations. Another danger is that the technology opens the possibility of spying on employees. For example, a manager who wants to know what an employee is up to can ask the LLM about which documents an employee has been working on.

Granted, highlighting the above might make some corporate customers wary of rushing in to implement LLM technologies within their organisations. However, I would argue that this is a good thing for vendors in the long run, as it demonstrates a commitment to implementing AI ethically.

–x–

It is appropriate to end this piece by making a final point via another historical note.

The breakthrough that led to the development LLMs was first reported in a highly cited 2017 paper entitled. “Attention is all you need”. The paper describes an architecture (called transformer) that enables neural networks to accurately learn the multiple contexts in which words occur in a large volume of text. If the volume of text is large enough – say a representative chunk of the internet – then a big enough neural network with billions of nodes can be trained to encode the entire vocabulary of the English language in all possible contexts.

The authors’ choice of the “attention” metaphor is inspired because it suggests that the network “learns to attend to” what is important. In the context of humans, however, the word “attention” means much more than just attending to what is important. It also refers to the deep sense of engagement with what we are attending to. The machines we use should help us deepen that engagement, not reduce (let alone eliminate) it. And therein lies the ethical challenge for AI vendors.

–x–x–