For over 40 years, people have been learning to speak the language of computers. But with the latest developments in AI, computers are learning to understand the language of humans. This leads to a significant shift in the way we interact with digital products and services. By entering structured data and following strict processes, we are heading towards a more natural, dialogue-based interaction that follows the individual needs and context.

The idea of bringing digital interaction closer to a human conversation has been around for a long time. However, chatbots have only been moderately successful so far, as their answers are often inaccurate and very limited in scope. There are several reasons for this: 

• Understanding natural language is complex
• Understanding the meaning of a question correctly is even more complex
• The answers are often based on a decision tree and tend to be very limited

As soon as you move away from the intended flow, conventional chatbots quickly lose their appeal as they start to give very generic answers. This is one of the main reasons why they’ve only been moderately successful so far.

When ChatGPT went live in November 2022, the astonishment and the subsequent hype were enormous. That’s because the text comprehension of the elaborately trained large language models (LLM) is way better than anything that has been achieved before. An LLM processes each word as a token and makes a prediction of which word is most likely to follow next. This creates a very natural dialogue, trained on the basis of billions of texts written by humans.

The exciting thing about large language models is that the basic understanding of language and context is already ingrained through pre-training which takes weeks and costs millions of CHF in cloud computing power.

Based on this generic pre-trained model, specific domain knowledge can now be very easily be embedded. This allows high-quality, specialised chatbots to be developed with little effort, even allowing for on-premise operation.

So far, interaction via speech or text has been predominant. However, newer models (like Google Gemini or GPT-4) can also process multimodal input, including images, sound, or videos. This provides the chatbot with much more context to reference in the dialogue. For example, I can ask the chatbot if this outfit is suitable for an occasion, or if I can eat the food on my plate with my allergy.

On the 13th May 2024, OpenAI has announced their new GPT-4o (omni) Digital Assistant. Just one day later Google announced Project Astra, essentially the same idea, a multi modal digital assistant that can see, hear, and understand human context and answer questions.

The ability of AI bots to hear, see, and understand humans opens up a variety of new use cases and has the potential to fundamentally change digital interaction.

By adding the ability to autonomously perform actions, a multimodal chatbot becomes a digital assistant capable of taking on and executing human tasks within a limited scope.

The fact that computers have learned to understand the context and language of humans leads to a paradigm shift in the design of digital interactions. Today, interactions must adhere to a strict process and clear form. In the future, we will experience much more natural, tolerant, and personalised interactions that are more relevant and of higher quality. This will not only increase efficiency and effectiveness but also the user experience.

Even though it will take some time for chatbots to become widespread, I recommend designing new products or features today as rich, personalised conversations. Because as digitisation progresses to this extent, we will not want more notifications, pop-ups, forms, and lists in our everyday lives, but fewer, more relevant, and higher quality conversations.