Episode transcript: Should we trust AI as a creative collaborator

DAVID ELLIMAN 

Hello and welcome to Tech Tomorrow. I'm David Elliman, chief of Software Engineering at Zühlke. Each episode we tackle a big question to help you make sense of the fast-changing world of emerging tech. 

Today I'm joined by Anjana Susarla, Professor of Responsible AI at the Eli Broad College of Business at Michigan State University. Our question today: Can we trust AI as a creative collaborator? 

ANJANA SUSARLA 

So, one of the things that we learned in our research is when we are doing something creative, whether it's writing an essay or creating a piece of code, it's an iterative process, it doesn't happen in one stage. So, can we study this process by which we have a set of people? Can we experimentally almost sort of randomize this? 

Give them suggestions, and then do they act on the suggestions? Do they not act on the suggestions? And then what happens to the next step and what do we conclude from that? Well, they're kind of almost two different effects. One effect is people sort of trust everything that AI is telling them to do. 

It's kind of called automation bias, but there is a countervailing effect too. Which is, you know, algorithm aversion. We sort of don't like something because it's coming from the AI and I think that's the balance that is very important in understanding where do we go with AI.  

DAVID ELLIMAN 

So, there's almost like the effect on us using something that is helping us, you could argue, with our creativity and formulating a dependency, but there's also, I think you kind of alluded to, there's a sort of a trust element there. 

ANJANA SUSARLA 

Yes. 

DAVID ELLIMAN 

Because as you said, you know, you create something, it's an iterative process. And yet, we have to therefore trust that process. We have to trust the steps. 

However many steps are in that process, we have to look at the outcome and make a judgment in it. And there's all this debate about how close humans are to that judgment point. And a lot of discussion about pushing responsibility into some sort of iterative process and letting it run on its own. 

There's a lot of agentic discussion where we're giving personas and profiles to individual bits of software that control an LLM or are guided by an LLM, and they're allowed to run. And the question is, when do we step in? When do we step out and when do we let it run? And when do we not?  

ANJANA SUSARLA 

One of the things I would also say is that we as people, some of us are almost predisposed, we have a familiarity with AI tools. And some of us are less familiar with AI tools. And there's this interesting dynamic where when we see a piece of art, do we suspect, is this created by AI or is it created by human beings?  

DAVID ELLIMAN 

And that causes quite a visceral response in people, doesn't it?  

ANJANA SUSARLA 

Yes, exactly. Exactly.  

DAVID ELLIMAN 

I'm interested in your thoughts in the sort of the dynamic, this trust element. So, we are working as a creative partner with a tool, so therefore we must place some sort of trust in it. Yet, if you talk to people more generally, they will say ‘Oh, I'm not sure how much I trust the AI’. So, there's this holding back and saying, maybe I don't trust it and I need to be the person that verifies, which sounds sensible. 

ANJANA SUSARLA 

Yes. 

DAVID ELLIMAN 

And yet they will interact with it freely and maybe even give it information freely. So, there's the sort of a lack of knowledge about how to trust it.  

ANJANA SUSARLA 

Yes, and even how much do we like the outputs of AI? If there's some favorite movies or favorite pieces of art that's kind of a part of our lives, we use it to sort of craft our narratives, as it were. 

Are we going to use all this AI generated content the same way? You know, there's almost a derogatory term nowadays. It's called AI slop.  

DAVID ELLIMAN 

Hmm. Exactly.  

ANJANA SUSARLA 

It's so easy to create stuff. It's almost eroding the premium we place on creativity when anything is machine generated. I think there are all these questions, and that goes back to trust in AI. It should not be seen as ‘I trust it, I don't trust it’. It's almost like so many different elements to that trust, like you said. Do we audit it? Do we verify it? Is there an external benchmark or norm that I can compare it with? And there's of course this kind of two-way interaction between AI and humans, as it were. 

Or how do we sort of measure that teamwork, that results from that AI-human collaboration? 

DAVID ELLIMAN 

I think it's really tough to try and associate a qualitative metric like creativity with AI or indeed any generated material. If we take the example of music or even literature, there's so much derivative work out there. Music's a great example because we can look at maybe even music from the last sort of 70 years. 

You look at Rock and Roll that's come from Blues and then rhythm and blues and Soul, and then you know, various forms of Rock, and you can see a lineage of sort of family trees spreading out. Now what's happened is that we've taken all of that material and we've put it in a big pot that we've called the training corpus of an LLM, and we've said ‘generate some music’. Or ‘write a novel or a novella’ and it can do it. So, does that have value? Well, that's in the eye of the beholder. So, it very much comes down to how I think the receiver views the quality of the work as opposed to some judgment about whether it's right or wrong. If it moves you and if it's important, then you know it's up to the person looking at it or hearing it. 

And it's not just new creative work, we need to think about, it's how AI represents existing work too.  

ANJANA SUSARLA 

I will give, you know, one small like anecdote, the works that I'm studying on AI creativity. I had posted a working paper on a, a repository and then what happened was a reporter from Scientific American, she was doing this article for Notebook LM, which is a Google tool, which generates podcasts. 

So, she put my paper through the notebook LM, and created this podcast, and she interviewed me, and she said, here, by the way, here is this podcast generated from your study and she said, can you tell me if this is accurately representing your study? I said, I don't know the answer to that because here is Notebook LM. 

It's this gigantic AI prediction machine, as it were, and it's taking my paper and it's kind of situating my work within a whole body of knowledge. When I wrote my paper, I read some papers and my collaborators and I discussed and we created something, a body of work. I don't know all those connections to all that other stuff. 

I don't know if notebook LM, is it representing these fields of thought in an accurate manner? Is it making up these inferences? Or maybe those are legitimate, you know, at one level we can audit stuff at a technical level by imposing some sort of a retrieval, augmented generation, or there's some technical approaches so we can have maybe a layer of verification. 

But beyond that, who is to say that when we are using AI for create work, maybe there is a tradition that people are trained in Eastern type of, you know, ways of painting like Japanese art. Or people trained in a different kind of tradition. And if AI is synthesizing these elements, we do not know if this is actually happening in a correct manner. 

So, I think this is just one way I would say is when we are talking about auditing AI, there is a technical component, but there's also a social technical component. And that could be qualitative. It could be based on some interpretivist kind of approach or constructive. Or it can also be like there is a long tradition and scholarship in legal fields and other areas. 

DAVID ELLIMAN 

It's funny, isn't it? We look at the origin of the training of the models and where it's got its data from, how it's iterated across it to generate its weights and maybe it would be incredibly complicated, but maybe you can draw a path, like a golden thread from the sources to the decisions that are made by the LLM, which are effectively probabilistic next word generations, for text at least anyway, and then you say, well, actually, is this, bringing everything together that maybe generates an idea, which you could label as creativity. Is that any different from studying a body of work and being influenced by it? As a writer, as a human writer? As a musician, you know, I know that there are so many influences that, you know, I can play something and you know that there's things that are similar. 

So, we could be really specific and look at the musical harmony elements about where it's come from and find similarities and a pathway through that. But we could also say this sounds a bit like that. There's so many similarities, but like we seem to sort of allow that without it necessarily getting too close, but it's gonna be quite hard I think, when we say that an AI has collaborated with us and has given us an idea when it's effectively just an amalgam of all of the things that it's trained and brought forward. It's gonna be very difficult for us to sort of try and apply that same, is this plagiarism or is this influence?  

ANJANA SUSARLA 

I think there are also two issues when I think about it, which is one we can risk like homogenizing, you know, everyone is depending on the same AIs to generate stuff and there may be an element of homogeneity there where we sort of lose all our individual tastes and expression. 

And I think that's, you think about the second part is it's not only these ar. AI created works are also being curated through algorithms and there is, you know, a flattening effect there as well because we are creating this work so that it will be picked up by another algorithm. Spotify recommendation filter or YouTube or TikTok or whichever it is. 

So, it's kind of AI meets AI and where is the personalization or creativity where you just go and you know, you're talking to a friend and the person says, oh, I think you should listen to this album or read this book. And so where is that element of real human curated experiences, that authenticity, and are we going to pay that premium? 

DAVID ELLIMAN 

Do you think we understand enough about personalization and in this case, creativity to be able to even question it because, you go to a supermarket and you buy something, people have been looking at products for years and placing things next to them, you know, the, the classic halo effect of like trying to place products in places where people are more likely to say, I'll shop for this and maybe I'll need that. 

And they're not necessarily the same kind of things, but they lay out their shops in order to try and attract you to buy more stuff, and it's almost like we've tried to understand the nature of recommendations, similarity, understanding people's likes, wants, and needs fundamentally, then we're asking for an algorithm to replicate something that we actually don't really understand ourselves anyway. 

ANJANA SUSARLA

Yeah, and I think what we should also keep in mind is that the algorithms, there's a feedback loop because when something is being picked up by YouTube or Spotify or TikTok as a trending, this is a trending video or trending post, then more people will watch it and it's going to end up, there's a reinforcing cycle, so the algorithmic filters are kind of generating this, the popularity, the skewness in popularity as well. 

DAVID ELLIMAN 

So, in order to understand how the nature of prediction works, AI systems are often set up and used as recommendation engine as an example. How do we know what somebody's gonna buy? So, you know, if we replicate the halo effects with recommendation engines, AI and machine learning is used to try and suggest things to people in a similar fashion. 

If other people have bought this, maybe you might wanna consider that as well, because you may have bought something similar to other people. Now, the reason why this works, it's not just as simple as this person bought this product and that product, so therefore we're gonna suggest that product to you as well. 

This is relationships that form over multiple dimensions. So, the products may be dependent on things like at the time of the week, the geographical placement, your buying habits, because you may do certain things at certain times, so it tries to store a lot of information about you or your persona. So, with all of your buying habits considered together, then it might be able to suggest things to you that it thinks is helpful. 

Now, some people like that, for example, you might see that when you log into a music streaming platform and you get suggestions for other things. Some people like that whole idea. Some people really rail against it because they think it's somewhat intrusive. So, this his idea of recommendation is something that's been in machine learning now for about 10 or 15 years, and we find that it can have fairly polarizing effects with people. 

A lot of people don't like it, in fact. 

So, there was a piece of your work that I found fascinating. You talked about, I guess, the relationship between, as you said, the flattening, and you could also argue that the flattening might mean leveling and democratization of certain levels of information. And then you also talked just before about the homogeneity that might result because everyone's working from the same data sets from the common models that are being used that themselves probably use an overlapping set of data sets. And in the past, you've talked about things like the nature of academia might change and that it actually might help people with the academic process but itself, you know, if people aren't aware, the academic process, there's an article or a paper that's gonna be written and in simple terms it’s gonna be targeted for a particular publication and it goes through a peer review process and that's anonymous. And then you go through the couple of iterations of that. So, it, it's had a fair bit of scrutiny that you have your authors describing the methodology and the outcomes that they're publishing and then obviously you're saying that as a creative partner, AI could help with that process, but it's kind of interesting to sort of see where in that same way that people listening to this might sort of think, oh yeah, well I could use Chat GPT because I can ask it questions. But now we're stepping up a bit and we're saying we now need to go to a rigorous environment where we expect the AI collaborating partner to be rigorous with us. 

So, it might help us to do some of the things, but how much of that sort of rigorous oversight do you think that AI will ever kind of replace in us?  

ANJANA SUSARLA 

That is a fantastic question, and I think that's very needed in today's world. So, one of the things that you know, we should talk about essentially, is some of the newer AI tools you can ask them to show their thinking process, as it were, right? So, the important thing for us is to make sure that we preserve our thinking ability in the face of, you know, the AI generated tools everywhere. My own children actually have complained that there's sometimes teachers will tell them, oh, use AI to write this report, use AI to generate, and they're not happy about it. 

There's this kind of anxiety that children or college students will be left behind if they don't use AI.  

DAVID ELLIMAN 

I think that there is a tendency for people to anthropomorphize the AI because of the style in which they're asking and answering questions, and I'm drawn to the emergent properties that might occur and not view it as magic. 

Interestingly, your paper on the Janus Effect quoting a Roman God of two faces. That's kind of interesting, isn't it? Because that sort of brings into focus the nature of our trust and our relationship with AI, that on the one hand we may be a bit too trusting, and on the other hand, we kind of don't trust it and we don't really understand what our relationship with it is. 

ANJANA SUSARLA 

Yes. And I think this is where we are. Suddenly we have all these almost magical tools as they were, and we have to worry about a lot of different things. Is there, you know, if you're using large language models, all these AI tools to maybe write reports or create scientific works, or there's some biases in these AI, most of the training data for these models is coming from Europe and North America. 

So, what about are we representing the rest of the world in a fair manner? There's copyright, there's trustworthiness, there's auditability. There is also the security risks. If you are a lawyer and you are putting stuff into Chat GPT, are you violating some of the confidentiality and you know, is there a sort of a leakage of privileged confidential information that is now being used to train these AI outputs? It's like we've just opened Pandora's Box and you know, we can't put it back. Right? But then how do we deal with it?  

DAVID ELLIMAN 

Yeah, that's the thing, isn't it again, that relationship between trust. You know, almost like a hope and a peril. You know, you allude to your Janus effect. 

Maybe the more people anthropomorphize it and the more desperate they feel, the more willing they are to forego their trust and ask a question. Maybe it's about a diagnosis and maybe the answer's cogent, but we don't actually know that it's right. I mean, obviously, something that we've talked a lot about in the public discourse is hallucination. 

And hallucination happens in different ways depending on the LLM, the size of data within the context window and so forth. But you don't know what you are being told ultimately.  

ANJANA SUSARLA 

I also would say that, you know, in addition to some of these hallucination type problems, I think it's just important for us to think also about the auditability of AI outputs. 

It's something that we need best practices. So, you know, I actually had some work in IT consulting. So, if you are writing a piece of code using AI, is that enough? Somebody has to also maintain that software later on. Right. So, think of all these AI generated creative outputs and what happens to them. The second life, or think of a system’s life cycle, and so who is maintaining all these AI outputs? 

It's one thing if you are writing a book, but if you're using it for a more critical purpose where the reliability and the repeatability and the interoperability, all these are important. Then I think that requires a, it's also an engineering problem. Like I said, it's a managerial problem. It's a social technical problem. 

DAVID ELLIMAN 

So, when an AI tool enters the workplace, we often see a number of things happen pretty quickly. There's people that immediately get excited by it. And I think that excitement falls into different categories. You get the practitioner excitement because people think, oh wow, this is great. Look at how quickly I can do something. 

And then you get the kind of, maybe the more cynical response to that with like, look at all the things that it can't do. So, within each conversation you get a sort of a dichotomy. So, the hyperbole just keeps growing. So, within a workplace, what happens is that you get people that have tried out and will test it and if they're not careful, they can either get met with unreal expectations because everyone's got too excited. Can this do X, Y, Z for us? Can I now reduce all my costs because I can reduce the staff? And the answer is kind of. No, not really, and certainly not to begin with. It could be the case, like any new tool that is optimizing that there might be changes over time, but I think that there is this expectation of immediacy and speed that people see within AI. It's gonna do this now, it's gonna reach general intelligence now or next. And there's no actual evidence of that. And there's no actual evidence that any of these tools will supplant people from their work, for example, within a short space of time. 

So, I think that the problem that people will have is these unreal expectations get set and then people will develop a proof of concept. And then the proof of concept either gets enough excitement to get another proof of concept. So, you've got a workforce that has a certain level of skills, you've got tooling, you've got approaches, and whatever it is. 

And the world is not set up for that. What do people do when faced with something new? They can often, you know, resist and you get an inertia. 

Anjana has good reason to warn about the threat of brain drain. It's happened before. 

ANJANA SUSARLA 

When the industrial revolution occurred you know, some of the artisanal folks, they suddenly have to go work on mills. And so that is also kind of de-skilling. There are newer kinds of skills like the AI models. We need to train models by actually having folks with creative degrees, so maybe AI fine tuning, AI model training. 

There could be some new skills that emerge. Will this be an equilibrium state? I don't know. You know what happens to countries like in India where we had a lot of people move from villages or smaller towns to the metro areas, and they're working in these call centers, will call center jobs be replaced by AI? 

What happens to economic development? I saw an interesting experiment. I thought this is really fascinating, where there were a group of students who were given Chat GPT to help solve math problems. When they were using Chat GPT, their problem-solving ability increased. But then there was something really almost scary when we took away access to Chat GPT, their performance worsened than before. 

You know, I think it's important for us to think about all these processes like cognitive debt and de-skilling and using AI as a shallow crutch. That sort of minimizes the fact that the creative process involves extensive iteration and rewriting and reimagination and discussion. Are we sort of teaching younger students that all that doesn't matter? 

DAVID ELLIMAN 

So, if we return back to the key question of the episode, we've touched on the nature of creativity, the nature of collaboration, the nature of trust, the potential and the hyperbole associated with that, the risks, and having covered all of that, how do you feel? We've just been talking about some of the downsides, maybe in terms of the human cost, but I guess that probably brings you to the question, should we trust AI as a creative collaborator? 

ANJANA SUSARLA 

Yeah. I think AI is definitely helpful for me as a research assistant. I'm all for transparent use of AI, but I think that's the important thing. Just use it transparently, use it with some caution and be mindful of where is the data training data coming from? What sort of inferences? As long as there's some sort of auditability of outputs as well. 

That we are not ending up replicating a lot of bad stuff that's out there. 

DAVID ELLIMAN 

So, returning to the core question, can AI be seen as a creative collaborator? I think the answer, like everything, is nuanced. I think there are cases where you can look at a particular process or something you have to do, and AI is really gonna be a big help to you. And it is to me, you know, I use it every day, but there are times when you just can't use it. 

It can't be relied upon for certain things. And then maybe that human element is just gonna be of a greater premium over time. Because I think true creativity comes from people and all the AI is doing is just gathering up and representing, remixing things that people have done. Now. I think that we will see more of it. It will get better over time, and I think that we'll just adapt to it and it will become a tool that we use. 

Thanks for listening to Tech Tomorrow, brought to you by Zühlke. If you want to know more about what we do, you can find links to our website and more resources in this episode’s show notes. Until next time.