Amy Barrett Hello and welcome to the Science Focus Podcast. I'm Amy Barrett, editorial assistant at BBC Science Focus magazine. Many of us have had a one-to-one interaction with artificial intelligence, whether that's through an automated chat service for customer service or trying our hand at beating an eye built to play chess. But these experiences aren't flawless and they're not as smooth as our interactions with other human beings. One researcher trying to improve the language abilities of AI is Lara Martin, a computing innovation fellow and a postdoc at the University of Pennsylvania. More specifically, Lara is trying to teach AI to tell stories. I'm really pleased to be joined by her today. And I believe you have an AI generated story to share with us.
Lara Martin Yes, I have a story here and I think what's good about the story is it kind of shows the the strengths and weaknesses of the system that I built for my PhD. So it goes like this. Christopher attacks Harriet. The king tries to take him away. Christopher is taken away, but is quickly saved by Eigner. The king is able to escape the tower. Christopher and Christopher are able to get Christopher out of the tower where the tower is still on the king. Penelope and the king are able to save Christopher.
LM So what's interesting about this story is that it's starting to make causal and coherent sense until it reaches a point where it just spits out garbage. So, the way that I've been doing my work is I'm trying to combine neural network output. So, like, GPT, there's these like really large language models that people use for a lot of natural language generation work. And I combine that with a more classic AI, symbolic methods. And so here I am trying to make the the story, I'm fighting with GPT to make it more coherent, but there's still there's still work to be done. With the Christopher and Christopher sentence, my parser just breaks. It's a work in progress, but that's what research is.
AB Yeah, absolutely, we should really be impressed by hearing that story, shouldn't we? Because it's one thing to hear it with the frame of reference of all these amazing stories that come out of humans. But this is kind of really novel, isn't it?
LM Right. So what's interesting about having an AI tell stories is that if you think about what you're feeding an AI or... I don't know how familiar everyone is with neural networks, but neural networks are our way of having the the computer generalise over a large amount of data. So they learn to pick up these patterns over time and the way people do do this, so like I mentioned, GPT. GPT is where they took this massive amount of data, what what we didn't think was possible a couple of years back, and just crammed it into one neural network and the neural network just learnt patterns of this data. And so I take that and I tweak it a bit so that it learns to tell, like I have a corpus of science fiction stories. So I actually kind of retrained it. Or they call fine fine tuning it to tell science fiction stories. But going back to what you said, what's interesting is that now the AI has this great idea of what a story is, but that doesn't give it any idea of what it's like in the real world. So there's this thing called the principle of minimal departure by Marie-Laure Ryan. And I love this because it really sums up this idea that when you tell a fictional story, you try to match it as close as possible to the real world. So we know even when you're listening to a story, you make assumptions that it's going to be as close as possible to the real world. So I don't have to reintroduce gravity every time I tell a news story. That's going to be an assumption you make. I think in the book she talks about how Don Quixote doesn't have to talk about how to use currency and waste time with with that, because these are just cultural or physical assumptions that we make. And so therefore, they're not in the stories. And that makes that makes training these neural networks really difficult because the neural networks aren't able to understand what is and what isn't needed for a story. Well, it's that. So they end up learning kind of temporal relations, but that can mean anything. It could mean like I have to load the gun before I shoot it. Or it could mean, like Susan jumped up and down and then Evan flew to Paris. Like, maybe that's something that happens. Maybe. But so it's learning what sequences happen across time. But that doesn't mean that it's cause and effect. And so what I do is I kind of inject that into these systems to guide the generation and have it make more sense.
AB So we know that AI are capable of playing specific games that we've told it to or following kind of instructions. But why is it important for AI to be able to tell stories and convincing stories?
LM Yeah, so. Well, I mean, I kind of like to think of a world where imagine you have this instead of using like a keyboard and mouse, maybe your interface is talking to the computer like kind of like Alexa. But imagine having like a full blown conversation with her or saying, like, you could tell a story like, OK, so I need to plan a birthday party. Well, in my birthday party, I'm going to have these wonderful balloons and then all the guests will arrive. And that. So you're telling this. It's like all almost all conversations and all types of talking that we do is actually kind of telling a story. So I think it would really improve our interface or interfacing with the computer if we were able to tell stories with it or add it.
AB So it needs to be able to both understand what we mean when we tell a story and also tell a story back.
LM Yeah, so, you know, telling a story back is another step, though. So when I do story generation, it is doing story understanding and then, well, it's kind of bulk story understanding. And then it generates its own story and being able to generate its own stories is just like, OK, did you understand me? Are you on the same page as me? Will you plan this birthday party the way I asked you to?
LM I think, like in the closer future, I think it would be really great to see systems like that that train you in certain scenarios or if you have writer's block and it's actually a lot of people are working on this where maybe you stop for a while and the system gives you prompts and how how relevant are those prompts and does the user use them and how do they use them? Things like that. And so there's different, um, there's different ways that stories can be used that people don't realise, partly because we are telling stories all the time.
AB So it is kind of fundamental to what it is to be kind of human within our relationships.
LM Yeah, I often start out my my talks with like where we've learnt to tell stories before we could write. I mean, this is part of our human nature and it's not easy. For a computer to learn how to tell stories or learn how to understand stories, it needs to know all this other basic information as well. Like that's why a lot of natural language processing, which includes like linguistic information, is necessary in order to build up systems like this.
AB So are there certain rules that go along with certain stories? Are there kind of basic patterns that you can teach the AI?
LM Well, there's a couple of ways to teach the AI certain patterns, some of which is like going back to what I said about inserting symbolic methods which meaning that. So when I say symbolic, I mean like feeding into the system these discrete tags. So instead of saying 'dog' as this like string of numbers, I'm just using the word dog and this is what I'm feeding into the computer and the computer does not understand this, but it can operate on it because I gave it these rules to operate on it.
LM So if you see dog, then get a treat. So so there's been work by Nanyun Peng, who is now at UCLA, and she has looked at having these keywords. So a user enters in a sequential keywords like eat, leave, drive or something, and the system will create a story like using these keywords and having, like each sentence contain the words in order. So it becomes like 'Charlie ate his breakfast and he left this house and drove to work'. And so that work is actually done mostly just with neural networks and there's stuff on the more symbolic side where you can have a lot of control over the types of stories that you're generating because the these systems are basically hand engineered. And so you're instead of having just this data that who knows what the model is learning from it, you're actually just creating these nodes there, these plot points of the story. And then you have another system that will plan paths across these plot points to create a story. And that is that type of symbolic story. Generation has been going on since at least the 70s. This neural techniques are more more modern. And yeah, I mean, not a lot of people have worked on the story generation until about 2018 or so.
AB So it's a research in its infancy.
LM What's interesting is that, like you said, there's these systems used to have a lot of control. We used to have a lot of control over them. So we knew what types of stories they're making. And now we're diving into this other end where who knows what, who, maybe we don't even know what data it's feeding it. And this is actually a big problem that is across all of artificial intelligence where we have this huge amount of data. And it could be like racist, sexist, like all kinds of garbage in this data that we just don't know is there, because it's so huge and we're just training on it. And so we end up having these systems that it's like gambling, trying to see what what it outputs.
AB So what is the data come from?
LM It comes from a variety of sources, so I have personally I have scraped, for my science fiction dataset, I scraped from fan wikis because science fiction TV show nerds are very thorough with their plot summaries. And so that was a great source of data. But people have been like scraping all of the internet basically to create these data sets. And the internet can be a pretty toxic place.
AB Yeah, gosh, it's scary to think that the fate of artificial intelligence could potentially lie in the hands of the people on the internet.
LM Yeah, there was actually a pretty big disaster that Microsoft had a few years back where they had a Twitter bot learn from the interactions that she had on Twitter and she quickly became a Nazi and it was not good. So, yeah, and they took that down very quickly. But it made a ripple and things like this keep popping up. And I even though I don't research bias in data in particular, I like to be informed about it and try to be careful about what data I'm using.
AB Hmm. So there is a responsibility on your end?
LM Absolutely. And I think all researchers should have this or do have this responsibility, whether or not they work towards making things better.
AB So for the system that you've built, what would you say is kind of the ultimate test for this kind of thing? Is there something that's like the Turing test that you can do?
LM OK, yeah. So yeah, I think that Dungeons and Dragons would be a really great test. I wouldn't call it a Turing test. But it would really show how open ended and flexible and useful these types of systems can be, because you just think about the types of things that you do when you play these games. It's pretty remarkable. Like you're not just telling a story, but you're telling a story with like three or four other people. And if you're the Dungeon Master, it might even be a harder task. I think it is kind of a harder task because you're creating a whole world and you have to relay this information on to other people to make to allow them to create a theory of this fictional world that you're trying to share. And so there's a lot of theory of mind going on, so trying to see are you all on the same page? Do I understand what you think the world is like? There's a lot of intrinsic reward. So unlike playing Go, for instance, which has an extrinsic reward, which says that, like, if you do this, I actually don't know how to play. So if you if you do this and this, you get points for that and stuff. Maybe I'll stick to the Atari games. So like, if you shoot all the aliens in Invader game, then you get points for each alien that you kill and then you have this like extrinsic reward that you get. And that's very clear. Like, you can't debate that as if you have an intrinsic reward. It's like, well, maybe my character will get a reward for or feel good about taking care of this orphan that we run across. And that's not something that you can necessarily build into the system. It it's something that's both kind of personal and also personal to the character.
LM Yeah, and there's just like a ton of these challenges that we haven't even begun to look at. It's pretty, pretty remarkable.
AB Do you think it's something that, you know, you and I could see in our lifetimes, an actual AI Dungeon Master?
LM Well, I mean, there is the, uh, AI dungeon, but to actually see something that acts like a human... And there's like different levels of being a dungeon master, maybe someone's a good dungeon master and someone a poor dungeon master. So to see a good Dungeon Master AI in our lifetime, I think, um. I don't know, I'm I'm a little sceptical that'll happen.
AB Yeah, and can AI ever be creative with what you've given it, or is it just going to kind of rehash the things that you've told it? Can it actually come up with new ideas?
LM So, yeah, computational creativity is a really fascinating field in general because there's just so many kind of philosophical questions that we don't know how to answer. Like if it creates something that's brand new. So there's this idea of there's different types of creativity, there's personal creativity. So maybe a two-year-old drops something on the ground, then they discover that their cup reacts to gravity in this way. So that's like personally creative, but historically creative is a lot more complicated because you have to compete basically with the rest of the population.
LM And so if an AI comes up with something that's like historically creative or at least like. New and interesting, and then it brings up the questions like, well, was that the AI's work or was that the work of the the developer that created it or the research group that created it?
LM Like every agent that you make, you're putting your imprint on it, whether you like it or not. The more on the rule-based symbolic side you head towards, the more of your imprint is into this agent.
LM To say that, like, can an agent come up with something that's creative by itself is a tricky question. I mean, if it's like purely by itself, quote unquote, it might just be like it did something completely random and in that case, it wasn't intentional, you know, and is that creative, then? There's just a lot of sort of legal questions here that are. Yeah, just open ended. And I'm not a philosopher, so.
AB And of course, there's the ethics side of it as well. Like, there's a kind of a lot this is really a new route for exploration. And what's the kind of most surprising thing that is thrown up for you?
LM There's been some pretty interesting things that these systems have come up with over the years. Like the one of my early systems. I had a story about like a horse becoming a lawn chair entrepreneur, and I just think that that concept is really, really interesting and I think about that every now and then. And so there's these really like quirky, funny things that they come up with, but. Like that system, for instance, that was random, like I had well, it was partially random, like I had these tags that said, like, OK, I need an agent or some kind of job maybe and something.
LM So it filled in. But then it was pretty early on, so I just had it fill things in randomly and it just happened to come up with this interesting concept. But most of the other times it would come up with these weird things like I don't even know what, like a really specific type of flower or something like because it would just pull this from the database it had. It's not as surprising when you can't understand it.
AB Yeah. But I can definitely see kind of an AI Terry Pratchett style thing happening in the future with a horse who sells lawn chairs. Amazing. Do you think that this kind of I mean, we're talking way into the future. So this is probably kind of speculative fiction right now. But will, you know, art and storytelling be the next big frontier that this is an AI boom? Are we going to see AI books pop up on the shelves or are we going to see an Oscar-winning AI in the future?
LM Well, I, I think it would be great to see creative AI being made. Having like an Oscar-winning AI, I think is not as great as this, but I think that it goes into that problem. Like who who has the ownership of this? Is it the AI? Like, did the AI win the award or was it the people that created it or was it the the data that it was trained on, the people who created that data? Like, there's a lot of weirdness there. And the the computer is not a living thing. And I think that it's really important for people to to realise that computers are not as smart as they think they are. Because they're not people, they don't have agency, they're just tools that other people have used to work on these things. And so I think the best use of computational creativity or creative AI is creativity and using it as a tool to augment human creativity, because computers are really good at looking through large, large spaces of data so they can come up with things that you've never seen before and never thought of connected with this. But humans are really good at making those connections, connecting ideas that that the computer might present to them, so like going back to the example of the horse being the lawn chair entrepreneur, like the computer knows nothing about what that means is just spitting out stuff, but having a human take that and run with it, maybe they go and make a story about this horse. That would be fantastic. This is a really interesting idea. And humans have that ability to connect these things, and I think that I think that's a really good symbiotic relationship that needs to be used more.
AB Yeah, I guess I've never really thought of it that way. The kind of tools and aids to our own creativity as opposed to having their own agency. For you was kind of the most exciting thing about your research? Is it where it could go or some particular use or...?
LM It's interesting because I am very proud of the work that I have done during my PhD, but it is not the work that I wanted to do when I started my PhD. So I started I came into the programme wanting to add speech and like spoken technologies to something cool. And I came across Mark Riddell's work on storytelling and I thought that would be really, really great. And then I realised, oh, this is not ready for speech.
LM And well, maybe it's more like Mark told me that this we're not quite ready for speech yet. But now that I'm done, it kind of opens the door to all these different possibilities of what can I do with just computational creativity and both like language understanding and speech technologies? Because in speech, there's this thing called prosody, which is basically all of the extra information that you convey to someone in addition to the words. So it's like the intonation and the the like, the melody, the timing.
LM And and a lot of times this has meaning behind it because people talk in particular where you can you can sound sarcastic and you don't pick that up in text. I think those are really fascinating and I think that there can be a lot of work there, too. I mean, there is a lot of work going on in speech technology. But, yeah, I think that all these things are really interesting. And I'm very, very curious to see where my career takes me and that sort of thing.
AB You can see kind of the benefit of people doing this research from a kind of AI perspective will actually help us. As humans, like for people who maybe struggle with understanding intonations.
LM Yeah, I tend to I consider myself a cognitive scientist. And so that includes like artificial intelligence and linguistics because I did linguistics in my bachelor's degree in addition to computer science. And I kind of combine the two and then and there's other things in cognitive science like philosophy and psychology. And it's a very humanistic way of thinking of science. I think that by looking at AI through a cognitive science lens really puts more of the focus on the human. And so, yeah, you can make more and more tools that either act like a human or you use what you learn about humans to make better AI, but to work with humans.
That was Lara Martin from the University of Pennsylvania talking about her research teaching AI to tell stories. If you've enjoyed listening to this episode of the Science Focus Podcast, please do leave us a review and check out the February issue of the Science Focus magazine, which is out now. In this issue, we explore how your brain creates reality. We look into the baffling science of dark boson stars, and as always, a panel of experts answer your questions. Of course, there's much more inside and on sciencefocus.com.
Let us know what you think of the episode with a review or a comment wherever you listen to your podcasts.
Listen to more episodes of theScience Focus Podcast:
- Rana el Kaliouby: What if computers could read our emotions?
- Aleks Krotoski: What happens to your data when you die?
- Robert Elliott Smith: Are algorithms inherently biased?
- Pete Etchells: Are video games good for us?
- Jamie Susskind: How technology is changing politics
- Jim Al-Khalili: Why AI is not the enemy