In this episode, we talked to Jill Walker Rettberg, Professor of Digital Culture at the University of Bergen in Norway. In this wide-ranging conversation, we talk about machine vision's origins in polished volcanic glass, whether or not we'll actually have self-driving cars, and that famous photo-shopped Mother's Day Photo released by Kate Middleton in March, 2024.
Jill Walker Rettberg is Professor of Digital Culture and Co-Director of the Center for Digital Narrative (CDN), a Norwegian Center of Research Excellence that has received a €15 million grant from the Norwegian Research Council (2023-2033). She is also Principal Investigator of the ERC project Machine Vision in Everyday Life: Playful Interactions with Visual Technologies in Digital Art, Games, Narratives and Social Media (2018-2024), and of the ERC Advanced grant project AI Stories: Narrative Archetypes for Artificial Intelligence (2024-2029).
Reading (and Listening!) List:
Machine Vision: How Algorithms are Changing the Way We See the World (Polity Press, 2023).
Browne, Simone. Dark Matters : On the Surveillance of Blackness. 2015. (on sousveillance)
Danielle Wong; Sleepy Asians. Representations 1 November 2024; 168 (1): 202–210. doi: https://doi.org/10.1525/rep.2024.168.13.202
Linda Kronman: https://radical-openness.org/en/programm/2024/disputing-biases-artistic-tactics-hack-ai-powered-vision
Jake Elwes: https://www.jakeelwes.com/project-zizi-2019.html
Transcript:
KERRY:
Hi, I'm Dr. Kerry McInerney. Dr. Eleanor Drage and I are the hosts of the Good Robot podcast. Join us as we ask the experts, what is good technology? Is it even possible? And how can feminism help us work towards it? If you want to learn more about today's topic, head over to our website, www.thegoodrobot.co.uk, where we've got a full transcript of the episode and a sample. Especially curated reading list by every guest. We love hearing from listeners, so feel free to tweet or email us. And we'd also so appreciate you leaving us a review on the podcast app, but until then sit back, relax, and enjoy the episode.
ELEANOR:
In this episode, we talked to Jill Walker Rettberg, Professor of Digital Culture at the University of Bergen in Norway. We talk about machine vision's origins in polished volcanic glass, whether or not we'll actually have self-driving cars, and that famous photo-shopped Mother's Day photo released by Kate Middleton in March, 2024. We hope you enjoy the show.
KERRY:
Thank you so much for joining us today. We've really been looking forward to talking to you. So just to kick us off, could you tell us a little bit about who you are, what you do, and what's brought you to thinking about feminism, gender, and technology?
JILL:
Yeah, I'm a professor of digital culture at the Center for Digital Narrative at the University of Bergen. And I started off in the 90s excited about the internet and hypertext and came into blogging as a PhD student and really got interested in how people connect through these new media and how they, Shape us in different ways.
My original background is in comparative literature, actually, but that sort of broadened into media studies, digital culture studies, and so forth. So for the last five years, I've been running an project on machine vision looking at different kinds of technologies.
Feminism has always been something I've been really interested in. And I think it's just become more and more integrated into my work the more I go on.
ELEANOR:
All right. We are the good robot. So can you tell us what is good technology? Is it even possible? And how can feminist ideas help us get there?
JILL:
This is such a big and important and impossible question, I've been fretting over how to answer it.
I think it's much easier to define bad technology, and I think, I'm coming out of the humanities and I'm really well trained at critiquing things so it's super easy to say that's bad, but what's good is much harder and ultimately I think more important, which is one reason I really love your perspective in The Good Robot.
I think it's obviously it's important that this technology is not exploiting people. It's trying to make something better. But to be completely honest, I think most people making technology they're not trying to be evil. They may be not thinking through all the consequences.
For me as an individual, I want technology that I can be in control of, that I know what it's doing. Like I just signed up for yet another app. And it's got all those um, where, where your data's going, but it doesn't quite really tell you, right? Is it selling it? What is it doing? What does it giggle there?
So technology that I understand that I'm able to control in some ways is important to me. And I think that's also a feminist I think that's a, an important part of feminism to me is this, that we care for each other and we try to be fair and inclusive and so forth.
ELEANOR:
Can you give us an example of a technology you think is good? Or good in a particular circumstance.
JILL:
Oh, this is so difficult, because there's hardly a technology I could give an example of that's not also got the evil side. That's true.
ELEANOR:
That's totally fine. I think, it's important that things can be seen for all their complexity.
JILL:
Gotta say, I think social media and blogging and the internet are amazing.
I love it. I absolutely, obviously there's all sorts of problems you get with the data harvesting and the misuse and people like the spread of misinformation and fear and anxiety. But I think the basic thing that we can actually communicate with each other and not least that. We can get hold of all this digitized information and we can actually find out, we can actually reach the source that wasn't possible, even 30 years ago, it was much, much harder.
Yeah, absolutely. I like the internet.
ELEANOR:
Yeah, I like that. It's always fun to, to see, how hypertext began and take those, the utopian ideas and how pleasurable and fun that was to explore that, and bring that into our present era. You, one of the things you specialize in is machine vision, and these are technologies of looking.
So can you tell us about the technologies and the institutions through which machines see, in inverted commas?
JILL:
Oh, In my book, I go way, way back to 8,000 year old mirrors that are made out of polished obsidian, which is, doesn't sound like technology or very related to, smartphones, but one of the things I am fascinated by is how we use the technology socially.
And if you look at this very old mirror, perhaps the first mirror, Or at least the first, the oldest that we know of. It fits in your hand. It's carved a half sphere. So that, they've got the sphere against the palm of your hand, and then there's this polished circle that you can see your face in.
And that's almost just the same way as we see, look at ourselves in a phone, and that's the other end of this machine vision. So how you look at yourself in the phone, you take a selfie, et cetera. And this sort of intimate relationship between yourself and the technology really fascinates me.
But machine vision is I'm choosing a really broad understanding of what that is, really any sort of technology that you use to see things with or augment how you see, so everything from drones to AI, like facial recognition algorithms or emotion recognition algorithms or image generation or a telescope or microscope or x ray photograph, photography, I like thinking broadly about this.
KERRY:
I really like that you're challenging and problematizing what we might conventionally think of as machine vision, because a lot of people, I have a very distinct idea of what machine vision is. And it's always, whenever you're trying to go into a website for me, I'm probably trying to get like a recipe or something.
And it's click everything with a traffic light. And you're like, I don't want to do this. I just want my like hummus recipe or whatever you're trying to make. I think when people hear about machine vision, it's often within these very narrow parameters. So we'd love to hear a little bit more about what are some of the unexpected or surprising ways that people are using machine vision, for example, in digital art or in games.
JILL
Yeah, I, so with the, one of the great things about having had the project on this was that I had PhD students who were researching how machine vision is used in video games and digital art and stories online and stuff. And Linda Kronman, who's just finished a PhD, brilliant work. Did a lot of studies of how it's used in digital art.
And she found so many examples of digital artists hacking machine vision. So for instance, one of the earlier examples would be those baby monitors with video feeds and stuff. A lot of them were probably still are set to this or default password, right? So artists have gone around the streets of Berlin with these little wifi scanners logging onto people's baby cameras, which of course is creepy, but also certainly a very sort of visceral way of demonstrating some problems with the technology. So that's one example of how, artists hack this system in order to display it. And then there's so many examples of artists doing generative.
What we now call image generation, but doing it before it was easy. And really exploring the possibilities, these sort of dreamy images of what could be done. Which I think are a really important way of pushing the boundaries and, the showing it, showing what could be possible.
And then often the technology follows. Linda Kronman talks about how digital art, in a sense, is a bit like auditing. You know how people researching AI bias often audit the algorithms? Linda Cronman talks about digital art as doing that, but in a more sort of aesthetic, explorative way, which I think is a great idea.
Jake Elwes was querying the data set, sorry, is another great example of this Just pushing at the boundaries, like what if we changed it a bit? What happens when when you challenge the sort of assumptions that are in the machine learning.
KERRY:
That's really fascinating.
And I love this idea of artists providing a different method of auditing systems, because we often think of audits as being very dry. Okay. So for full clarification, my mom is a tax accountant, she loves an audit, and I do not inherit this gene, I do not. And actually one of our previous podcast guests and friends, Os Keyes, has a really fantastic co authored paper where they also challenge this idea of the audit as this very dry, routine, inaffective process.
And they talk about the emotional experience of auditing this really awful data set, which profiled and used videos of trans people going through hormone replacement therapy, and then became a data set that was used to train all manner of dangerous and spurious gender recognition tools. And so we'll actually link that paper in the reading list with this episode.
But to hear about this aesthetic approach to auditing, as well as the effective approach is really interesting and different. But something you also mentioned was artists doing gen AI before gen AI. And so obviously now our understanding of what machine vision is, has been really changed by the advent of public facing generative AI tools, but we'd love to hear from you, an expert.
How is gen AI changing machine vision?
JILL:
It's certainly, oh, I was going to say democratizing it, but it's bringing it to everybody anyway.
ELEANOR:
Not everyone, you know. Oh, you're right. for the record, sorry we have to be like boring critical theorists.
JILL:
No, you're right. You're right. You're bringing, it's bringing it to a lot more people.
It's making it a lot easier. And actually one thing we saw with the digital art is There's less of it in digital art now, or at least in the sort of, avant garde art. Now that it's become easier, it's not as challenging. It's not pushing the boundaries anymore. But what it's doing to machine vision in general when I started the problem, the project, One of my sort of hypotheses, my sort of idea was that machine vision and just the, deep fakes and all this would actually stop us, lead us to believing images left less because the photography theorist, Susan Sontag said in the 1970s that people treat images as if they're like bits of reality.
Almost like we think of data today, like that same critique of data could be made. Like we, we assume data is a bit of reality, but actually, of course, a photo, a photograph or a piece of data or data is always constructed, right? To some extent. And so I thought that as people get more and more used to us being able to use filters and alter things, and generative AI you'd think would just increase that even more, we would trust images even less. And to some extent, yes, people are skeptical, like you see, like, when Kate Middleton shared the Mother's Day photograph, people are fairly fast to say Actually, there's some issues there.
So yes, in that sense, it increases the sort of data literacy or criticality, but the other thing that I've seen is that people seem to want to trust machine vision more. And I'm thinking of things like surveillance cameras and there's a section in My book that I published, which has got one of the chapters there is looking at how they, the debate about installing surveillance cameras that do license plate recognition or number plate recognition in a Chicago suburb, Oak Park.
And just the way that people have this real sort of hope or sort of desire for the technology to come and rescue us from. all the problems of the world. So increased crime or a sense of increased crime at least, right? Technology is what's going to save us. And that's the opposite from like us getting more critical.
So it's, it's like, yes, on the one hand, we're more skeptical about wooden images, but then on the other hand, we want, we think these technological images can rescue us and they have the truth. So it's this paradox.
ELEANOR:
When the Kate Middleton image came out, I talked to my mom about it because I went home for a bit and she was like, yeah, everyone does some photoshopping now and again.
And I was like, what? And she didn't, she thought that like Kate Middleton just got rid of too many wrinkles and in the process got rid of like a wrist or something. And I was like, Oh no, I don't think that's why people are upset. So it was interesting that actually, people were still, depending on age and demographic and, AI literacy, interpreting things in different ways.
And my mum had grasped that there was something wrong with the image, but she totally didn't understand the reality of what was going on still.
JILL:
There's always a lot of context, isn't there? Yeah.
ELEANOR:
Yeah. I really am interested in the doubt and ambiguity that happens when you're looking at the world and you as a machine vision tool making a decision about what it is that you're looking at.
And actually it gets it wrong.
Or it fails to predict something or it's got this partial vision of the world, which is normal of any model.
A model is a reduced vector space, a simplification of reality. So can you tell us a bit more about the doubt that an ambiguity that arises when machine vision is looking at something and may not get it quite right.
JILL:
Yeah so the example I always think of is actually the self driving cars and when they don't work, and the problem is when you're doing that kind of statistical recognition of things or, cause basically, These languages model all the kinds of AI that are being used for image recognition and generation are basically it's probabilistic sort of statistical model.
So what's most likely here and with statistics, if you've got to generalize or else it's not going to do anything, right? If you acknowledge that there are lots of details and lots of individual cases, that's called overfitting. And and you'll end up with a model that can't predict anything.
So you've got to, you've got to generalize for it to work. But then of course there are so many situations where that generalizing won't work. So you've got things like that was it in 2018, one of the first self driving cars that killed a woman, and she was crossing, a pedestrian crossing, and no she wasn't crossing a pedestrian crossing, that's the point, the self driving cars algorithms were trained that if there's a pedestrian crossing and there's a moving object it's probably a human, stop. But moving object that has a bike and looks a bit unusual and is crossing and there's no pedestrian crossing, that's just a blip. It sorts it out because it doesn't fit into the general model and so it won't see it.
So that's like this kind of lack of context or acknowledging that specificity that can be a real problem. And I think that's a fundamental problem with these AI models that like, as a qualitative scientist, it's I'm not sure it's going to work if you scale it up, because I think it's still going to have those issues.
ELEANOR:
Yeah. Someone in our center, Maya Ganesh, who's her PhD was on self driving cars. She doesn't think that they're going to be a thing either. And I wonder whether actually they're more likely to happen in Germany where you're not supposed to jaywalk and people literally hiss at you out of the windows of cars.
Rather than me. I'm just like a very much a continental crosser. I just go for it.
KERRY:
Eleanor's from London and my experience having lived in London now for years, Londoners don't fear death or anything. Like I was walking to work, to the train station to get to work this morning and I was watching a crow eat the corpse of a rat and I was like, good morning, London.
ELEANOR:
It's not me.
KERRY:
Yeah, no, I saw Eleanor on the way, no, but no seriously, Londoners, they just walk and I don't have that energy. But no, I think this question of. The way that these machine learning models really struggle with any kind of ambiguity or they have to sort into like very static categories that don't reflect the complexity of the world is really interesting because I think we tend to think of technologies like machine vision in particular, because it sounds very futuristic and it sounds very complex as really high tech and invulnerable.
And as your work shows, they can often be A very low tech and B they can be very vulnerable and not just vulnerable in the like very classic cyber security sense of like vulnerable to outside attack and quote marks, but just vulnerable to any kind of disruption or anything that doesn't fit within this very narrow model of the world that they're based on.
And actually to come back to that point on low tech I was also just really interested in the various points you were raising around things like an obsidian rock polished being a kind of surveillance tech. So I think often when we're having these broader political conversations about machine vision and surveillance, we're thinking about the CCTV camera.
We're thinking about the body camera or these like very heavily carceral, militarized kinds of machine vision. But I do think just the advent of something as simple as this, I'm holding up my cell phone for those who aren't watching on YouTube, has become now this like mini device of both surveillance, being able to watch people in exact power over them, and surveillance, being able to speak back to power by filming.
ELEANOR:
Surveillance is S O U S, which in French means under So it's surveillance from below,
KERRY:
Yes. I really can't spell out loud.
ELEANOR:
In sousveillance, who is watching who?
KERRY:
So I think a good example of surveillance would be, for example, if you're a person of color in that you with and you are worried about how you're gonna be treated in like a customer interaction or police.
You might pull out your phone and start filming to show that you weren't doing anything wrong. So I think that's quite a good example of what surveillance might look like. I think low tech on the ground surveillance. With the ways we see these kinds of technologies being used in harmful ways. I think an example of this that I find just particularly weird and disturbing that I was reading about this morning, my background's like Asian American Studies, Asian Diaspora Studies was there was like this big internet trend in like the early 2010s where people would just like film Asian people sleeping, particularly in libraries, or on the train. And it was a very strange kind of Tumblr community that was mainly played off for laughs, but was this very weird racialized kind of surveillance. And I think, what those particular Tumblr and internet communities speak to is the fact that surveillance can, I think, be quite powerful and quite disturbing, even when it's something as simple as just like the general, I would assume, usually white bystander taking a photo of someone having a little nap in the library.
JILL:
Yeah, that's deeply creepy.
KERRY:
Danielle Wong does a lot of research on this. And whenever Danielle presents this research at like Asian American studies conferences, Danielle was like, yeah, usually everyone's just like they don't have any academic questions because they're too busy being like, oh, that's so weird.
Why are you doing that? But I digress. So this was the, really the longest ever way Of saying, speaking of narratives and storytelling you just shared some very exciting news with us. And so to close out this podcast episode, we would love to hear a little bit more about your new research project, which is going to be looking, I believe at AI and narratives.
So could you tell us more about it?
JILL:
I would love to tell you more about it because I just found out that I got another ERC grant. So this is a, an ERC advanced grant about AI narratives. My, my basic hypothesis is that the AI language models, ChatGB3, et cetera ChatGPT, et cetera, are not just spotting the sort of representational bias, oh, doctors are usually men, and prisoners are usually black, and terrorists are Muslim, et cetera, which we're aware of, and which is deeply problematic, but there is a lot of research on.
I think that Gen AI is Also spotting these deeper narrative kind of structures in all the training data because so much of this training data is actually narrative. It's trained on self published novels and regular novels, but also on things like Reddit posts or, am I an arsehole forums where people describe, narrate situations and stuff, right?
We know that, narratology is this this study, it's a bit 20th century, some of it gets quite structuralist but it's looking at how narratives are designed, things like, oh, a hero goes on a voyage, or there's a conflict, and then there's a balance afterwards, a resolution, or, if someone If someone forbids something, it's bound to have, that contract is bound to be broken.
So like Blackbeard says, or Bluebeard or whoever says, don't enter the seventh door, no matter what you do, then obviously someone's going to enter the seventh door. So there are some things with generative AI that suggest that the models are actually following some of those really quite deep structures.
So what I'm thinking is that this is kind of a deep level AI bias that we haven't really started looking at. And that could also be quite culturally specific. So if AI models are mostly at the moment trained on American data, then what does that mean if there are, if other cultures actually have different kinds of stories that might be being, might be endangered if all the narratives that AI is producing follow particular models.
So I'm really excited to start working on that.
KERRY:
Oh my goodness, that sounds incredibly exciting. And I often think about this in terms of the kinds of texts and stories. I think about this on a very surface level, to be clear, not on a multi year research project. I'm incredibly excited to hear about this, but I often think about this, like when models are trained on things like fan fiction or internet fiction and how the structure of those pieces of literature must be so different to things like a classical novel that was published like a hundred years ago or so.
And I wonder how that will change those kinds of stories that they tell.
JILL:
Yeah, fanfiction's a great example. The fanfiction community's been upset at everything. In fact, they're trying to deliberately get certain tropes into the next round of training data, which is going to be very funny, this sort of reciprocal sort of war that's going on there.
ELEANOR:
Maybe we can attach some fanfic in the podcast reading list. Partly because I if I had a hobby, which I don't, that's what I would start with. Just love, love the fanfic people.
JILL:
We should totally do that.
ELEANOR:
Thank you. It was so nice to talk to you today, Jill. Thank you so much for coming on.
JILL:
Thank you so much for having me. It's been a pleasure.
ELEANOR:
This episode was made possible thanks to the generosity of Christina Gore and the McArtle Foundation. It was produced by Eleanor Drage and Kerry McInerney and edited by Eleanor Drage.
Image Credit: Oleksandra Mukhachova & The Bigger Picture / Better Images of AI / Snapcat / CC-BY 4.0
Comments