Transcript of Sketches — Stories Are the Answer

Listen to this episode · 10 min

---

Please enjoy this transcript. Here's an episode in memory of Patrick Winston which opens the new Sketches series with a short piece on story understanding with artificial intelligence and my experience attending Winston's 6.034 lectures at MIT. "Don't just tell me it's a school bus. Tell me why you think it's a school bus." Transcripts may contain typos. You can find the episode notes here.

---

0:00Nono Martínez Alonso: Early morning on December 20, 2016, I found my way into a huge sports field at MIT plagued with evenly-spaced tables ready for an exam. Nervous, as if I were back to school, I was the first one to get there. Our professor would get there a bit later—that was Patrick Winston.

0:19 Hey. It's Nono. And this is The Getting Simple Podcast.

0:26 Welcome to The Getting Simple Podcast.

0:48 A year ago, on May 1, 2019, I started a personal challenge. I would sketch at least one thing every single day. And it's now with me—a new identity. I'm not someone who sketches but someone who sketches every day. And, except for January 25, 2020, I've sketched for the last 365 days, I decided not only to sketch but to start writing short stories and publishing them online every Tuesday. The first story went out on July 2, 2019. And today's the first time I'm telling you one of those stories in a podcast, with my voice.

1:23 All right, let's get back to the story.

1:26 Three months before that exam, on September 7, 2016, I would attend what was the first of a series of lectures of Winston's introductory course to artificial intelligence—6.034—and would sit in the first row of Huntington Hall, colloquially known as "Ten Two Fifty." Located right below the Great Dome of MIT.

1:46 Some days, I'd arrive and get a chance to talk to Patrick for a bit before class. What's the most dangerous power tool you've ever used? He asked me one day. Silence. I didn't know what to answer. I thought an architect will have used power tools. He followed. I was pleased to see he knew my name just a couple weeks into the course. In retrospect, I find most of my tools these days being virtual pieces of software.

2:11 In one of those classes, as if it were aligned from Jonathan Nolan and Lisa Joy's Westworld series, Patrick emphasized the relevance of the following question. Can you explain why you think so?

2:22Patrick Winston: So the "may rules." If I'm angry you, you may kill me. Thank God, we don't always kill people who anger us. But we humans always are searching for explanations. So if you kill me, and I've previously angered you, and you can't think of any other reason for why the killing took place, and the anger is the explanation.

2:40 Sometimes we use abduction. You might have a firm belief that anybody who kills somebody is crazy. You're presuming the antecedent from the presence of a consequence. So those are kinds of rules that work in the background to deal with a story.

2:53Nono Martínez Alonso: To Winston, whether a machine is able to answer questions of the type of why and how it reached a question in a humanlike way was as important, or even more, as the conclusion or the answer itself.

3:06Patrick Winston: No. Don't just tell me it's a school bus. Tell me why you think it's a school bus.

3:10Nono Martínez Alonso: "Genesis supports the steps towards story understanding," reads the headline of his draft paper with Dylan Holmes, titled, The Genesis Manifesto: Story Understanding and Human Intelligence as of December 13, 2016. And here's a quote from that paper. "To understand what makes humans uniquely intelligent, we build computational models of how humans tell and understand stories."

3:36Patrick Winston: So over the years, what we've done without intending it or expecting it or realizing it is that we have duplicated in Genesis the kinds of thinking that Marvin Minsky talks a lot about and his most recent book, The Emotion Machine. He likes to talk in terms of multiplicities. We have multiple ways of thinking, we have multiple representations, and those kinds of reasoning occur on multiple levels from instinctive reactions at the bottom to self-conscious reflection on the top.

4:00Nono Martínez Alonso: A system like Genesis is meant to be on top of all other technologies and make the system self-conscious. Genesis can understand stories, answer questions, and—unlike other narrow artificial intelligence systems—reason and explain why it reaches its conclusions.

4:15Patrick Winston: And so.. The instantaneous response is, "We don't know why Lu kills Shan." What Genesis does at this point is, when asked the question, "Did Lu kill Shan because America is individualistic?" He goes into his own memory, and said, "I am modeling an Asian reader. I believe that America is individualistic. I will insert that into the story. I will examine the consequences of that insertion, and then see what happens." And this is what happens. The question is asked and inserts into the story. Boom, boom, boom. And now Lu kills Shan is connected all the way back to America is individualistic. And so the machine can say yes.

4:53 But that's not the interesting part. The interesting part is this. It describes to itself what it's doing in its own language—which it treats as a story of its own behavior—and now has the capacity to introspect into what it itself is doing.

5:08 I think that's pretty cool.

5:10 You can say that this system is aware of its own behavior. By the way, this is one of the things that Turing addressed in his original paper, one of the arguments against AI was the disabilities argument. And people were saying, "Computers can never do these kinds of things," one of which has to be the subject of its own thought. But Genesis is now reading insde the story of its own behavior and being the subject of its own thought.

5:31Nono Martínez Alonso: Winston shared a fascinating (yet worrying) idea in class. He said, If you don't know how a program gets to a conclusion, you can't trust it. It's not possible to debug it. What I think is that, as a matter of fact, we rarely know how machines work but we still give away or trust for their convenience.

5:50 Three years ago, on April 20, 2017, I met with Patrick to ask for his feedback on the project I was working on at the time—Suggestive Drawing. He tested one of my first working prototypes, a drawing app running on an iPad with an Apple Pencil.

6:06 Patrick sketched two daisy-looking flowers which, obviously, you cannot see. But you can find at gettingsimple.com.

6:13Patrick Winston: So you're getting an image of a daisy and your input is the sketch?

6:17Nono Martínez Alonso: A few seconds later, the system returned a prediction for each of those flower line sketches using a generative machine learning model that only knew about daisies.

6:26 For the tech savvy I was running Pix2Pix, a generative adversarial network (or GAN) that had been trained to learn a mapping from line sketches of flowers to daisy flower photo textures.

6:36Patrick Winston: That's pretty cool.

6:37Nono Martínez Alonso: "That's pretty cool," Patrick said. One of the most interesting parts of the project was when a user tries to misuse the machine. So you might use a model that is trained to render flowers to draw a house or to draw something else.

6:53Patrick Winston: Sometimes you get a new idea because you misunderstood somebody else's idea.

6:57 You know, I sometimes tell a story about trying to explain an idea to Marvin Minsky. Half way through your explanation he guesses the idea and has a better idea.

7:06Nono Martínez Alonso: We discussed the project for half an hour and I left his office at Stata center.

7:11 That was the last time I saw him.

7:13 Sadly, Patrick passed away on July 20, 2019. His Memorial, held in October 2019, surfaces the fact that Patrick influenced many people's lives in profound positive ways, not only as a teacher or mentor, but as someone who loved sharing the experience he acquired over years of teaching.

7:31 I've never had the chance to interview him for the podcast, but would have loved hearing more about his worldview. Luckily, he contributed a great amount with numerous online lectures, talks and learning resources.

7:43 And if there is a sentence that Patrick said that will stick with me for the rest of my life, that's Stories are the answer.

7:50Patrick Winston: And the first thing I do when I get to San Diego is I go to the zoo, and I look at the orangutans. And I asked myself, "How come I'm out here and they're in there? How come you're not all covered with orange hair instead of hardly any hair at all?" Well, my answer to that is.. is that we can tell stories and they can't.

8:11Nono Martínez Alonso: As we've heard, he will emphasize that, as humans, we're driven by causality. If we see two data points and don't know how to connect them, we put our brain to work and try to figure out what happened, making up a story. And, as opposed to Elon Musk's point of view, "with artificial intelligence we are summoning the demon," Patrick saw artificial intelligence as essential to the survival of our species.

8:36Patrick Winston: And one last thing, I think it was Cato the Elder who said, "Carthago delenda est," every speech to the Roman senate ended with "Carthage must be destroyed." It could be talking about the sewer system and the final words would be "Carthage must be destroyed." Well, here's my analog to "Carthage must be destroyed." I think this is so because I think well, many people consider artificial intelligence products to be dangerous, I think understanding our own intelligence is essential to the survival of the species. We really do need to understand ourselves better.

9:06Nono Martínez Alonso: Patrick Winston was the Ford Professor of Artificial Intelligence and Computer Science at the Massachusetts Institute of Technology. I'd highly invite you to watch his Hello World, Hello MIT talk of 2019 to learn more about his worldview and his contributions, and to watch his 6.034 lectures online.

9:25 On my sketches blog, this story was shared along with a sketch of Killian Court—that's where you can see the Great Dome of MIT from Charles River. The day I sketched this view was the day I met Pier Gustafson for the first time, an artist whom you can get to know on episode 20. He showed up biking across Killian Court, right in front of the building that was named after MIT's 10th president, James Rhyne Killian Jr. I often passed through these location when running along the Charles River.

9:56 I think it's funny that I had this sketch on the back-burner for a while now and by chance I decided to prepare you for Tuesday—exactly three years after the last time I met with Patrick.

10:07 Thanks so much for listening. We'll see you next time.

10:10 If you enjoyed the show, remember that you can find a detailed list of episode notes, transcripts and previous episodes at gettingsimple.com.

10:18 You can read the original story, both in English and Spanish, at Sketch.Nono.MA.

10:24 This episode is provided under a Creative Commons license and uses audio from Patrick Winston's Story Understanding lecture in the Brain, Minds and Machines summer course of 2015, available in MIT OpenCourseWare. You can find links in the episode notes.

April 30, 2020

---