How Algorithms See Us—and How We Should Look At Them: Q&A with Wearn Lecturer Jon Kleinberg

In today’s world, “the algorithm” is shorthand for a mysterious, impossibly complex system that only computer scientists can understand.  Naturally, that invites suspicion. How can algorithms be made more transparent? Should they be more comprehensible?

The word ‘algorithm’ can sound a little more mysterious than it is. At the most basic level, an algorithm is just a step-by-step process. A recipe is an algorithm.

We've always had algorithms. But a first big change came when computers could automate the algorithms that we've been doing by hand. They could do things we knew how to do at much bigger scales, much more quickly, much more cheaply. The algorithm was simply an automation of something that we had all been taught to do at school.

A second big change happened with the growth of machine learning and artificial intelligence. They started doing tasks that even as humans, we don't really know how we do them. For example, we don't really know how it is that we look at a picture and recognize what's in it.

But image recognition is crucial for something like autonomous vehicles. An autonomous vehicle needs to look out and say, “Where are the cars ahead of me? Where are the pedestrians ahead of me?” And so you might tell the computer to look for a kind of boxy thing on little wheels.

That was the original approach to doing this. But it didn’t really work.

What you can do, however, is show a computer billions of images and say, all of these have a car and all of these don't. And the computer figures out its own rules for how to do it. That's basically the premise of machine learning. We write an algorithm that doesn't actually solve the task. Instead, we write an algorithm that lets the computer figure out a solution.

It’s just trial and error on a massive scale but the rules the computer makes might be very complicated, sort of incomprehensible to us.

So, we truly don’t understand what the algorithm is doing?

Well, it's not unknowable. You can figure out what the computer is thinking at some point or how the algorithm is working. It just might be extremely difficult and time-consuming. It might take you a year of effort or multiple years of effort to actually trace a single decision this thing makes because it might involve millions or billions of arithmetic computations.

It's knowable in some very indirect sense, but it's not really knowable in any practical way.

That means that the question quickly becomes: Do we need to give up some level of comprehensibility to achieve this level of performance? Are we willing to do that? What kind of algorithms do we want to be deploying when they're making decisions that affect people?

A year or so ago, “ChatGPT” would not have meant anything to most of the world. Now, it’s an everyday subject – and a major disruption in higher education. How are you thinking about AI assistants in the classroom?

One thought experiment I encourage people to do is: If you were getting this assistance from a human friend, would this seem acceptable?

This is going to be a boundary-drawing exercise for what forms of assistance we find acceptable.

And our thinking is likely to evolve, as we get more experience with these changes. For those of us who went to school in the 1970s, eventually we came to accept that calculators are just going to be a part of math classes. The norms changed. We realized we're going to have to teach math classes in a way that accepts the omnipresence of calculators. We're going to be facing a similar set of questions as this new technology becomes standard.

Fifteen or 20 years ago, you were at the forefront of research that used computational methods for analyzing and understanding large social networks, work that foretold the emergence of social media and its power. And now, you’re working on ensuring fairness of algorithms—which seems to address some of the central technological questions of this moment. Do you have a sense of what we might be talking about 10 years from now?

These things are very hard to anticipate. The increasing power of AI is certainly going to be with us for a while. We're going to be seeing certain tasks that we thought of as inherently human start to get more algorithmic assistance than we were expecting.

As scientists, we're going to start having to ask the question, what’s the boundary between what people do and what algorithms do?

If an increasing number of our scientific theories are made with computer or AI systems and they're not comprehensible to us as human scientists, do we still think of that as science the way we always have? Or do we think of this as some other activity?

We might have very predictive models of human physiology, of climate, of new chemical compounds and drugs. And we might be arriving at these by means that are not really comprehensible to us as people. The outcome seems effective, but we have to decide what to do with models like this.

How do we reason with that? How do we know when we can rely on that? That's going to be one of the big challenges coming out of all this.

Jon Kleinberg is the recipient of MacArthur, Packard, and Sloan Foundation fellowships. Other honors include the Nevanlinna Prize, the Lanchester Prize, and the ACM-Infosys Foundation Award in the Computing Sciences. He is also a member of the National Academy of Sciences, the National Academy of Engineering, and the American Academy of Arts and Sciences, and serves on the Computer and Information Science and Engineering (CISE) Advisory Committee of the National Science Foundation, and the Computer Science and Telecommunications Board (CSTB) of the National Research Council.