The Ethics of Algorithms

Josh Landy
Would you be willing to trust your life to an algorithm?

Ken Taylor
Well, aren’t computers less prone to bias than human beings?

Josh Landy
Do we really want to turn over our moral agency to software?

Ken Taylor
Welcome to Philosophy Talk, the program that questions everything…

Josh Landy
… except your intelligence. I’m Josh Landy.

Ken Taylor
And I’m Ken Taylor. We’re here at the studios of KALW San Francisco.

Josh Landy
Continuing conversations that begin at Philosophers Corner on the Stanford campus, where Ken teaches philosophy, and I direct the philosophy and literature initiative.

Ken Taylor
And today we’re thinking about the ethics of algorithms.

Josh Landy
Ethics of algorithms—isn’t that a contradiction in terms?

Ken Taylor
Why would you say that?

Josh Landy
Well, algorithms aren’t moral agents. And yet, in spite of that, we seem to be turning over morally fraught decisions over to them at an alarming rate.

Ken Taylor
Alarming, really? I mean, we can put computers in charge of air traffic control a long time ago, and it made air travel safer for everybody. There’s nothing alarming about that.

Josh Landy
Okay. But all that software had to do was keep the planes far enough apart, so they wouldn’t bump into each other and give me something more challenging.

Ken Taylor
Okay, take the new frontier of this stuff, self driving cars, I mean, all those crowded city streets with cars and pedestrians and cyclists traveling every which way, and in much closer proximity to each other than planes ever again. What about that example?

Josh Landy
Well, that’s a good example for me. I mean, do you really want computers to decide when your car should swerve to avoid a pedestrian may be killing you or your passenger?

Ken Taylor
Well, Josh, I gotta tell you, it doesn’t really matter what I want. It doesn’t really matter what you want. Because the day is coming, dude. And it’s coming fast.

Josh Landy
Too fast. If you ask me. I mean, look, computers are already in almost complete control of the stock market, they are gradually taking over medical diagnosis. And some people even want to turn sentencing decisions over to them now it’s out of control.

Ken Taylor
Uut of control, Josh? Come on. Why on earth? Would you trust a fallible human judge over a well programmed computer? I mean, have you seen the mess that judges have made of sentencing decisions?

Josh Landy
Well at least judges are living breathing human beings with a sense of duty and responsibility and empathy.

Ken Taylor
Yeah, and racial biases and hidden political agendas and overblown emotional reactions, just look with a computer, you just encode the sentencing guidelines that, you know, we decided to put into the law, encode those into your algorithm, and then you sit back and the computer will decide without fear or favoritism that’s vastly superior.

Josh Landy
Okay, but who’s going to program those computers to play judge? Well, human beings, of course, oh, the same human beings with those racial biases and hidden agendas?

Ken Taylor
Oh, I was talking about the judges. I wasn’t talking about the program.

Josh Landy
Oh, so So programmers are less biased. The judges all of a sudden, right? So they are they’re like Mr. Spock, all green blood, no emotion?

Ken Taylor
No, no, no, like, programmers, although some of them may seem like they’re not Vulcans. They’re humans do but but my point is that we can train them not to write their biases into their algorithms.

Josh Landy
But how? I mean, most of us aren’t even aware of our bias.

Ken Taylor
Well, even if we’re not at least we can test and debug and tweak the programs. Josh, you can’t rewrite a judge’s Neural Code, when you discover that the dude’s got this thing against black people. So look, give me the computer over the judge any day of the week.

Josh Landy
I just think it’s harder than you imagined to get our biases out of these algorithms. I mean, consider the the machine learning algorithms that are all the rage these days.

Ken Taylor
Yeah, they are all the rage. You got something against machine learning Java?

Josh Landy
You know, the problem is that you don’t any more program every single line of code, right like they used to in the old days. Instead, you basically give the computer a problem and let it figure out how to solve it on its own.

Ken Taylor
It’s a beautiful technique. It’s a really powerful technique, making all kinds of advances. What’s your problem?

Josh Landy
Now my problem is this. Let’s say you got a bunch of tech bros in Silicon Valley. And they pick their friends to be the training sample, you could be darn sure they won’t be representative of the population.

Ken Taylor
Look, I’ve got no brief for tech bro culture, but you just have to make sure to use the statistically sound methods to design your training sets for these things. We know what to do that we know we know how to do good statistics.

Josh Landy
Yeah, but it doesn’t end there. Ken, look think about it this way. machine learning algorithms are also often black boxes. They’re inscrutable yet yet, you can’t see what’s going on in there. And that means that if something goes wrong, you can’t get in there. Like you were saying we should tweak and debug, right? I mean, like I was no huge fan of old fashioned AI, but at least you knew what the software was doing.

Ken Taylor
So just let me get this straight. Okay. You really long for the days when humans were in charge of all these decisions. If you can’t have those days, you’ll take the good old fashioned AI programs, what 30 years ago or something, but the last thing you want to deal with Is the newfangled machine learning algorithms taking over? I gotta tell you something, Josh. You sound a little like a Luddite my friend.

Josh Landy
No, no, no, I think technology is wonderful. I I just not in a hurry to farm our moral agency out to machines.

Ken Taylor
Well look whether you want it or not whether I want it or not. It’s happening Josh courtrooms around the country for example, turning to computers for advice on who should stay behind bars while awaiting trial. And so we sent our Roving Philosophical Reporter Holly J. McDede, to see the process up close. She files this report.

Judge Ross Moody
That’ll be your next court date starting tomorrow morning

Holly McDede
Judge Ross Moody is inside a San Francisco courtroom, robed up for an afternoon of bail hearings. First to take the stand is a man charged with carrying a loaded gun in public. The defense attorney argues that the defendant has a stable job and always makes his court dates.

Defense Attorney
So I’m asking them to be given up to be released while this case is pending.

Holly McDede
The prosecutor argues the man only showed up because what other option did he have. The suspect was already in jail at the time. He says he went on to commit another crime once released.

Judge Ross Moody
And the fact that he was again walking with a loaded firearm with an extended magazine is shows us a serious threat to public safety.

Holly McDede
Judge Moody has three options: detain the defendant until trial, let the defendant go free until then, or ask that he pay a big chunk of cash for his freedom. But before judgment, he decides he looks at a recommendation from an algorithm. The one this court uses bases its recommendations on the case histories of more than 1.5 million people taking into account the person’s age, criminal history and record of showing up in court. The algorithm, also called a Public Safety Assessment, says let the man go.

Judge Ross Moody
This is one of those cases where I disagree very strongly with policy assessment. I think that safety threat is self evident. So I’m not going to follow the PSA today. Follow the recommendation the prosecutor set bail at $150,000 on a felony.

Holly McDede
So we’re not turning courtroom decisions over to algorithms just yet. San Francisco District Attorney George Gascon introduced this tool to the city’s courtrooms in 2016. He wanted to bring more fairness to money bail decisions.

George Gascon
One of the concerns that I have with money bail is it really creates two systems that creates a system for the people that can afford to post bail and the system for those that cannot.

Holly McDede
Even with help from a computer judges ultimately decide who to release before trial. A defendant shot at freedom depends on who is on the bench. According to research, female judges and Democrats are more lenient. Another study shows judges are more forgiving after snack breaks Glasgow and has faith in the consistency of algorithms.

George Gascon
The reality is that you have a system that is has developed a body of knowledge that is way superior to what a human being will ever be able to.

Holly McDede
But not everyone trust the machines. In July 2018 More than 100 civil rights groups signed a paper expressing concern the algorithm is biased San Francisco’s algorithm designed by a nonprofit called the Laura and John Arnold Foundation, ways of defendants criminal history. San Francisco Public Defender Chesa Boudin is worried about how the algorithm calculates risk.

Chesa Boudin
Someone who’s been stopped by the police numerous times because of the color of their skin, someone who’s pled guilty to previous crimes because they were too poor to bail out. In other words, the exact people that the Arnold Foundation seeks to protect by implementing this new tool are prejudiced because of the built in legacy of prejudice and racism in our criminal justice system.

Holly McDede
So do we trust the humans or the computers? Sharad Goel is a professor of engineering at Stanford who studies algorithms. He and his researchers found that risk assessment tools are more likely to be accurate at predicting risk than the people are. But he says humans can see nuances that computers can’t.

Sharad Goel
When a defendant comes to court, they expect that their particular individual story will be heard and a decision we made on the basis of that more broadly.

Holly McDede
Goel says that though algorithms can be helpful guide they can’t completely fix disparities in the criminal justice system.

Sharad Goel
My worry is that people will strictly use algorithms and not consider broader policy fixes. For example, eliminating money bail in entirety.

Holly McDede
States like New Jersey already have since eliminating the money bail system that states jail population has dropped dramatically. People in the bail industry like reality TV star dog, the bounty hunter have since sued the state over bail reform. In those cases the judge has ruled on New Jersey side but the judge is human. Next time maybe try asking a machine.

Dog Bounty Hunter
I just want my guy right now, I just want somebody that’s wanted.

Holly McDede
For Philosophy Talk, I’m Holly J. McDede.

Ken Taylor
Thanks, Holly for that guided tour the new frontier of algorithmic decision making in the courtroom. I’m Ken Taylor with me as my Stanford colleague, Joshua Landy. And today we’re asking about the ethics of algorithms.

Josh Landy
We’re joined now by Angèle Christin, Professor of Communication at Stanford University, and author of “Algorithms in Practice: Comparing Web Journalism and Criminal Justice.” Welcome to Philosophy Talk, Angèle.

Angèle Christin
Thank you.

Ken Taylor
So Angèle you’re in communications. And I understand you’re a sociologist by training. You’re not a computer scientist. So what first got you interested in this question about the ethics of algorithms?

Angèle Christin
I’ve been studying criminal justice for a long time. When I was a student in France, I did this long fieldwork in prisons and jails and criminal courts in the outskirts of Paris. And I’ve been doing a lot of interviews with judges and prosecutors asking them how they make decisions, right? I mean, it’s a big decision to send someone to jail. How can you be sure that you’re right. And what I realized at that point was that judges and prosecutors didn’t know. And they weren’t worried about it. So I was just like, I, you know, I’m not sure I’m making the right call. And so when I realized recently that algorithms were being developed to precisely help judges and prosecutors make these decisions. I decided to go look it look into it.

Josh Landy
That’s yeah, well, I mean, yeah, I agree. It’s a really serious issue. I’m curious about, you know, we’ve been, we were talking in the opening, Ken and I about the question whether bias can creep into algorithms. But you know, definitionally, an algorithm is just a step by step mechanical procedure for solving some kind of computational problem. So So what does even mean to say that an algorithm can be biased?

Angèle Christin
Yeah, in a way is a definition of algorithm. I mean, it’s a proper definition. But it’s a bit not helpful now, because, okay, it’s a procedure. But who creates a procedure? Well, it’s a human being, who decides on the step by step process that leads to an output? Well, it’s a human being, who decides on the output, it’s also human being? And who is affected by the algorithm in general?

Ken Taylor
Yeah, I mean, I think you’re right. I mean, I think there are kind of two notions of algorithms out there. There’s the one that you’ll find in a computer science textbook, right, which is just this step by step procedure. And, you know, it’s like long division is an algorithm, you know, if you, if you follow the rules, you’ll get the right answer. That’s what algorithms are, that’s in the mathematical computer science sense. But there’s another sense it’s like any kind of decision procedure that can be automated. And then some of these decisions that we automate, they don’t have definite answers, right? And we just turn over the making of an answer to the automation. That’s kind of tricky.

Angèle Christin
Absolutely. And I think that’s a big deal was algorithms, which is that in order to have an algorithm, you need to have a clear goal, right? Because an algorithm is always going to optimize for some kind of output. But in some cases, the output is complex. And people are not clear about what’s going on.

Josh Landy
Right, youthink about the bail case that Holly was talking about a moment ago, presumably, you could optimize either for maximum public safety where nobody gets bail. Or you could optimize for maximum opportunity to peep for people to their lives and things like that.

Ken Taylor
I think actually talking about this stuff in terms of algorithmic decision making, actually does a kind of disservice. Because it makes us think, well, there’s a mathematical function that determines who gets bail, and who doesn’t get bail, there isn’t a mathematical function that’s already there, that determines and you just apply this procedure and you compute that function. That’s a fiction, right?

Angèle Christin
And following up on what you just said, Josh, I mean, I think said, you know, really, so yeah, you could optimize for public safety, you could optimize for fairness, making sure that two different defendants with kind of similar characteristics are getting the same kind of sentence. And you could optimize for rehabilitation, just giving a chance to give it a shot led people to kind of re insert themselves into society. And these are three different goals that—

Ken Taylor
And that’s completely up to us. And we’ll have to, we’ll have to dig more deeply into this. You’re listening to Philosophy Talk. Today we’re asking about the morality and ethics of algorithms with Angèle Christin from Stanford University.

Josh Landy
Would you rather have a doctor or a computer diagnose your illness? Do you want software deciding who should go to jail and for how long would you trust your life to an algorithm?

Ken Taylor
Algorithms: to trust or not to trust—plus your calls and emails (if you’re not a bot), when Philosophy Talk continues.

Kraftwerk
Computer world

Ken Taylor
Crime travel, communications, entertainment—all managed for you by your friendly neighborhood algorithm. I’m Ken Taylor. This is Philosophy Talk, the program that questions everything…

Josh Landy
…except your intelligence. I’m Josh Landy and we’re thinking today about the morality of algorithms. Our guest is Angèle Christin from Stanford University.

Ken Taylor
So I want to take you back to the debate Josh and I were having at the beginning, I Josh clearly thinks it’s a recipe for moral disaster that we’re turning more and more morally fraught decisions over to computers and their their algorithms. I think it’s the wave of the future. I think it’s often a good thing, maybe mostly a good thing. Whose side are you on?

Angèle Christin
In the case of criminal justice, I think I’m on Josh’s. side. So for several reasons. But first, I think, you know, what you in the interview with Sharad grill, he was saying, you know, fixing on algorism kind of distract us from some of the bigger policy questions. And I think in the case of criminal justice, this is absolutely true.

Josh Landy
So you think in other areas like medicine, for example, it’s fine, and maybe even a good thing.

Ken Taylor
It’s not just maybe even a good thing. I gotta say, I don’t know what you’re gonna say, it’s definitely a good thing. Computers right now, although we haven’t turned medical diagnosis, completely over to them right now. Can out diagnose a doctor by a mile. And eventually, right now, there’s a lot of a computer aided diagnosis going on, eventually, it’s all gonna be it’s all gonna be computerized.

Angèle Christin
But again, that’s because the output is simple. Like you don’t want them it’s pretty easy. You want people to be healthy, you don’t want them to intake, you don’t want them to die. Everybody’s kind of on the same page, as far as the output is concerned was in the case of criminal justice, it’s much more complicated.

Josh Landy
I mean, there’s still some complications about how do you want to optimize for false false positives or false negatives? It’s not 100% Straightforward. Well write the value, but I agree with you at least the values are simple. The value is health.

Ken Taylor
And there’s a yes or no of it, do you have this disease or not? And you can just test it, whether who make who miss diagnosis more, the human physician misdiagnosis more, as we’re always medical, a lot of the medical malpractice comes from misdiagnosed that not all of it, mistreatment is some of it, but a misdiagnosis is a lot of it. And it’s just a fact, the machine assisted diagnosis is much better than just a plain human.

Angèle Christin
And you have a very clear test, as you were saying, well as in the case of criminal justice. I mean, it’s a question of false positives, false negatives, like, that’s just more complicated. Because I mean, you know, for example, when you’re giving risk off to someone, and then you send them to jail, and then you’re like, Oh, well, there is like, you know, engage in recidivism, therefore, we well, right to give them a high risk, or, well, yeah, but in the meantime, the person went to jail. And we know that people in jail are much more likely to be offended afterwards, because they created an ATM, in jail in a network of criminals, etc. Right? And so is that something that the algorithm doesn’t take into account.

Ken Taylor
And that thing that Holly said, that somebody said in Holly’s piece about the dataset is already infected? And it’s, I said to Josh, well, we’ll just we just do statistical sound methods. And we’ll get out the bias on the dataset, but in with a historical data set on the basis of which you’re bound to have to train these people, you can’t just invent a new data set. It’s already got all this built in history of segregation and discrimination.

Josh Landy
I mean, you know, look at the arrest rate for African Americans for drug offenses compared to Caucasians. I mean, it’s astronomically higher, and not because they’re offending more, right? So if you just set an algorithm out, to look to examine the world, as it’s currently constituted, it’s just gonna encode the biases of this society.

Angèle Christin
Absolutely. And that also, because arrest data, doesn’t equal crime, right, and the rest doesn’t equal the crimes, a lot of crimes out there, where people don’t get arrested. And vice versa, a lot of people get the rest of the differences. He didn’t commit a crime. And so here, you get, again, of social construction of data problem, which is a data sets never emerge in a vacuum. They are always dependent on a specific set of institution. In that case, it’s the police forces, for example. Right. Another problem, I think, and like that’s, you know, we talked about that a bit is also that some variables that are put into the data sets can also serve as proxies for race. And so in the case of San Francisco that you had in your, in your segment, you know, that women serve versus valuables. But for example, with Compass, which is one of the most widely used risk assessment tools in the countries, there are many variables about, like, for example, do you have friends who have a criminal record? Now, that kind of question, in fact, serve as a proxy for race.

Josh Landy
So even if you try to eliminate race from your data set, or at least from the way in which we’re analyzing the data set, it can creep back in.

Ken Taylor
So this is something that comes with these machine learning algorithms, which as Josh said, Are there rage, they’re kind of taking over this, this domain, in the old days, wasn’t really that long ago, in the old days of good old fashioned artificial intelligence and people used to call it where what you try to do is stuff a lot of human like knowledge into the thing in the algorithm through what was called knowledge representation. You would you would do this thing that I said you would think, okay, the law says these are the factors and these are the weights of the factors and you and you’d write every one line of code, you wouldn’t let the machine figure out on its own. What to weigh in how much? I mean, could if would, that just turned out to be really hard? Right? Stuffing the human like knowledge into the machine, I just turned out to be really, really hard. And then these deep learning things took over, right where the old fashioned approaches failed. But it sounds like maybe that old fashioned approach would be more acceptable in some ways, more morally, less morally problematic, because we would do it, we would do we, we know what we put in there. And we know what Wait, wait, what do you think about that?

Angèle Christin
I think that’s yeah, that’s a very good point, especially as far as accountability is concerned. So like, you know, in the case of criminal justice, basically, when you get to sentence like, you are able to appeal, right? And that’s a very big deal. Well, as in the case of algorithms, you’re not able to appeal, if you have a risk on your back, well, usually don’t know that you have a risk on your back, you don’t know what it is. And if it’s wrong, it’s really hard to get it fixed. So I think that’s a question of accountability in that case, and responsibility is very different from machines and humans.

Ken Taylor
When is it—one point about that and then we’ll get to some callers. But there’s one point about that. Here’s another thing about machine learning, we haven’t focused on them a lot. Because they set the weights themselves, they uses various propagate various algorithms for readjusting their weight and letter, their error signals. Some of them are really opaque, you have no idea how they’re representing the data set, what weights they’re assigning or why they give you an answer. But why they gave you that answer. It’s like, I don’t know. So here’s the thing I worry about when these if there are people working on this phone, well, here’s the thing I worry about when these things take over everything. And for example, that self driving car drives into a pedestrian and kills them and somebody wants to sue the company. And they want to try and diagnose the algorithm and find out why it did what it did.

Josh Landy
Nobody knows.

Angèle Christin
The companies usually don’t want to reveal the codes, right? Because they’re like, Listen, this is intellectual property. That’s our business. Why should we give you the codes that we use to categorize these cases right up to make this decision? So there is also I mean, you have different kinds of opacity. Really, you have one, which is that nobody knows what the algorithm is doing, because it’s machine learning. And so it’s evolving on its own, it’s really complicated, etc. But you also have kind of corporate secrecy.

Josh Landy
Right. So how does the defendant mount a defense? I mean, usually a defendant knows what the arguments are, right? The prosecution’s making, what happens now?

Angèle Christin
That’s a very good, that’s a very good question. So actually, I talked with lawyers with defense attorneys who told me that they are buying the algorithms themselves, in order to replicate the process through which defendants get categorized now that helps clients basically, assess their chances right before they go in front of the prosecutor and the judge. So there is that option.

Ken Taylor
So you’re listening to Philosophy Talk, we’re talking about the ethics of algorithms, we’d love to have you join this conversation. And Cheyenne—I think that’s how you say it—from Alameda is on the line. Welcome to Philosophy Talk, Cheyenne. Did I say that right?

Cheyenne
Thank you. Yeah, my name is Cheyenne. Yeah, well, I was gonna make a comment just about how imbued with racial bias the justice system is in any of the data we have from, you know, from beginning to end from the birth and the history of the racial justice system and the way it operates on a daily basis. But I’m, you know, a lot of those points are made less and waiting online. And I’m just wondering, I’m, I’m wondering about the racial bias inherent in using the racial justice system, and, you know, data on traffic accidents as to sort of reasonable examples to talk about algorithms, and that, that even a racial joke that even at the criminal justice system is part of the conversation is evidence of racial bias. This is a system that is so broken, and so born of racial bias, and the research is so abundantly clear that it would even be held up as an example.

Ken Taylor
Cheyenne, you’re making a good point. And I’m going to put it though to I’m going to put it to gel in a different way. Because the people who are going to hold this kind of thing, they say, well, right shines, right? But here’s a way to begin to overcome this system. Right? Because look, where does the racial bias come from? It comes from human beings and this and the other thing, and the thing I was saying to Josh, is, at least there is a prospect of us somehow getting this stuff out of the algorithms.

Angèle Christin
But no, that’s not possible. And I think Cheyenne is absolutely right. I mean, I think that using this data, which is fraught with racial prejudice, from the get go, I mean, how can you help solve a situation using the data that stems from that?

Ken Taylor
But where do you start though, where do you start to correct it, though? I mean, you have to start somewhere, right? And here’s the here’s the the human beings that Judge enhanced is increases the racial bias because he or she brings her own, the algorithm does not add to the racial bias.

Angèle Christin
Well, even so, you could kind of argue against it, for example, by saying that like you know, if The algorithm basically categorize black people as being higher risk compared to similar white people with a similar record that being lower risk, well, then it’s going to send like judges are going to send black people to prison much more. And then that’s going to be fed back into the data. And it’s going to be a kind of self fulfilling prophecy. So you could actually even argue that it does reinforce racial inequalities, right?

Josh Landy
And people are going to trust and that’s the other insidious part.

Ken Taylor
Well, that’s the part about I say, the algorithm is an honorific term, there’s a positive definite procedure that will get you the right answer if you just do it in a fight. But these things, really algorithms, they’re really kind of heuristics. I mean, I mean, looked at the problem, there’s south who should go to prison and who shouldn’t who for how long? It’s not really an algorithm for deciding that there’s a heuristic, but that heuristic imports value judgments and deciding the other thing and you Ristic is going to be challenged, algorithms can’t be challenged, they just deliver the right answer.

Josh Landy
And yet the the average person on the street like me is gonna say, Well, I mean, the algorithm says, Yes, exactly. It’s objective, right?

Angèle Christin
And that’s the thing, right? Because also, because we think that algorithms are so objective, then we don’t think about who is actually having an influence on the algorithm. So in the case of criminal justice, for example, you need people to fill in the variables of the defendants into the algorithms, right. And so I’ve been doing research and other people have been doing research showing that to pretrial officers and social workers who feelings variables into computer system like we all do, right, being like age, sex, criminal record of the defendant, and then those little things that says, calculate risk, and the people who do that actually know how to manipulate the variables in order to get the results they want. Yeah. So you know, you never eliminate discretion, even when you use this kind of system.

Ken Taylor
Right. We’ve got another caller on the line. Margo from Berkeley. Welcome to Philosophy Talk, Marga, what’s your comment or question?

Marga
Welcome. Great. Well, I just know that as far as medicine goes, I would have been dead eight times because the system does not medical knowledge in general does not apply to my polio body. Or, and the same thing, maybe for law and decisions? Isn’t the human mind more able to nuance things away from the greatest number kind of thinking?

Ken Taylor
Not quite. So you’re opposed to MediCal? I’m not quite sure the gist of your question.

Marga
Oh, I’m saying that because of my own unique medical conditions. The medical knowledge is wrong. If I had trusted it, I would be dead six times. I’m happily alive. I happily do not believe that. General knowledge of but that’s a different program. Yeah, I’m saying now is also it but it’s similar. Aren’t humans able to make wiser decision? No, they’re not going to the general average?

Ken Taylor
I want to say something, I would say no, I think the medical system is headed inexorably, the way of the air traffic control system. A human is a node in a complicated system, right? Where all this general, all this knowledge is in the system, not in the human system calls on the pilot, when it needs to call on the pilot, the medical system is going to go the same way. And you we’re never going to look back and we’re and we’re all going to be better off. We’re headed the medicine is headed the way of the air traffic control system.

Angèle Christin
See, but East criminal justice the same thing. So I mean, I would argue that it’s different, because justice is about individualized judgment. It’s about a judge, making sure that you as an individual deserves the sentence as your name, whereas algorithms right compare you to a group, that’s what they do. So just putting you in a class of people with similar characteristics. And then you all get the same risk of—

Ken Taylor
Oh no, I agree with I would be much more uncomfortable turning this, trying to make the criminal justice system like the air traffic control system. But I actually think the air traffic control system is a wondrous thing. I mean, air travel is so much more safe than it used to be. And it’s because basically, of the role of automation and computers and the human computer interaction, the here the system, all the knowledge is in the system, and it calls on a human to exercise certain judgment at certain times. And otherwise, it doesn’t, right. And and it’s advising the guy sitting at the council, and it’s suggesting trajectories and all that. And, and you can do the same eventually with the medical system, because you don’t need all that knowledge to be in this doctor’s head at this moment. It’s in the system, it says to the doctor, okay, try this intervene here. And we do want that for the medical system. So I would speak up for these guys, when it talks about automating that kind of thing. But I think you’re right, it doesn’t follow that. Everything should be like that

Josh Landy
Right, I think one thing to bear in mind look, you know, obviously I have my worries about a number of these systems, but you One thing to bear in mind is that we can’t hope for perfection. Right? I mean, we’re, you know, we’ve got a choice of either a set of algorithm procedures that are going to get it right. Some of the time, maybe all of the time or a set of human beings are also going to get it right some of the time, not all of the time. So it’s it’s not like we there’s a system out there of any kind that’s going to produce perfect results in every single case. The question is, which is going to do a better job? And it sounds like you’re saying, medicine? Yes. criminal justice system now?

Angèle Christin
Yeah. But you know, I don’t study medicine. So I’m sure that like medical scholars who are like, no, don’t do this. Right. But like, from the outside, I would say, Yeah, because just the output is very, is simpler.

Ken Taylor
Yeah. It’s simply, although the medicine is really complicated, used to be fact, a cancer was regarded as a 10 factor disease. It’s now 1000 factor disease, and doctors can’t keep up with it. But the whole system can, but—

Josh Landy
But still the value question, honestly, I was saying earlier is is much simpler.

Ken Taylor
That’s true. You’re listening to Philosophy Talk, we’re thinking about the ethics of algorithms with Angèle Christin from Stanford University.

Josh Landy
When algorithms are biased, just the problem lie in our software or in our hearts? Shouldn’t we work on eliminating our biases and blind spots, so we don’t offload them onto the machines?

Ken Taylor
Curing machines by curing ourselves—when Philosophy Talk continues.

New Musik
Living by numbers, living by numbers now.

Ken Taylor
Living by algorithm, does it all add up to you? I’m Ken Taylor, and this is Philosophy Talk, the program that questions everything…

Josh Landy
…except your intelligence. I’m Josh Landy. And our guest is Angèle Christin from Stanford University. Today, we’re thinking about the morality of algorithms.

Ken Taylor
So I have a question that I think I don’t believe the presuppositions of, I’m gonna ask you anyway, let’s say that the ultimate source of biases in our machines is always us, then can’t we just do our machines by queuing ourselves?

Angèle Christin
I think we can train ourselves to be a bit more careful about what we feed into the system. So you know, depends what you call us. Right? But like, I think that training computer scientists and developers to be more careful about the kind of stuff that goes into the algorithms that they make is a valuable goal.

Ken Taylor
Yeah, I think it’s a valuable goal. But what reason I don’t believe it’s sufficient. So be a fair minded, try and guard against your own biases. Okay. But look, you’re going to use this data as your training set, which is the history of decision sentencing decisions in America, right? And it’s already got the bias built into it, okay, you’re going to do a machine learning algorithm, it’s going to figure out its own biases, and it’s going to give you the answers you think you want or something, right. Cuz this guy got this guy went back to jail, this guy didn’t vote, but figure out what factors were, you know.

Angèle Christin
I think that training goes fails us, and that it’s not only about fixing the models and fixing the biases, it’s also about deciding that sometimes we can build things, but that doesn’t necessarily mean that we should build them. Right. So in such cases, it’s like, well, yeah, we could build an algorithm predicting stuff. But that’s we shouldn’t build it.

Josh Landy
Oh, there was a case recently of a facial recognition algorithm that purported to detect sexual orientation.

Ken Taylor
That’s our Stanford colleague. He got hammered for that. Research is much more nuanced, and people gave it credit to, but I gotta say, he did get hammered for that.

Angèle Christin
So the question is, I mean, so a couple of questions, right. The first one is like, so is the case of like sexual predicting sexual orientation based on pictures, like, you know, is that like, you know, accurate or not. But the second question, like, should we do that?

Ken Taylor
Exactly. Yeah. So you mean, you think there are questions about whether we should deploy a technology that’s there? You don’t think that technology that says, I’m here use me? That was a joke.

Angèle Christin
Well, depends on the cases. No, but I do think that sometimes like that, you know, right. Now, the first step is like, it’s technical. It’s always about, okay, we have to figure out the technical problem. But I think the next step is like, well, do we need to build an algorithm for that kind of issues? And perhaps we should not, and we should hold off. And, you know, in order to do that, you need a specific mindset. You need a specific training to be able to kind of take a step back and be like, okay, perhaps we shouldn’t build it.

Ken Taylor
I totally agree. Here. We got a caller on the line. I think you’re a real human and not a bot. Ryan from Oakland. Welcome to Philosophy Talk, Ryan, assuming you’re a human and not a bot.

Ryan
I am I am assuredly human.

Ken Taylor
So what’s your comment?

Ryan
I just—my question for the panel was if we’re concerned with racial bias and algorithms for the criminal justice system, what’s to keep us from creating the algorithm to just remove that race as a data set when it’s making its verdicts?

Ken Taylor
Good question.

Angèle Christin
Yeah, So people are trying to do that. Basically, they’re trying to clean the algorithms from bias, right? By cleaning data by cleanings valuables by cleanings and models. I mean, so there’s a whole section of computer science called fairness, accountability and transparency in machine learning, which kind of does exactly that. devotes a whole time to exam. Now, I mean, for me, my question is like, okay, so we could clean it for race. But what about gender? What about class? What about religion? What about age? What about—all of these are variables, where do we stop?

Ken Taylor
Plus if race is a significant variable, because these things decide how to how to partition the data into different categories, it could discover something that’s exactly coextensive with race. Right? And, and, and this thing that I was talking about this transparency thing, this is a really big deal, because you can do these machine learning algorithms. And this is kind of how it first went in a way that they’re totally opaque what they’re representing over. And because of various issues, people have started to think, well, maybe we can’t have these things be so opaque. So there’s lots of researchers trying to have sort of transparent machine learning, right? There’s—that’s a big, big deal, but we’re not there 100%

Josh Landy
And of course, it’s not in the criminal justice system, only we think about banks, who’s who gets a loan, there are about two. And again, you can try to clean the data of race, but there are going to be these covariant factors. And so it’s, it seems like an almost intractable problem. But also it sounds like you’re saying we should be giving programmers training in sociology, ethics, I mean—

Ken Taylor
Philosophy!

Josh Landy
I didn’t want to say.

Angèle Christin
You know, what the solution is? world needs is? Yeah, precisely. So, for example, I’m teaching this class called the politics of algorithms, where I get a lot of computer science students at Stanford, and And what surprised me, I mean, now students can always say things, right, but it’s just like, oh, it’s the first time I take a class like this. And I’m a senior, and I’m going to work at Facebook next month. How come I wasn’t trained into this kind of question early on.

Ken Taylor
I think we’re right to insist the answer is more accurate. And that is to say, I think we are Stanford and all their leading university, we’re turning out tons and tons of technology leaders, tons and tons. If we turn them out insensitive to these political, social, moral, ethical problems. We have not done the world a service, we’ve unleashed the potential creators of Frankenstein’s monsters on them. And that’s not what we’re here for.

Angèle Christin
I mean, I think that, you know, technology and we’re in the middle of the Bay Area, right? It’s just it’s not monsters, right. Obviously, it’s just that, you know, they’re very good at technical stuff.

Ken Taylor
I didn’t call them monsters—creators of the monsters.

Angèle Christin
But it just basically, you can have social and political questions come as an afterthought. Right. And I think that right now, we see that Facebook, Twitter, Google are basically just starting to realize and, you know, obviously, they’ve been working on this for a while, but you know, to realize that, that that all of the algorithms have this, like broad social and political impact that they hadn’t kind of anticipated.

Ken Taylor
Right, Tom from Berkeley is on the line. Welcome to Philosophy Talk, Tom, assuming that you, too, are a human rather than a bot.

Tom
I am a human. Thank you.

Josh Landy
They all say that.

Marga
Well, I do want to make an observation. I’ve been working in the courts for many years. And in places where the judge would be arbitrary and capricious, the person who’s going before him would not feel that what they did, had consequences that the judge was going to screw him over. And the thing that occurs to me is that any system that is willing to make those mistakes against the individual who’s trying to make his life better, is is going to take away the sense that what they do, actually helps them move up the ladder, and that they can’t win in that situation. Labeling is really a tough thing. It’s like sending people to the gas chamber. Well, I don’t do that anymore. But to the death death chamber, I would like to see us fine. One or two people in all of that, who are innocent as being a really important thing over is this going to just get more people that shouldn’t be out there into the death chamber.

Angèle Christin
I mean, I think that, you know, just Cimber metaphor, but like, there is real real point here, which is that cooks are overloaded. I mean, just like think about it, like basically the number of judges and prosecutors and Ghana pretrial officers hasn’t followed with a number of cases. that has exploded over the past 30 years, mostly because of, you know, repressive kind of loose that led to mock and criminalization etc. But you know, what you have to realize, as you’re saying is that in court system, basically judges and prosecutors don’t have the time to give this kind of individualized attention to every single case. So what I mean by this is that we need to be careful what we compare the algorithmic situation for. And here, I’m kind of criticizing myself, right? It’s like, we’re saying, Oh, you have this individual judge who is paying careful attention to every single case. And then you have these algorithms that does kind of mass traffic stuff, right. But basically, judges are so rushed, so So basically, algorithms perhaps are not that bad.

Ken Taylor
No, so so. So we can’t blind ourselves. So we’re talking about how we got to turn our technologists into humanists and social scientists as well, right, and deeply reflective human beings. But at the same time, you’re saying, and I think you’re right, there is real world pressure, right? It’s not just that we’re using this technology, because well, hey, we like technology, it’s real world pressure that drives people to use this technology. And that technology actually relieves real world pressure. So I mean, it’s a really complicated thing, deciding when to deploy this, and you can’t just say, it’s like, it’s like we’ve done shows about the disappearance of work. I mean, if I’m a competitor, and I’m trying to win in this economic competition, and I can reduce my overall costs by automating this and eliminating the humans with their need for time off, and this that in the other thing, and fair wages, I’m gonna do it. So there’s real world pressure that drives the deployment of technology, and balancing out those real load pressures and our competing values. That’s a really hard thing. It’s not at all simple.

Angèle Christin
I think that’s right. But at the end of the day, I think it’s a matter of like, your kind of political orientation. Right? So are you basically a reformist? And you’re like, Okay, we should take small steps to reform the criminal justice system, because everybody agrees it’s a situation right now is not tenable. Right, like, something needs to change. So or you’re like, Okay, so let’s introduce algorithms one step at a time to at least make things a tiny bit less problematic. And, oh, you might want to kind of, you know, Revolutionary Road when you’re just like, Okay, let’s abolish bail. Let’s just decriminalize a whole lot of stuff. And, you know, these are just different ways of dealing with a problem.

Josh Landy
Ikay, I mean, I agree with that. But I’m wondering, you know, given that the reformists, many farmers think, yes, we should introduce algorithms, but the algorithms have these biases built in, you’ve talked about solutions on the front end training program has to be more aware of social issues. What about on the back end? How about discretion? How about judgment? I mean, can is there could there be training for, for example, judges to, to read the data more, you know, more responsibly?

Ken Taylor
Yeah, this’ll be your last comment. So give us your last bit of wisdom.

Angèle Christin
Okay, you can train users, that’s possible, it’s probably going to improve saying so in the case of legal professionals having statistics and you know, some CS classes would be a good idea, probably. Now, I’m a bit cautious about this idea of training users just because it kind of seems to like, put the blame on the victims. I mean, I do think that it is the creators of the algorithms at the end of the day is that Belzer responsibility of producing tools that is not unfair, nor to users. So you know, that’s kind of why—

Josh Landy
I want you to take your class.

Ken Taylor
On that note, I’m gonna thank you for joining us. This has been a very non-algorithmic conversation, because there’s no definite answer. But thanks for joining us.

Angèle Christin
Thank you so much for inviting me.

Ken Taylor
Our guest has been Angèle Christin. She’s a professor of communications at Stanford University, author of “Algorithms in Practice: Comparing Web Journalism and Criminal Justice.” So Josh got one last thought?

Josh Landy
I’m just wondering when algorithms are gonna replace us—well, maybe maybe not you, but me.

Ken Taylor
I don’t know if algorithms ever replace English professors. I don’t know. But on that note, I will say that the conversation continues at Philosophers Corner, at our online community of thinkers where our motto is with apologies to Descartes and my good friend, Cogito ergo Blogo, I think, therefore I blog, and you can become a partner in the community by visiting our website, philosophytalk.org.

Josh Landy
And if you have a question that wasn’t addressed in today’s show, we’d love to hear from you. Send it to us at comments@philosophytalk.org, and we might feature it on our blog. Now, no algorithm could possibly make this guy talk any faster—it’s Ian Shoales the Sixty-Second Philosopher.

Ian Shoales
Ian Shoales… So the other week on Facebook, I made some snarky post about an article I’d read about cryptocurrency. Somebody came back at me, defending cryptocurrency, luckily for cryptocurrency, otherwise my withering scorn might have sassed it out of existence, at least for the hundred or so people who read my Facebook posts, none of whom do business in Bitcoin I don’t think. Anyway, one commenter told me that when fire was discovered, I was probably a caveman in the dark pooh poohing the whole idea. May be true! Probably depended on what the weather was like when the original log was lit. If it was hot, I would have failed to see the need, and if it was cold, I would have probably killed the guy who disovered fire, and tried to claim the idea as my own. That’s how I roll. Same with cryptocurrency. I have money in my wallet right now. What need have I of bitcoin? If I was broke, well, I dunno, I’d probably feel the same, frankly. Who needs Bitcoin when I can get a predatory loan? Thanks, Mr. Trump! So you might say I have a bias against algorithms, but it’s seasonal, or it depends on what problem it’s supposed to fix. If it gets me packages from Amazon faster, thanks I guess, but frankly, I do not care. There are algorithms on the way that supposedly can read your face and know what you want before you do. Read my face, Amazon. I do not want that thing you think I want. And if I do decide to order a CD from you, because no place in the free world outside of bargain bins even has CDs any more, I can wait a week before I get it. You don’t have to have some Frisbee hurling robot throw it at my face before I even place my order. I realize that many people think this heightened efficiency is amazing and “cool,” to use the hipster and techie word of choice. Cool used to be Nat King Cole in a skinny necktie. That’s the trouble with algorithms, bias aside, their coolness is implicit and inescapable. You can only roll over and obey. Any hesitation brands you a luddite. And anybody whose job is disappeared because of algorithmic whims, well, you are told, you should learn how to code. And should problematic problems arise, as with a program called CAMPOS, which allegedly is used to determine the probability that somebody in a given neighborhood may commit a crime. It also could be racist, allegedly, but since the algorithm therein is proprietary, who knows? Content guarded by legal teams, proprietary algorithms fuel overnight deliveries, risk assessments, parole hearings, travel itineraries, all kinds of things. It all sounds so magical, but keep in mind, an algorithm is nothing but a recipe. Could make a nice cake, could make a noxious brew. Depends on what you put into it. And overnight delivery is a fabulous thing. Unless you live on a mountain. Or in a ghetto. In which case, shut up and wait. The latest thing that is either great or alarming or both is facial recognition at airports. Who needs tickets? Smile at the camera. The algorithm knows it’s you. Welcome aboard! The fear is that this facial data is stored, and could be misused somewhere down the line- how, I don’t know. The great bugaboo, our personal information, is at risk, I guess. Read in a glance- our credit score, yearly income, health, sexual habits. Put that on the dark web and stand back. It could also be used to target people as potential terrorists, or whatever. Each twitch tells a story, and also that inappropriate nervous smile could mark you as a sucker for targeted marketing of self-help books, mood elevators, and diet supplements. Most of us won’t care. Why? Because now we don’t have to carry around a ticket any more! Our lives are enormously benefitted by this little piece of paper no longer to worry about. With empty pockets and clear heads, we take our seat, with a steaming soy latte already waiting. That’s right. It knows you like soy lattes. I don’t like soy lattes. You love soy lattes. Please, I don’t want the soy latte. Take the soy latte. There’s a bias implicit in algorithms, in that we are biased in favor of having them, even though, well, we were never asked, were we? Algorithms say we already said yes. So drink the latte. It’s your favorite. You know it is. I gotta go.

Ken Taylor
Philosophy Talk is a presentation of KALW local public radio San Francisco and the trustees of Leland Stanford Junior University, copyright 2018.

Josh Landy
Our executive producers are David Demarest and Matt Martin.

Ken Taylor
The Senior Producer is Devon Strolovitch. Laura Maguire is our Director of Research. Cindy Prince Baum is our Director of Marketing.

Josh Landy
Thanks also to Merle Kessler, Angela Johnston, and Lauren Schecter.

Ken Taylor
Support for philosophy top comes from Stanford University and from the partners at our online community of thinkers.

Josh Landy
The views expressed or (mis-expressed) on this program do not necessarily represent the opinions of Stanford University or of our other funders.

Ken Taylor
Not even when they’re true and reasonable.

Josh Landy
The conversation continues on our website, philosophytalk.org, where you too can become a partner in our community of thinkers. I’m Josh Landy.

Ken Taylor
And I’m Ken Taylor. Thank you for listening.

Josh Landy
And thank you for thinking.

Little Britain
Can I get to Toronto the following day instead? Computer says no. You didn’t even type anything in there. It says no.