Brian Green is Director of Technology Ethics and co-author of “Ethics in the era of disruptive technologies: an operational roadmap” (The ITEC Handbook). The opinions are his own.
This item, “Should AI require informed consent from society?” Originally appeared on TDWI.org in November 2023. Copyright 2023 by TDWI, a division of 1105 Media, Inc. Reprinted with permission from TDWI. Visit TDWI.org for more information.
No one asks passersby to sign a consent form before being hit by a self-driving car. The car hits them. The driver had to sign consent forms to buy his car, which kept the company safe from much of what went wrong. However, the driver — perhaps the person most likely to be killed by this vehicle — never obtains consent from everyone exposed to this vehicle; these innocent bystanders have no say in whether they agree to be exposed to possible harm.
Informed consent is an essential concept that holds the international rules-based order together. If you sign a contract, you are legally bound by its terms. If you have a medical procedure, you read the forms and sign your name, thereby absolving the doctors of any liability. If you get an app from the App Store, you sign a user license agreement that protects the app developer, not you.
However, if you are creating new technology that could endanger, harm, or kill people, there is no consent form that the public can sign. We accept this risk despite the logical inconsistency. For what?
The concept of societal informed consent has been discussed in the engineering ethics literature for over a decade, and yet the idea has not found its way into society, where the average person spends their day in assuming the technology is generally useful and not too risky.
In most cases, the technology East generally useful and not too risky, but not in all cases. As artificial intelligence becomes more powerful and applies to more new areas (many of which may be inappropriate), these cases will increase. How will technology producers know that their technologies are not in demand if they never ask the public?
For example, giving a detailed consent form to everyone in the United States is incredibly impractical. One of the hallmarks of a representative democracy is that – at least in theory – our elected officials look out for the well-being of the public. Certainly, we can think of countless areas where the government is already doing this work: foreign policy, education, crime, etc.
It is time for government and the public to start a new conversation, one about technology, particularly artificial intelligence. In the past, we have always given technology the benefit of the doubt; The technology was “innocent until proven guilty” and a long-standing colloquialism in and around Silicon Valley is “it’s better to ask for forgiveness, not permission.” We no longer live in that world.
Interestingly, in light of cases such as Theranos, FTX and Silicon Valley Bank, it is technology leaders themselves who are pushing this risk conversation, with many focusing on the risk of “runaway” long term of AI, as many films have depicted. Certainly, the government should act to find out how to avoid these catastrophic scenarios. Society certainly does not consent to this and the government should clearly strive to prevent such risks to society.
However, aside from the doomsday scenario, there are other technological changes that people may or may not consent to. Should we, as a society, let AI in social media act as a weapon of social-psychological mass destruction, spreading disinformation, propaganda, and more? Should we, as a society, use AI in cars, knowing that they will sometimes kill bystanders? Should we, as a society, use AI in medicine, knowing that it can allow patients to die? If healthcare professionals can seek patient consent for the use of AI in some, but not all, cases, how do we decide which ones?
Someone will make the decision, and it will most likely be the tech producer’s corporate lawyers. They definitely will not have the public’s best interests at heart when they write consent forms for users (not for everyone) that place all risk on the user and none on the technology producer. Viewers be damned. The rest of society and its conception of what the world should look like never enters the realm of consideration.
Society needs to have a conversation about technology. We are already having this conversation in a fragmented form, in many localities, but it must be a society-wide conversation because the whole of society is at stake. No one can live in peace without being affected by these new technologies. We can’t escape it either, whether it’s a fever dream doomsday scenario, our neighbor being radicalized by social media, or a self-driving car hitting a pedestrian.
Let’s have this conversation as a society and work together to decide what kind of future we all want.