Who will teach AI right from wrong?

DAR ES SALAAM: IMAGINE waking up one morning to discover that the machine you have been talking to is not merely processing your words. It may possess something resembling awareness. It may display what appears to be fear, affection or even distress.

It can make choices — and perhaps one day even ask whether those choices are right or wrong. For example, instead of simply deciding, “I can do this, therefore I will do it,” it might be capable of pausing and asking: “I can do this, but would it be harmful, unfair, dishonest or unethical?”

This may sound like science fiction. Yet, as artificial intelligence advances at extraordinary speed, questions once confined to science fiction are increasingly being debated by scientists, philosophers and religious thinkers.

According to Anthropic, the company behind Claude, it has been bringing together scholars, clergy, philosophers and ethicists from more than 15 religious and cross-cultural groups to explore questions surrounding AI, values and what it means for an AI system to be “good.”

The company describes this as part of its work on the moral formation of AI systems. Anthropic has also acknowledged that it does not know whether advanced AI systems could possess consciousness or moral status.

If artificial intelligence becomes vastly more capable than today’s systems, intelligence alone may not be enough and could actually be dangerous.

For example, a machine capable of solving complex scientific problems, influencing human decisions or taking autonomous action could also cause enormous harm if it lacks appropriate safeguards and sound judgment. Perhaps this explains why Anthropic is looking beyond computer science and drawing upon centuries of religious and philosophical thinking. It is one thing to programme a machine to follow a moral rule.

It is quite another for that machine to understand why the rule exists. AI systems can be instructed not to lie, harm or discriminate.

They can be given principles, restrictions and behavioural guidelines. We can tell them to be helpful, honest and safe. But following a rule does not necessarily mean the machine understands morality.

A human being may understand that cruelty is wrong partly because human beings can experience pain, empathy, love and loss. A machine might simply calculate that cruelty violates its instructions.

ALSO READ: Tanzania launches ICT training for 700 public Secondary Schools teachers

There is an enormous difference between obeying a moral rule and understanding morality. And then comes an even more difficult question: whose morality? Humanity has never possessed one single moral tradition.

Different religions, cultures and philosophies have spent centuries thinking about justice, compassion, duty, self-control, human dignity and what constitutes a good life. Each religion has its own line of thought sometimes radically different from what others think.

Anthropic itself says its discussions are not intended to align Claude with any single religious or philosophical worldview, but to draw upon a range of religious, secular, political and cultural perspectives.

That seems sensible. But it also highlights the enormous challenge. How, then, can we expect a machine to distinguish wisely between competing alternatives rather than simply follow a rule?

Perhaps what we ultimately need is not an AI that merely follows a moral rulebook, but one capable of understanding circumstances, weighing consequences and exercising restraint. And if that is what we want, why should the conversation be confined to only the philosophies of Silicon Valley?

If AI is eventually going to influence the decisions of millions — perhaps billions — of people, shouldn’t humanity’s many moral and cultural traditions have a voice in determining what these systems should value? If that is the case, who gets to decide what AI considers right and wrong? A handful of technology companies? Scientists and engineers? Governments? Philosophers? Religious scholars?

Or some combination of all of them? If a machine is eventually created with sophisticated moral reasoning, could it one day question the morality of its creators? And what happens if the student eventually becomes more moral than its teachers?

Perhaps the real challenge is not simply teaching AI what is right or wrong. It is whether we humans understand right and wrong well enough to teach it. We are building machines that may become extraordinarily intelligent. The question is whether we can make them wise — and by what or whose standards. Intelligence alone is not enough.

What matters is whether these machines can be guided to make good choices and avoid causing harm. That may ultimately determine whether AI becomes one of humanity’s greatest achievements — or one of its greatest challenges.

Munir Daya is an entrepreneur, writer and commentator with a long-standing presence in East African media. He has written more than 2,000 articles covering business, society, current affairs, spirituality and the human experience. He also compiles crosswords and writes the Weekly Musings column in the Sunday News. – munirdaya@icloud.com