In a blog post, one scenario Amnesty foresees AI being used for is autonomous systems choosing military targets with little-to-no human oversight.

Military AI Fears

The development of AI has been likened to another arms race. Much like nuclear weapons, there is the argument if a nation doesn’t develop its capabilities then others will. Furthermore, there’s a greater incentive to use it with the knowledge of having the upper-hand.

Much progress has been made on nuclear disarmament, although the US and Russia still hold — and modernise — huge arsenals (approximately 6,800 and 7,000 warheads, respectively.)

This rivalry shows no signs of letting up and Russia continues to be linked with rogue state-like actions including hacking, interference with Western diplomatic processes, misinformation campaigns, and even assassinations.

Last week, it was unveiled that New York-based artificial intelligence startup Clarifai saw a server compromised while it was conducting secretive work on the U.S. Defense Department’s Project Maven.

Project Maven, which Google has decided it will not renew its contract to lend its expertise to following backlash, aims to automate the processing of drone images. While it’s unclear whether the hack was state-sponsored, it allegedly originated from Russia.

AI Discrimination

The next concern on Amnesty’s list is discrimination by biased algorithms — whether intentional, or not.

Unfortunately, the current under-representation problem in STEM fields is causing unintentional bias.

Here in the West, technologies are still mostly developed by white males and can often unintentionally perform better for this group.

A 2010 study by researchers at NIST and the University of Texas in Dallas found that algorithms designed and tested in East Asia are better at recognising East Asians, while those designed in Western countries are more accurate at detecting Caucasians.

Digital rights campaigners Access Now recently wrote in a post:

“From policing, to welfare systems, online discourse, and healthcare – to name a few examples – systems employing machine learning technologies can vastly and rapidly change or reinforce power structures or inequalities on an unprecedented scale and with significant harm to human rights.”

One company, Pymetrics, recently unveiled an open source tool for detecting unintentional bias in algorithms. As long as such tools are used, it could be very important to ensuring digital equality.

Meanwhile, some companies are deliberately implementing bias in their algorithms.

Russian startup NtechLab has come under fire after building an ‘ethnicity detection’ feature into its facial recognition system. Considering the existing problem of racial profiling, the idea of it becoming automated naturally raises some concern.

In a bid to quell fears about the use of its own technology for nefarious purposes, Google published its ethical principles for AI development.

Google says it will not develop technologies or weapons that cause harm, or anything which can be used for surveillance violating “internationally accepted norms” or “widely accepted principles of international law and human rights.”

Last month, Amnesty International and Access Now circulated the Toronto Declaration (PDF) which proposed a set of principles to prevent discrimination from AI and help to ensure its responsible development.

Then you’ve got MIT, who deliberately built a psychopathic AI based on a serial killer.

What are your thoughts on Amnesty International’s AI concerns?

The post Amnesty International warns of AI ‘nightmare scenarios’ appeared first on AI News.

Amnesty International and Access Now prepared the ‘Toronto Declaration’ (PDF) that’s also been signed by Human Rights Watch and the Wikimedia Foundation. As an open declaration; other companies, governments, and organisations are being called on to add their endorsement.

In a post, Access Now wrote:

“As machine learning systems advance in capability and increase in use, we must examine the positive and negative implications of these technologies.

We acknowledge the potential for these technologies to be used for good and to promote human rights, but also the potential to intentionally or inadvertently discriminate against individuals or groups of people.

We must keep our focus on how these technologies will affect individual human beings and human rights. In a world of machine learning systems, who will bear accountability for harming human rights?”

Ethics have become a major talking point in the AI industry. However, much of the conversation so far has focused on drawing red lines when it comes to surveillance and military applications.

There’s a big debate over AIs potential impact to jobs. Some believe automation will cause a work shortage, while others argue that most will simply be enhanced by AI.

If jobs are being replaced, ideas like a universal income will have to be re-examined. If jobs are being enhanced, ensuring AI is indiscriminate will be even more important.

AI has already shown discrimination

Technologies developed and used in the West are typically developed by white males.

Research has been performed into the gender and race gap of executives in Silicon Valley. This data at least provides some indication of the representation problem:

What this means is that, unintentionally, products often perform better for this particular group. Today, that could just mean something relatively trivial like Siri recognising an American male voice with greater accuracy (even as a British male, Silicon Valley-developed products often struggle with my accent!)

A 2010 study by researchers at NIST and the University of Texas in Dallas found that algorithms designed and tested in East Asia are better at recognising East Asians, while those designed in western countries are more accurate at detecting Caucasians.

However, if jobs are becoming more reliant on AI, they need to work as well for everyone who uses them. Failing to do so will put certain groups at a greater advantage than others.

“From policing, to welfare systems, online discourse, and healthcare – to name a few examples – systems employing machine learning technologies can vastly and rapidly change or reinforce power structures or inequalities on an unprecedented scale and with significant harm to human rights,” wrote Access Now.

Policing is one area of particular concern. An investigative report by ProPublica revealed that computer-generated ‘risk assessment scores’ used to determine eligibility for parole are almost twice as likely to label black defendants as potential repeat offenders, despite evidence to the contrary.

Similarly, a 2012 study (paywall) by the IEEE  found that police surveillance cameras using facial recognition to identify suspected criminals are five to 10 percent less accurate when identifying African Americans – which could lead to more innocent black people being arrested.

Machine learning models for AI are often trained on public data and therefore we must be careful about what sources are used. Microsoft’s attempt to create a chatbot which learns from the public, Tay, infamously ended up becoming a rather unsavoury character spouting racist and sexist remarks.

The declaration signed today is a great start to keep these issues in mind as AI technologies are being developed, but it will require tackling inequalities across the whole of society to make developments truly representative of those it serves.

What are your thoughts on the AI discrimination issue?

The post Editorial: Stopping AI’s discrimination will be difficult, but vital appeared first on AI News.