“If we would use the software only [for subject identification], we would not solve the case 95-97 percent of the time,” Craig said. “If we were just to use the technology by itself to identify someone, I would say 96 percent of the time it would misidentify.”

Craig’s comments arrive just days after the ACLU (American Civil Liberties Union) lodged a complaint against the Detroit police following the harrowing wrongful arrest of black male Robert Williams due to a facial recognition error.

Detroit Police arrested Williams for allegedly stealing five watches valued at $3800 from a store in October 2018. A blurry CCTV image was matched by a facial recognition algorithm to Williams’ driver’s license photo.

Current AI algorithms are known to have a racism issue. Extensive studies have repeatedly shown that facial recognition algorithms are almost 100 percent accurate when used on white males, but have serious problems when it comes to darker skin colours and the fairer sex.

This racism issue was shown again this week after an AI designed to upscale blurry photos, such as those often taken from security cameras, was applied to a variety of people from the BAME communities.

Here’s a particularly famous one:

🤔🤔🤔 pic.twitter.com/LG2cimkCFm

— Chicken3gg (@Chicken3gg) June 20, 2020

And some other examples:

Last week, Boston followed in the footsteps of an increasing number of cities like San Francisco, Oakland, and California in banning facial recognition technology over human rights concerns.

“Facial recognition is inherently dangerous and inherently oppressive. It cannot be reformed or regulated. It must be abolished,” said Evan Greer, deputy director of the digital rights group Fight for the Future.

Over the other side of the pond, facial recognition tests in the UK so far have also been nothing short of a complete failure. An initial trial at the 2016 Notting Hill Carnival led to not a single person being identified. A follow-up trial the following year led to no legitimate matches but 35 false positives.

An independent report into the Met Police’s facial recognition trials, conducted last year by Professor Peter Fussey and Dr Daragh Murray, concluded that it was only verifiably accurate in just 19 percent of cases.

The next chilling step for AI in surveillance is using it to predict crime. Following news of an imminent publication called ‘A Deep Neural Network Model to Predict Criminality Using Image Processing,’ over 1000 experts signed an open letter last week opposing the use of AI for such purposes.

“Machine learning programs are not neutral; research agendas and the data sets they work with often inherit dominant cultural beliefs about the world,” warned the letter’s authors.

The acknowledgement from Detroit’s police chief that current facial recognition technologies do not work in around 96 percent of cases should be reason enough to halt its use, especially for law enforcement, at least until serious improvements are made.

(Photo by Joshua Hoehne on Unsplash)

The post Detroit Police chief says AI face recognition doesn’t work ‘96% of the time’ appeared first on AI News.

While it’s been suspected that documented racial bias with facial recognition algorithms has led to false arrests, it’s been difficult to prove.

On Wednesday, the ACLU lodged a complaint against the Detroit police after black male Robert Williams was arrested on his front lawn “as his wife Melissa looked on and as his daughters wept from the trauma”. Williams was held in a “crowded and filthy” cell overnight without being given any reason.

Detroit Police arrested Williams for allegedly stealing five watches valued at $3800 from a store in October 2018. A blurry CCTV image was matched by a facial recognition algorithm to Williams’ driver’s license photo.

During an interrogation the day after his arrest, the police admitted that “the computer must have gotten it wrong”. Williams was kept incarcerated until after dark “at which point he was released out the front door, on a cold and rainy January night, where he was forced to wait outside on the curb for approximately an hour while his wife scrambled to find child care for the children so that she could come pick him up.”

Speaking to the NY Times, a Detroit police spokesperson said the department “does not make arrests based solely on facial recognition,” and claims witness interviews and a photo lineup were used.

However, a response from the Wayne County prosecutor’s office confirms the department used facial recognition to identify Williams using the security footage and an eyewitness to the crime was not shown the alleged photo lineup.

In its complaint, the ACLU demands that Detroit police end the use of facial recognition “as the facts of Mr. Williams’ case prove both that the technology is flawed and that DPD investigators are not competent in making use of such technology.”

This week, Boston became the latest city to ban facial recognition technology for municipal use. Boston follows an increasing number of cities like San Francisco, Oakland, and California who’ve banned the technology over human rights concerns.

“Facial recognition is inherently dangerous and inherently oppressive. It cannot be reformed or regulated. It must be abolished,” said Evan Greer, deputy director of the digital rights group Fight for the Future.

“Boston just became the latest major city to stop the use of this extraordinary and toxic surveillance technology. Every other city should follow suit.”

Cases like Mr Williams’ are certainly strengthening such calls. Over 1,000 experts signed an open letter this week against the use of AI for the next chilling step, crime prediction.

(Photo by ev on Unsplash)

The post The ACLU uncovers the first known wrongful arrest due to AI error appeared first on AI News.

“Companies like Clearview will end privacy as we know it, and must be stopped,” said Nathan Freed Wessler, senior staff attorney with the ACLU’s Speech, Privacy, and Technology Project.

“The ACLU is taking its fight to defend privacy rights against the growing threat of this unregulated surveillance technology to the courts, even as we double down on our work in legislatures and city councils nationwide.”

Clearview AI has repeatedly come under fire due to its practice of scraping billions of photos from across the internet and storing them in a database for powerful facial recognition services.

“Common law has never recognised a right to privacy for your face,” Clearview AI lawyer Tor Ekeland said recently.

The company’s facial recognition system is used by over 2,200 law enforcement agencies around the world – and even commercial businesses like Best Buy and Macy’s, according to a recent leak.

In a press release, the ACLU wrote:

“The New York Times revealed the company was secretly capturing untold numbers of biometric identifiers for purposes of surveillance and tracking, without notice to the individuals affected.

The company’s actions embodied the nightmare scenario privacy advocates long warned of, and accomplished what many companies — such as Google — refused to try due to ethical concerns.”

However, even more concerning is Clearview AI’s extensive ties with the far-right.

Clearview AI founder Hoan Ton-That claims to have since disassociated from far-right views, movements, and individuals. Ekeland, meanwhile, has gained notoriety as “The Troll’s Lawyer” for defending clients such as neo-Nazi troll Andrew Auernheimer.

The ACLU says its lawsuit represents the first “to force any face recognition surveillance company to answer directly to groups representing survivors of domestic violence and sexual assault, undocumented immigrants, and other vulnerable communities uniquely harmed by face recognition surveillance.”

Facial recognition technologies have become a key focus for the ACLU.

Back in March, AI News reported the ACLU was suing the US government for blocking a probe into the use of facial recognition technology at airports. In 2018, the union caught our attention for highlighting the inaccuracy of Amazon’s facial recognition algorithm – especially when identifying people of colour and females.

“Clearview’s actions represent one of the largest threats to personal privacy by a private company our country has faced,” said Jay Edelson of Edelson PC, lead counsel handling this case on a pro bono basis.

“If a well-funded, politically connected company can simply amass information to track all of us, we are living in a different America.”

The post ACLU sues Clearview AI calling it a ‘nightmare scenario’ for privacy appeared first on AI News.

Michael Punke, VP of Global Public Policy at Amazon Web Services, said.

“Over the past several months, we’ve talked to customers, researchers, academics, policymakers, and others to understand how to best balance the benefits of facial recognition with the potential risks.

It’s critical that any legislation protect civil rights while also allowing for continued innovation and practical application of the technology.”

In a blog post today, Amazon highlighted five guidelines to ensure facial recognition is developed and used ethically.

The first of the five calls for facial recognition to follow existing laws which protect civil liberties. To ensure accountability, the second guideline wants all facial recognition to be reviewed by humans before any decision is taken.

Other guidelines include a call for transparancy in how agencies are using facial recognition technology, and visual notices placed where it’s being used in public or commercial settings.

Facial Recognition Concerns

The company has faced criticism of its ‘Rekognition’ system which is used by police forces and has been pitched to agencies such as US Immigration and Customs Enforcement (ICE).

In a letter addressed to Amazon CEO Jeff Bezos, employees wrote:

“We refuse to build the platform that powers ICE, and we refuse to contribute to tools that violate human rights.

As ethically concerned Amazonians, we demand a choice in what we build and a say in how it is used.”

The letter was sent following ICE’s separation of immigrant children from their families at the US border and subsequent detainment. There’s no evidence ICE ultimately purchased or used Amazon’s technology.

In July last year, the American Civil Liberties Union tested Amazon’s facial recognition technology on members of Congress to see if they match with a database of criminal mugshots.

Rekognition compared pictures of all members of the House and Senate against 25,000 arrest photos. The false matches disproportionately affected members of the Congressional Black Caucus.

Dr Matt Wood, General Manager of AI at Amazon Web Services, commented on the ACLU’s findings later that month. He said the ACLU left Rekognition’s default confidence setting of 80 percent on when it suggests 95 percent or higher for law enforcement.

Wood, however, went on to say it showed how standards are needed to ensure facial recognition systems are used properly. He called for “the government to weigh in and specify what temperature (or confidence levels) it wants law enforcement agencies to meet to assist in their public safety work.”

The call for facial recognition standards extends beyond the US. In China, the CEO of SenseTime – the world’s most funded AI startup – also said he wants to see facial recognition standards established for a ‘healthier’ industry.

In the UK, Information Commissioner Elizabeth Denham announced her office has identified facial recognition technology as a priority to establish what protections are needed for the public.

SenseTime is so well-funded not just because of its powerful facial recognition technology, but also from adoption by the Chinese government. The firm aims to process and analyse over 100,000 simultaneous real-time streams from traffic cameras, ATMs, and more as part of its ‘Viper’ system.

If such a system was deployed with biased algorithms, it will exacerbate current societal problems. Algorithmic Justice League founder Joy Buolamwini gave a fantastic presentation during the World Economic Forum last month on the need to fight AI bias.

As Spider-Man’s Uncle Ben would say: “With great power, comes great responsibility”.

The post Amazon joins calls to establish facial recognition standards appeared first on AI News.

AI can be unpredictable, and Microsoft already has experience. Back in 2016, a Microsoft chatbot named Tay became a racist, sexist, generally-rather-unsavoury character after internet users took advantage of its machine learning capabilities

The chatbot was covered in media around the world and itself was bound to have caused Microsoft some reputational damage.

In the company’s latest quarterly report, Microsoft warned investors:

“AI algorithms may be flawed. Datasets may be insufficient or contain biased information. Inappropriate or controversial data practices by Microsoft or others could impair the acceptance of AI solutions.

These deficiencies could undermine the decisions, predictions, or analysis AI applications produce, subjecting us to competitive harm, legal liability, and brand or reputational harm.

Some AI scenarios present ethical issues. If we enable or offer AI solutions that are controversial because of their impact on human rights, privacy, employment, or other social issues, we may experience brand or reputational harm.”

Several companies have been criticised for unethical AI development, including several of Microsoft’s competitors.

AI Backlash

Google was embroiled in a backlash over its ‘Project Maven’ defence contract to supply drone analysing AI to the Pentagon. The contract received both internal and external criticism.

Several Googlers left the company and others threatened if the contract was not dropped. Over 4,000 signed a petition demanding their management cease the project and never again “build warfare technology.”

Following the Project Maven backlash, Google CEO Sundar Pichai said the contract would not be renewed.

Pichai also promised in a blog post the company will not develop technologies or weapons that cause harm or anything which can be used for surveillance violating ‘internationally accepted norms’ or ‘widely accepted principles of international law and human rights’.

In June last year, Microsoft faced its own internal revolt over a $19 million contract with ICE (Immigration and Customs Enforcement) during a time when authorities were splitting up immigrant families.

Microsoft CEO Satya Nadella was forced to clarify that Microsoft isn’t directly involved with the government’s policy of separating families at the US-Mexico border.

A report from the American Civil Liberties Union found bias in Amazon’s facial recognition algorithm. Amazonians wrote a letter to CEO Jeff Bezos expressing their concerns.

Problems with AI bias keep arising and likely will continue for some time. It’s an issue which needs to be tackled before any mass rollouts.

Last month, Algorithmic Justice League founder Joy Buolamwini gave an impactful presentation during the World Economic Forum on the AI bias issue.

Microsoft is clearly preparing investors for some controversial slip-ups of its own along its AI development journey.

deepgeniusai.com/">AI & Big Data Expo events with upcoming shows in Silicon Valley, London, and Amsterdam to learn more. Co-located with the IoT Tech Expo, , & .

The post Microsoft warns its AI offerings ‘may result in reputational harm’ appeared first on AI News.

During the Data for Black Lives conference last weekend, several experts shared how AI is evolving America’s controversial prison system.

America imprisons more people than any other nation. It’s not just a result of the population of the country, the incarceration per head is the highest in the world at ~716 per 100,000 of the national population. The second largest, Russia, incarcerates ~455 per 100,000 population.

Black males are, by far, America’s most incarcerated:

AI has been proven to have bias problems. Last year, the American Civil Liberties Union found that Amazon’s facial recognition technology disproportionately flagged those with darker skin colours as criminals more often.

The bias is not intentional but a result of a wider problem in STEM career diversity. In the West, the fields are dominated by white males.

A 2010 study by researchers at NIST and the University of Texas in Dallas found (PDF) algorithms designed and tested in East Asia are better at recognising East Asians, while those developed in Western countries are more accurate when detecting Caucasians.

Deploying such inherently-biased AIs is bound to exacerbate societal problems. Most concerning, US courtrooms are using AI tools for ‘risk’ assessments to make sentencing decisions.

Using a defendant’s profile, the AI generates a recidivism score – a number which aims to estimate if an individual will reoffend. A judge then uses that score to make decisions such as the severity of their sentence, what services the individual should be provided, and if a person should be held in jail before trial.

Last July, a statement (PDF) was signed by over 100 civil rights organisations – including the ACLU – calling for AI to be kept clear of risk assessments.

When the bias problem with AIs is solved, their use in the justice system could improve trust in decisions. Current questions over whether a judge was prejudiced in their sentencing will be reduced. However, we’re yet to be anywhere near that point.

deepgeniusai.com/">AI & Big Data Expo events with upcoming shows in Silicon Valley, London, and Amsterdam to learn more. Co-located with the IoT Tech Expo, , & .

The post AI is sentencing people based on their ‘risk’ assessment appeared first on AI News.

Dr. Matt Wood, GM of Deep Learning and AI at Amazon Web Services, made the suggestion in a blog post responding to the ACLU’s (American Civil Liberties Union) findings of a racial bias in the ‘Rekognition’ facial recognition algorithm by Amazon.

In their findings, the ACLU found Rekognition erroneously labelled those with darker skin colours as criminals more often when members of Congress were matched against a database of 25,000 arrest photos.

Amazon argued the ACLU left Rekognition’s default confidence setting of 80 percent on when it suggests 95 percent or higher for law enforcement.

Commenting on the ACLU’s findings, Wood wrote:

“The default confidence threshold for facial recognition APIs in Rekognition is 80%, which is good for a broad set of general use cases (such as identifying celebrities on social media or family members who look alike in photos apps), but it’s not the right setting for public safety use cases.

The 80% confidence threshold used by the ACLU is far too low to ensure the accurate identification of individuals; we would expect to see false positives at this level of confidence.”

Wood provided a case example of their own test where – using a dataset of over 850,000 faces commonly used in academia – the company searched against public photos of all members of US Congress ‘in a similar way’ to the ACLU.

Using the 99 percent confidence threshold, the misidentification rate dropped to zero despite comparing against a larger number of faces (30x larger than the ACLU test).

Amazon is naturally keen to highlight the positive uses its technology has been used for. The company says it’s been used for things such as fighting human trafficking and reuniting lost children with their families.

However, the ACLU’s test shows the potential for the technology to be misused to disastrous effect. Without oversight, civil liberties could be impacted and lead to increased persecution of minorities.

To help prevent this from happening, Wood calls it “a very reasonable idea” for “the government to weigh in and specify what temperature (or confidence levels) it wants law enforcement agencies to meet to assist in their public safety work.”

A 2010 study by researchers at NIST and the University of Texas in Dallas found that algorithms designed and tested in East Asia are better at recognising East Asians, while those designed in Western countries are more accurate at detecting Caucasians.

When a clear bias problem remains in AI algorithms, it’s little wonder there’s concern about the use of inaccurate facial recognition for things such as police body cams.

Should a minimum confidence level be set for law enforcement?

The post Amazon expert suggests AI regulation after ACLU’s bias findings appeared first on AI News.

Bias in AI technology, when used by law enforcement, has raised concerns of infringing on civil rights by automated racial profiling.

A 2010 study by researchers at NIST and the University of Texas in Dallas found that algorithms designed and tested in East Asia are better at recognising East Asians, while those designed in Western countries are more accurate at detecting Caucasians.

The ACLU (American Civil Liberties Union) ran a test of Amazon’s facial recognition technology on members of Congress to see if they match with a database of criminal mugshots.

Amazon’s Rekognition tool was used to compare pictures of all members of the House and Senate against 25,000 arrest photos, the false matches disproportionately affected members of the Congressional Black Caucus.

In a blog post, the ACLU said:

“The members of Congress who were falsely matched with the mugshot database we used in the test include Republicans and Democrats, men and women, and legislators of all ages, from all across the country.

These results demonstrate why Congress should join the ACLU in calling for a moratorium on law enforcement use of face surveillance.”

AWS (Amazon Web Services) disputed the methodology used by the ACLU.

The company says the default setting of 80 percent confidence was left on. For law enforcement, Amazon says it suggests the option to only register matches of 95 percent confidence or above.

Only 28 members of Congress were incorrectly flagged as being among the criminal mugshots, but nearly 40 percent of the wrong matches were people of darker skin colours. What really puts it all into perspective, however, is that only 20 percent of Congress are people of colour.

“Our test reinforces that face surveillance is not safe for government use,” said Jacob Snow, Technology and Civil Liberties Attorney at the ACLU Foundation of Northern California. “Face surveillance will be used to power discriminatory surveillance and policing that targets communities of color, immigrants, and activists. Once unleashed, that damage can’t be undone.”

Amazon is actively marketing its facial recognition technology to law enforcement such as police in Washington County, Oregon, and Orlando, Florida. They promote it as a way to identify people in real-time from both surveillance footage and officers’ body cameras.

What are your thoughts on the ACLU’s findings?

The post ACLU finds Amazon’s facial recognition AI is racially biased appeared first on AI News.