Google’s artificial intelligence company DeepMind has published „really significant“ research showing its algorithm can identify around 50 eye diseases by looking at retinal eye scans.
DeepMind said its AI was as good as expert clinicians, and that it could help prevent people from losing their sight.
DeepMind has been criticised for its practices around medical data, but cofounder Mustafa Suleyman said all the information in this research project was anonymised.
The company plans to hand the technology over for free to NHS hospitals for five years, provided it passes the next phase of research.
Google’s artificial intelligence company, DeepMind, has developed an AI which can successfully detect more than 50 types of eye disease just by looking at 3D retinal scans.
DeepMind published on Monday the results of joint research with Moorfields Eye Hospital, a renowned centre for treating eye conditions in London, in Nature Medicine.
The company said its AI was as accurate as expert clinicians when it came to detecting diseases, such as diabetic eye disease and macular degeneration. It could also recommend the best course of action for patients and suggest which needed urgent care.
A technician examines an OCT scan.DeepMind
What is especially significant about the research, according to DeepMind cofounder Mustafa Suleyman, is that the AI has a level of „explainability“ that could boost doctors‘ trust in its recommendations.
„It’s possible for the clinician to interpret what the algorithm is thinking,“ he told Business Insider. „[They can] look at the underlying segmentation.“
In other words, the AI looks less like a mysterious black box that’s spitting out results. It labels pixels on the eye scan that corresponds to signs of a particular disease, Suleyman explained, and can calculate its confidence in its own findings with a percentage score. „That’s really significant,“ he said.
DeepMind’s AI analysing an OCT scan.DeepMind
Suleyman described the findings as a „research breakthrough“ and said the next step was to prove the AI works in a clinical setting. That, he said, would take a number of years. Once DeepMind is in a position to deploy its AI across NHS hospitals in the UK, it will provide the service for free for five years.
Patients are at risk of losing their sight because doctors can’t look at their eye scans in time
British eye specialists have been warning for years that patients are at risk of losing their sight because the NHS is overstretched, and because the UK has an ageing population.
Part of the reason DeepMind and Moorfields took up the research project was because clinicians are „overwhelmed“ by the demand for eye scans, Suleyman said.
„If you have a sight-threatening disease, you want treatment as soon as possible,“ he explained. „And unlike in A&E, where a staff nurse will talk to you and make an evaluation of how serious your condition is, then use that evaluation to decide how quickly you are seen. When an [eye] scan is submitted, there isn’t a triage of your scan according to its severity.“
A patient having an OCT scan.DeepMind
Putting eye scans through the AI could speed the entire process up.
„In the future, I could envisage a person going into their local high street optician, and have an OCT scan done and this algorithm would identify those patients with sight-threatening disease at the very early stage of the condition,“ said Dr Pearse Keane, consultant ophthalmologist at Moorfields Eye Hospital.
DeepMind’s AI was trained on a database of almost 15,000 eye scans, stripped of any identifying information. DeepMind worked with clinicians to label areas of disease, then ran those labelled images through its system. Suleyman said the two-and-a-half project required „huge investment“ from DeepMind and involved 25 staffers, as well as the researchers from Moorfields.
People are still worried about a Google-linked company having access to medical data
While DeepMind has remained UK-based and independent from Google, the relationship has attracted scrutiny. The main question is whether Google, a private US company, should have access to the sensitive medical data required for DeepMind’s health arm.
Microsoft has helped innovate facial recognition software. Now it’s urging the US government to enact regulation to control the use of the technology.
In a blog post, Microsoft (MSFT)President Brad Smith said new laws are necessary given the technology’s „broad societal ramifications and potential for abuse.“
He urged lawmakers to form „a government initiative to regulate the proper use of facial recognition technology, informed first by a bipartisan and expert commission.“
Facial recognition — a computer’s ability to identify or verify people’s faces from a photo or through a camera — has been developing rapidly. Apple (AAPL), Google (GOOG), Amazon and Microsoft are among the big tech companies developing and selling such systems. The technology is being used across a range of industries, from private businesses like hotels and casinos, to social media and law enforcement.
Supporters say facial recognition software improves safety for companies and customers and can help police track police down criminals or find missing children. Civil rights groups warn it can infringe on privacy and allow for illegal surveillance and monitoring. There is also room for error, they argue, since the still-emerging technology can result in false identifications.
The accuracy of facial recognition technologies varies, with women and people of color being identified with less accuracy, according to MIT research.
„Facial recognition raises a critical question: what role do we want this type of technology to play in everyday society?“ Smith wrote on Friday.
Smith’s call for a regulatory framework to control the technology comes as tech companies face criticism over how they’ve handled and shared customer data, as well as their cooperation with government agencies.
Last month, Microsoft was scrutinized for its working relationship with US Immigration and Customs Enforcement. ICE had been enforcing the Trump administration’s „zero tolerance“ immigration policy that separated children from their parents when they crossed the US border illegally. The administration has since abandoned the policy.
Microsoft wrote a blog post in January about ICE’s use of its cloud technology Azure, saying it could help it „accelerate facial recognition and identification.“
After questions arose about whether Microsoft’s technology had been used by ICE agents to carry out the controversial border separations, the company released a statement calling the policy „cruel“ and „abusive.“
In his post, Smith reiterated Microsoft’s opposition to the policy and said he had confirmed its contract with ICE does not include facial recognition technology.
Amazon(AMZN) has also come under fire from its own shareholders and civil rights groups over local police forces using its face identifying software Rekognition, which can identify up to 100 people in a single photo.
Some Amazon shareholders coauthored a letter pressuring Amazon to stop selling the technology to the government, saying it was aiding in mass surveillance and posed a threat to privacy rights.
And Facebook (FB) is embroiled in a class-action lawsuit that alleges the social media giant used facial recognition on photos without user permission. Its facial recognition tool scans your photos and suggests you tag friends.
Neither Amazon nor Facebook immediately responded to a request for comment about Smith’s call for new regulations on face ID technology.
Smith said companies have a responsibility to police their own innovations, control how they are deployed and ensure that they are used in a „a manner consistent with broadly held societal values.“
„It may seem unusual for a company to ask for government regulation of its products, but there are many markets where thoughtful regulation contributes to a healthier dynamic for consumers and producers alike,“ he said.
Artificial Intelligence — The Revolution Hasn’t Happened Yet
Artificial Intelligence (AI) is the mantra of the current era. The phrase is intoned by technologists, academicians, journalists and venture capitalists alike. As with many phrases that cross over from technical academic fields into general circulation, there is significant misunderstanding accompanying the use of the phrase. But this is not the classical case of the public not understanding the scientists — here the scientists are often as befuddled as the public. The idea that our era is somehow seeing the emergence of an intelligence in silicon that rivals our own entertains all of us — enthralling us and frightening us in equal measure. And, unfortunately, it distracts us.
There is a different narrative that one can tell about the current era. Consider the following story, which involves humans, computers, data and life-or-death decisions, but where the focus is something other than intelligence-in-silicon fantasies. When my spouse was pregnant 14 years ago, we had an ultrasound. There was a geneticist in the room, and she pointed out some white spots around the heart of the fetus. “Those are markers for Down syndrome,” she noted, “and your risk has now gone up to 1 in 20.” She further let us know that we could learn whether the fetus in fact had the genetic modification underlying Down syndrome via an amniocentesis. But amniocentesis was risky — the risk of killing the fetus during the procedure was roughly 1 in 300. Being a statistician, I determined to find out where these numbers were coming from. To cut a long story short, I discovered that a statistical analysis had been done a decade previously in the UK, where these white spots, which reflect calcium buildup, were indeed established as a predictor of Down syndrome. But I also noticed that the imaging machine used in our test had a few hundred more pixels per square inch than the machine used in the UK study. I went back to tell the geneticist that I believed that the white spots were likely false positives — that they were literally “white noise.” She said “Ah, that explains why we started seeing an uptick in Down syndrome diagnoses a few years ago; it’s when the new machine arrived.”
We didn’t do the amniocentesis, and a healthy girl was born a few months later. But the episode troubled me, particularly after a back-of-the-envelope calculation convinced me that many thousands of people had gotten that diagnosis that same day worldwide, that many of them had opted for amniocentesis, and that a number of babies had died needlessly. And this happened day after day until it somehow got fixed. The problem that this episode revealed wasn’t about my individual medical care; it was about a medical system that measured variables and outcomes in various places and times, conducted statistical analyses, and made use of the results in other places and times. The problem had to do not just with data analysis per se, but with what database researchers call “provenance” — broadly, where did data arise, what inferences were drawn from the data, and how relevant are those inferences to the present situation? While a trained human might be able to work all of this out on a case-by-case basis, the issue was that of designing a planetary-scale medical system that could do this without the need for such detailed human oversight.
I’m also a computer scientist, and it occurred to me that the principles needed to build planetary-scale inference-and-decision-making systems of this kind, blending computer science with statistics, and taking into account human utilities, were nowhere to be found in my education. And it occurred to me that the development of such principles — which will be needed not only in the medical domain but also in domains such as commerce, transportation and education — were at least as important as those of building AI systems that can dazzle us with their game-playing or sensorimotor skills.
Whether or not we come to understand “intelligence” any time soon, we do have a major challenge on our hands in bringing together computers and humans in ways that enhance human life. While this challenge is viewed by some as subservient to the creation of “artificial intelligence,” it can also be viewed more prosaically — but with no less reverence — as the creation of a new branch of engineering. Much like civil engineering and chemical engineering in decades past, this new discipline aims to corral the power of a few key ideas, bringing new resources and capabilities to people, and doing so safely. Whereas civil engineering and chemical engineering were built on physics and chemistry, this new engineering discipline will be built on ideas that the preceding century gave substance to — ideas such as “information,” “algorithm,” “data,” “uncertainty,” “computing,” “inference,” and “optimization.” Moreover, since much of the focus of the new discipline will be on data from and about humans, its development will require perspectives from the social sciences and humanities.
While the building blocks have begun to emerge, the principles for putting these blocks together have not yet emerged, and so the blocks are currently being put together in ad-hoc ways.
Thus, just as humans built buildings and bridges before there was civil engineering, humans are proceeding with the building of societal-scale, inference-and-decision-making systems that involve machines, humans and the environment. Just as early buildings and bridges sometimes fell to the ground — in unforeseen ways and with tragic consequences — many of our early societal-scale inference-and-decision-making systems are already exposing serious conceptual flaws.
And, unfortunately, we are not very good at anticipating what the next emerging serious flaw will be. What we’re missing is an engineering discipline with its principles of analysis and design.
The current public dialog about these issues too often uses “AI” as an intellectual wildcard, one that makes it difficult to reason about the scope and consequences of emerging technology. Let us begin by considering more carefully what “AI” has been used to refer to, both recently and historically.
Most of what is being called “AI” today, particularly in the public sphere, is what has been called “Machine Learning” (ML) for the past several decades. ML is an algorithmic field that blends ideas from statistics, computer science and many other disciplines (see below) to design algorithms that process data, make predictions and help make decisions. In terms of impact on the real world, ML is the real thing, and not just recently. Indeed, that ML would grow into massive industrial relevance was already clear in the early 1990s, and by the turn of the century forward-looking companies such as Amazon were already using ML throughout their business, solving mission-critical back-end problems in fraud detection and supply-chain prediction, and building innovative consumer-facing services such as recommendation systems. As datasets and computing resources grew rapidly over the ensuing two decades, it became clear that ML would soon power not only Amazon but essentially any company in which decisions could be tied to large-scale data. New business models would emerge. The phrase “Data Science” began to be used to refer to this phenomenon, reflecting the need of ML algorithms experts to partner with database and distributed-systems experts to build scalable, robust ML systems, and reflecting the larger social and environmental scope of the resulting systems.
This confluence of ideas and technology trends has been rebranded as “AI” over the past few years. This rebranding is worthy of some scrutiny.
Historically, the phrase “AI” was coined in the late 1950’s to refer to the heady aspiration of realizing in software and hardware an entity possessing human-level intelligence. We will use the phrase “human-imitative AI” to refer to this aspiration, emphasizing the notion that the artificially intelligent entity should seem to be one of us, if not physically at least mentally (whatever that might mean). This was largely an academic enterprise. While related academic fields such as operations research, statistics, pattern recognition, information theory and control theory already existed, and were often inspired by human intelligence (and animal intelligence), these fields were arguably focused on “low-level” signals and decisions. The ability of, say, a squirrel to perceive the three-dimensional structure of the forest it lives in, and to leap among its branches, was inspirational to these fields. “AI” was meant to focus on something different — the “high-level” or “cognitive” capability of humans to “reason” and to “think.” Sixty years later, however, high-level reasoning and thought remain elusive. The developments which are now being called “AI” arose mostly in the engineering fields associated with low-level pattern recognition and movement control, and in the field of statistics — the discipline focused on finding patterns in data and on making well-founded predictions, tests of hypotheses and decisions.
Indeed, the famous “backpropagation” algorithm that was rediscovered by David Rumelhart in the early 1980s, and which is now viewed as being at the core of the so-called “AI revolution,” first arose in the field of control theory in the 1950s and 1960s. One of its early applications was to optimize the thrusts of the Apollo spaceships as they headed towards the moon.
Since the 1960s much progress has been made, but it has arguably not come about from the pursuit of human-imitative AI. Rather, as in the case of the Apollo spaceships, these ideas have often been hidden behind the scenes, and have been the handiwork of researchers focused on specific engineering challenges. Although not visible to the general public, research and systems-building in areas such as document retrieval, text classification, fraud detection, recommendation systems, personalized search, social network analysis, planning, diagnostics and A/B testing have been a major success — these are the advances that have powered companies such as Google, Netflix, Facebook and Amazon.
One could simply agree to refer to all of this as “AI,” and indeed that is what appears to have happened. Such labeling may come as a surprise to optimization or statistics researchers, who wake up to find themselves suddenly referred to as “AI researchers.” But labeling of researchers aside, the bigger problem is that the use of this single, ill-defined acronym prevents a clear understanding of the range of intellectual and commercial issues at play.
The past two decades have seen major progress — in industry and academia — in a complementary aspiration to human-imitative AI that is often referred to as “Intelligence Augmentation” (IA). Here computation and data are used to create services that augment human intelligence and creativity. A search engine can be viewed as an example of IA (it augments human memory and factual knowledge), as can natural language translation (it augments the ability of a human to communicate). Computing-based generation of sounds and images serves as a palette and creativity enhancer for artists. While services of this kind could conceivably involve high-level reasoning and thought, currently they don’t — they mostly perform various kinds of string-matching and numerical operations that capture patterns that humans can make use of.
Hoping that the reader will tolerate one last acronym, let us conceive broadly of a discipline of “Intelligent Infrastructure” (II), whereby a web of computation, data and physical entities exists that makes human environments more supportive, interesting and safe. Such infrastructure is beginning to make its appearance in domains such as transportation, medicine, commerce and finance, with vast implications for individual humans and societies. This emergence sometimes arises in conversations about an “Internet of Things,” but that effort generally refers to the mere problem of getting “things” onto the Internet — not to the far grander set of challenges associated with these “things” capable of analyzing those data streams to discover facts about the world, and interacting with humans and other “things” at a far higher level of abstraction than mere bits.
For example, returning to my personal anecdote, we might imagine living our lives in a “societal-scale medical system” that sets up data flows, and data-analysis flows, between doctors and devices positioned in and around human bodies, thereby able to aid human intelligence in making diagnoses and providing care. The system would incorporate information from cells in the body, DNA, blood tests, environment, population genetics and the vast scientific literature on drugs and treatments. It would not just focus on a single patient and a doctor, but on relationships among all humans — just as current medical testing allows experiments done on one set of humans (or animals) to be brought to bear in the care of other humans. It would help maintain notions of relevance, provenance and reliability, in the way that the current banking system focuses on such challenges in the domain of finance and payment. And, while one can foresee many problems arising in such a system — involving privacy issues, liability issues, security issues, etc — these problems should properly be viewed as challenges, not show-stoppers.
We now come to a critical issue: Is working on classical human-imitative AI the best or only way to focus on these larger challenges? Some of the most heralded recent success stories of ML have in fact been in areas associated with human-imitative AI — areas such as computer vision, speech recognition, game-playing and robotics. So perhaps we should simply await further progress in domains such as these. There are two points to make here. First, although one would not know it from reading the newspapers, success in human-imitative AI has in fact been limited — we are very far from realizing human-imitative AI aspirations. Unfortunately the thrill (and fear) of making even limited progress on human-imitative AI gives rise to levels of over-exuberance and media attention that is not present in other areas of engineering.
Second, and more importantly, success in these domains is neither sufficient nor necessary to solve important IA and II problems. On the sufficiency side, consider self-driving cars. For such technology to be realized, a range of engineering problems will need to be solved that may have little relationship to human competencies (or human lack-of-competencies). The overall transportation system (an II system) will likely more closely resemble the current air-traffic control system than the current collection of loosely-coupled, forward-facing, inattentive human drivers. It will be vastly more complex than the current air-traffic control system, specifically in its use of massive amounts of data and adaptive statistical modeling to inform fine-grained decisions. It is those challenges that need to be in the forefront, and in such an effort a focus on human-imitative AI may be a distraction.
As for the necessity argument, it is sometimes argued that the human-imitative AI aspiration subsumes IA and II aspirations, because a human-imitative AI system would not only be able to solve the classical problems of AI (as embodied, e.g., in the Turing test), but it would also be our best bet for solving IA and II problems. Such an argument has little historical precedent. Did civil engineering develop by envisaging the creation of an artificial carpenter or bricklayer? Should chemical engineering have been framed in terms of creating an artificial chemist? Even more polemically: if our goal was to build chemical factories, should we have first created an artificial chemist who would have then worked out how to build a chemical factory?
A related argument is that human intelligence is the only kind of intelligence that we know, and that we should aim to mimic it as a first step. But humans are in fact not very good at some kinds of reasoning — we have our lapses, biases and limitations. Moreover, critically, we did not evolve to perform the kinds of large-scale decision-making that modern II systems must face, nor to cope with the kinds of uncertainty that arise in II contexts. One could argue
that an AI system would not only imitate human intelligence, but also “correct” it, and would also scale to arbitrarily large problems. But we are now in the realm of science fiction — such speculative arguments, while entertaining in the setting of fiction, should not be our principal strategy going forward in the face of the critical IA and II problems that are beginning to emerge. We need to solve IA and II problems on their own merits, not as a mere corollary to a human-imitative AI agenda.
It is not hard to pinpoint algorithmic and infrastructure challenges in II systems that are not central themes in human-imitative AI research. II systems require the ability to manage distributed repositories of knowledge that are rapidly changing and are likely to be globally incoherent. Such systems must cope with cloud-edge interactions in making timely, distributed decisions and they must deal with long-tail phenomena whereby there is lots of data on some individuals and little data on most individuals. They must address the difficulties of sharing data across administrative and competitive boundaries. Finally, and of particular importance, II systems must bring economic ideas such as incentives and pricing into the realm of the statistical and computational infrastructures that link humans to each other and to valued goods. Such II systems can be viewed as not merely providing a service, but as creating markets. There are domains such as music, literature and journalism that are crying out for the emergence of such markets, where data analysis links producers and consumers. And this must all be done within the context of evolving societal, ethical and legal norms.
Of course, classical human-imitative AI problems remain of great interest as well. However, the current focus on doing AI research via the gathering of data, the deployment of “deep learning” infrastructure, and the demonstration of systems that mimic certain narrowly-defined human skills — with little in the way of emerging explanatory principles — tends to deflect attention from major open problems in classical AI. These problems include the need to bring meaning and reasoning into systems that perform natural language processing, the need to infer and represent causality, the need to develop computationally-tractable representations of uncertainty and the need to develop systems that formulate and pursue long-term goals. These are classical goals in human-imitative AI, but in the current hubbub over the “AI revolution,” it is easy to forget that they are not yet solved.
IA will also remain quite essential, because for the foreseeable future, computers will not be able to match humans in their ability to reason abstractly about real-world situations. We will need well-thought-out interactions of humans and computers to solve our most pressing problems. And we will want computers to trigger new levels of human creativity, not replace human creativity (whatever that might mean).
It was John McCarthy (while a professor at Dartmouth, and soon to take a
position at MIT) who coined the term “AI,” apparently to distinguish his
budding research agenda from that of Norbert Wiener (then an older professor at MIT). Wiener had coined “cybernetics” to refer to his own vision of intelligent systems — a vision that was closely tied to operations research, statistics, pattern recognition, information theory and control theory. McCarthy, on the other hand, emphasized the ties to logic. In an interesting reversal, it is Wiener’s intellectual agenda that has come to dominate in the current era, under the banner of McCarthy’s terminology. (This state of affairs is surely, however, only temporary; the pendulum swings more in AI than
in most fields.)
But we need to move beyond the particular historical perspectives of McCarthy and Wiener.
We need to realize that the current public dialog on AI — which focuses on a narrow subset of industry and a narrow subset of academia — risks blinding us to the challenges and opportunities that are presented by the full scope of AI, IA and II.
This scope is less about the realization of science-fiction dreams or nightmares of super-human machines, and more about the need for humans to understand and shape technology as it becomes ever more present and influential in their daily lives. Moreover, in this understanding and shaping there is a need for a diverse set of voices from all walks of life, not merely a dialog among the technologically attuned. Focusing narrowly on human-imitative AI prevents an appropriately wide range of voices from being heard.
While industry will continue to drive many developments, academia will also continue to play an essential role, not only in providing some of the most innovative technical ideas, but also in bringing researchers from the computational and statistical disciplines together with researchers from other
disciplines whose contributions and perspectives are sorely needed — notably
the social sciences, the cognitive sciences and the humanities.
On the other hand, while the humanities and the sciences are essential as we go forward, we should also not pretend that we are talking about something other than an engineering effort of unprecedented scale and scope — society is aiming to build new kinds of artifacts. These artifacts should be built to work as claimed. We do not want to build systems that help us with medical treatments, transportation options and commercial opportunities to find out after the fact that these systems don’t really work — that they make errors that take their toll in terms of human lives and happiness. In this regard, as I have emphasized, there is an engineering discipline yet to emerge for the data-focused and learning-focused fields. As exciting as these latter fields appear to be, they cannot yet be viewed as constituting an engineering discipline.
Moreover, we should embrace the fact that what we are witnessing is the creation of a new branch of engineering. The term “engineering” is often
invoked in a narrow sense — in academia and beyond — with overtones of cold, affectless machinery, and negative connotations of loss of control by humans. But an engineering discipline can be what we want it to be.
In the current era, we have a real opportunity to conceive of something historically new — a human-centric engineering discipline.
I will resist giving this emerging discipline a name, but if the acronym “AI” continues to be used as placeholder nomenclature going forward, let’s be aware of the very real limitations of this placeholder. Let’s broaden our scope, tone down the hype and recognize the serious challenges ahead.
“Unless your parents purge it, your Alexa will hold on to every bit of data you have ever given it, all the way back to the first things you shouted at it as a 2-year-old.”
Among the more modern anxieties of parents today is how virtual assistants will train their children to act. The fear is that kids who habitually order Amazon’s Alexa to read them a story or command Google’s Assistant to tell them a joke are learning to communicate not as polite, considerate citizens, but as demanding little twerps.
This worry has become so widespread that Amazon and Google both announced this week that their voice assistants can now encourage kids to punctuate their requests with „please.“ The version of Alexa that inhabits the new Echo Dot Kids Edition will thank children for „asking so nicely.“ Google Assistant’s forthcoming Pretty Please feature will remind kids to „say the magic word“ before complying with their wishes.
But many psychologists think kids being polite to virtual assistants is less of an issue than parents think—and may even be a red herring. As virtual assistants become increasingly capable, conversational, and prevalent (assistant-embodied devices are forecasted to outnumber humans), psychologists and ethicists are asking deeper, more subtle questions than will Alexa make my kid bossy. And they want parents to do the same.
„When I built my first virtual child, I got a lot of pushback and flak,“ recalls developmental psychologist Justine Cassell, director emeritus of Carnegie Mellon’s Human-Computer Interaction Institute and an expert in the development of AI interfaces for children. It was the early aughts, and Cassell, then at MIT, was studying whether a life-sized, animated kid named Sam could help flesh-and-blood children hone their cognitive, social, and behavioral skills. „Critics worried that the kids would lose track of what was real and what was pretend,“ Cassel says. „That they’d no longer be able to tell the difference between virtual children and actual ones.“
But when you asked the kids whether Sam was a real child, they’d roll their eyes. Of course Sam isn’t real, they’d say. There was zero ambiguity.
Nobody knows for sure, and Cassel emphasizes that the question deserves study, but she suspects today’s children will grow up similarly attuned to the virtual nature of our device-dwelling digital sidekicks—and, by extension, the context in which they do or do not need to be polite. Kids excel, she says, at dividing the world into categories. As long as they continue to separate humans from machines, she says, there’s no need to worry. „Because isn’t that actually what we want children to learn—not that everything that has a voice should be thanked, but that people have feelings?“
Point taken. But what about Duplex, I ask, Google’s new human-sounding, phone calling AI? Well, Cassell says, that complicates matters. When you can’t tell if a voice belongs to a human or a machine, she says, perhaps it’s best to assume you’re talking to a person, to avoid hurting a human’s feelings. But the real issue there isn’t politeness, it’s disclosure; artificial intelligences should be designed to identify themselves as such.
What’s more, the implications of a kid interacting with an AI extend far deeper than whether she recognizes it as non-human. „Of course parents worry about these devices reinforcing negative behaviors, whether it’s being sassy or teasing a virtual assistant,” says Jenny Radesky, a developmental behavioral pediatrician at the University of Michigan and co-author of the latest guidelines for media use from the American Academy of Pediatrics. “But I think there are bigger questions surrounding things like kids’ cognitive development—the way they consume information and build knowledge.”
Consider, for example, that the way kids interact with virtual assistants may not actual help them learn. This advertisement for the Echo Dot Kids Edition ends with a girl asking her smart speaker the distance to the Andromeda Galaxy. As the camera zooms out, we hear Alexa rattle off the answer: „The Andromeda Galaxy is 14 quintillion, 931 quadrillion, 389 trillion, 517 billion, 400 million miles away“:
To parents it might register as a neat feature. Alexa knows answers to questions that you don’t! But most kids don’t learn by simply receiving information. „Learning happens happens when a child is challenged,“ Cassell says, „by a parent, by another child, a teacher—and they can argue back and forth.“
Virtual assistants can’t do that yet, which highlights the importance of parents using smart devices with their kids. At least for the time being. Our digital butlers could be capable of brain-building banter sooner than you think.
This week, Google announced its smart speakers will remain activated several seconds after you issue a command, allowing you to engage in continuous conversation without repeating „Hey, Google,“ or „OK, Google.“ For now, the feature will allow your virtual assistant to keep track of contextually dependent follow-up questions. (If you ask what movies George Clooney has starred in and then ask how tall he his, Google Assistant will recognize that „he“ is in reference to George Clooney.) It’s a far cry from a dialectic exchange, but it charts a clear path toward more conversational forms of inquiry and learning.
And, perhaps, something even more. „I think it’s reasonable to ask if parenting will become a skill that, like Go or chess, is better performed by a machine,“ says John Havens, executive director of the the IEEE Global Initiative on Ethics of Autonomous and Intelligent Systems. „What do we do if a kid starts saying: Look, I appreciate the parents in my house, because they put me on the map, biologically. But dad tells a lot of lame dad jokes. And mom is kind of a helicopter parent. And I really prefer the knowledge, wisdom, and insight given to me by my devices.„
Havens jokes that he sounds paranoid, because he’s speculating about what-if scenarios from the future. But what about the more near-term? If you start handing duties over to the machine, how do you take them back the day your kid decides Alexa is a higher authority than you are on, say, trigonometry?
Other experts I spoke with agreed it’s not too early for parents to begin thinking deeply about the long-term implications of raising kids in the company of virtual assistants. „I think these tools can be awesome, and provide quick fixes to situations that involve answering questions and telling stories that parents might not always have time for,“ Radesky says. „But I also want parents to consider how that might come to displace some of the experiences they enjoy sharing with kids.“
Other things Radesky, Cassell, and Havens think parents should consider? The extent to which kids understand privacy issues related to internet-connected toys. How their children interact with devices at their friends‘ houses. And what information other family’s devices should be permitted to collect about their kids. In other words: How do children conceptualize the algorithms that serve up facts and entertainment; learn about them; and potentially profit from them?
„The fact is, very few of us sit down and talk with our kids about the social constructs surrounding robots and virtual assistants,“ Radesky says.
Perhaps that—more than whether their children says „please“ and „thank you“ to the smart speaker in the living room—is what parents should be thinking about.
Lawmakers, child development experts, and privacy advocates are expressing concerns about two new Amazon products targeting children, questioning whether they prod kids to be too dependent on technology and potentially jeopardize their privacy.
In a letter to Amazon CEO Jeff Bezos on Friday, two members of the bipartisan Congressional Privacy Caucus raised concerns about Amazon’s smart speaker Echo Dot Kids and a companion service called FreeTime Unlimited that lets kids access a children’s version of Alexa, Amazon’s voice-controlled digital assistant.
“While these types of artificial intelligence and voice recognition technology offer potentially new educational and entertainment opportunities, Americans’ privacy, particularly children’s privacy, must be paramount,” wrote Senator Ed Markey (D-Massachusetts) and Representative Joe Barton (R-Texas), both cofounders of the privacy caucus.
The letter includes a dozen questions, including requests for details about how audio of children’s interactions is recorded and saved, parental control over deleting recordings, a list of third parties with access to the data, whether data will be used for marketing purposes, and Amazon’s intentions on maintaining a profile on kids who use these products.
In a statement, Amazon said it „takes privacy and security seriously.“ The company said „Echo Dot Kids Edition uses on-device software to detect the wake word and only the wake word. Only once the wake word is detected does it start streaming to the cloud, and it will present a visual indication (the light ring at the top of the device turns blue) to show that it is streaming to the cloud.“
Echo Dot Kids is the latest in a wave of products from dominant tech players targeting children, including Facebook’s communications app Messenger Kids and Google’s YouTube Kids, both of which have been criticized by child health experts concerned about privacy and developmental issues.
Like Amazon, toy manufacturers are also interested in developing smart speakers that would live in a child’s room. In September, Mattel pulled Aristotle, a smart speaker and digital assistant aimed at children, after a similar letter from Markey and Barton, as well as a petition that garnered more than 15,000 signatures.
One of the organizers of the petition, the nonprofit group Campaign for a Commercial Free Childhood, is now spearheading a similar effort against Amazon. In a press release Friday, timed to the letter from Congress, a group of child development and privacy advocates urged parents not to purchase Echo Dot Kids because the device and companion voice service pose a threat to children’s privacy and well-being.
“Amazon wants kids to be dependent on its data-gathering device from the moment they wake up until they go to bed at night,” said the group’s executive director Josh Golin. “The Echo Dot Kids is another unnecessary ‘must-have’ gadget, and it’s also potentially harmful. AI devices raise a host of privacy concerns and interfere with the face-to-face interactions and self-driven play that children need to thrive.”
FreeTime on Alexa includes content targeted at children, like kids’ books and Alexa skills from Disney, Nickelodeon, and National Geographic. It also features parental controls, such as song filtering, bedtime limits, disabled voice purchasing, and positive reinforcement for using the word “please.”
Despite such controls, the child health experts warning against Echo Dot Kids wrote, “Ultimately, though, the device is designed to make kids dependent on Alexa for information and entertainment. Amazon even encourages kids to tell the device ‘Alexa, I’m bored,’ to which Alexa will respond with branded games and content.”
In Amazon’s April press release announcing Echo Dot Kids, the company quoted one representative from a nonprofit group focused on children that supported the product, Stephen Balkam, founder and CEO of the Family Online Safety Institute. Balkam referenced a report from his institute, which found that the majority of parents were comfortable with their child using a smart speaker. Although it was not noted in the press release, Amazon is a member of FOSI and has an executive on the board.
In a statement to WIRED, Amazon said, „We believe one of the core benefits of FreeTime and FreeTime Unlimited is that the services provide parents the tools they need to help manage the interactions between their child and Alexa as they see fit.“ Amazon said parents can review and listen to their children’s voice recordings in the Alexa app, review FreeTime Unlimited activity via the Parent Dashboard, set bedtime limits or pause the device whenever they’d like.
Balkam said his institute disclosed Amazon’s funding of its research on its website and the cover of its report. Amazon did not initiate the study. Balkam said the institute annually proposes a research project, and reaches out to its members, a group that also includes Facebook, Google, and Microsoft, who pay an annual stipend of $30,000. “Amazon stepped up and we worked with them. They gave us editorial control and we obviously gave them recognition for the financial support,” he said.
Balkam says Echo Dot Kids addresses concerns from parents about excessive screen time. “It’s screen-less, it’s very interactive, it’s kid friendly,” he said, pointing out Alexa skills that encourage kids to go outside.
In its review of the product, BuzzFeed wrote, “Unless your parents purge it, your Alexa will hold on to every bit of data you have ever given it, all the way back to the first things you shouted at it as a 2-year-old.”
With Zuckerberg testifying to the US Congress over Facebook’s data privacy and the implementation of GDPRfast approaching, the debate around data ownership has suddenly burst into the public psyche. Collecting user data to serve targeted advertising in a free platform is one thing, harvesting the social graphs of people interacting with apps and using it to sway an election is somewhat worse.
Suffice to say that neither of the above compare to the indiscriminate collection of ordinary civilians’ data on behalf of governments every day.
In 2013, Edward Snowden blew the whistleon the systematic US spy program he helped to architect. Perhaps the largest revelation to come out of the trove of documents he released were the details of PRISM, an NSA program that collects internet communications data from US telecommunications companies like Microsoft, Yahoo, Google, Facebook and Apple. The data collected included audio and video chat logs, photographs, emails, documents and connection logs of anyone using the services of 9 leading US internet companies. PRISM benefited from changes to FISA that allowed warrantless domestic surveillance of any target without the need for probable cause. Bill Binney, former US intelligence official, explains how, for instances where corporate control wasn’t achievable, the NSA enticed third party countries to clandestinely tap internet communication lines on the internet backbone via the RAMPART-A program.What this means is that the NSA was able to assemble near complete dossiers of all web activity carried out by anyone using the internet.
But this is just in the US right?, policies like this wouldn’t be implemented in Europe.
GCHQ, the UK’s intelligence agency allegedly collects considerably more metadata than the NSA. Under Tempora, GCHQ can intercept all internet communications from submarine fibre optic cables and store the information for 30 days at the Bude facility in Cornwall. This includes complete web histories, the contents of all emails and facebook entires and given that more than 25% of all internet communications flow through these cables, the implications are astronomical. Elsewhere, JTRIG, a unit of GCHQ have intercepted private facebook pictures, changed the results of online polls and spoofed websites in real time. A lot of these techniques have been made possible by the 2016 Investigatory Powers Act which Snowden describes as the most “extreme surveillance in the history of western democracy”.
But despite all this, the age old reprise; “if you’ve got nothing to hide, you’ve got nothing to fear” often rings out in debates over privacy.
Indeed, the idea is so pervasive that politicians often lean on the phrase to justify ever more draconian methods of surveillance. Yes, they draw upon the selfsame rhetoric of Joseph Goebbels, propaganda minister for the Nazi regime.
When levelled against the fear of terrorism and death, its easy to see how people passively accept ever greater levels of surveillance. Indeed, Naomi Klein writes extensively in Shock Doctrine how the fear of external threats can be used as a smokescreen to implement ever more invasive policy. But indiscriminate mass surveillance should never be blindly accepted, privacy should and always will be a social norm, despite what Mark Zuckerberg said in 2010. Although I’m sure he may have a different answer now.
So you just read emails and look at cat memes online, why would you care about privacy?
In the same way we’re able to close our living room curtains and be alone and unmonitored, we should be able to explore our identities online un-impinged. Its a well rehearsed idea that nowadays we’re more honest to our web browsers than we are to each other but what happens when you become cognisant that everything you do online is intercepted and catalogued? As with CCTV, when we know we’re being watched, we alter our behaviour in line with whats expected.
As soon as this happens online, the liberating quality provided by the anonymity of the internet is lost. Your thinking aligns with the status quo and we lose the boundless ability of the internet to search and develop our identities. No progress can be made when everyone thinks the same way. Difference of opinion fuels innovation.
This draws obvious comparisons with Bentham’s Panopticon, a prison blueprint for enforcing control from within. The basic setup is as follows; there is a central guard tower surrounded by cells. In the cells are prisoners. The tower shines bright light so that the watchman can see each inmate silhouetted in their cell but the prisoners cannot see the watchman. The prisoners must assume they could be observed at any point and therefore act accordingly. In literature, the common comparison is Orwell’s 1984 where omnipresent government surveillance enforces control and distorts reality. With revelations about surveillance states, the relevance of these metaphors are plain to see.
In reality, theres actually a lot more at stake here.
With the Panopticon certain individuals are watched, in 1984 everyone is watched. On the modern internet, every person, irrespective of the threat they pose, is not only watched but their information is stored and archived for analysis.
Kafka’s The Trial, in which a bureaucracy uses citizens information to make decisions about them, but denies them the ability to participate in how their information is used, therefore seems a more apt comparison. The issue here is that corporations, more so, states have been allowed to comb our data and make decisions that affect us without our consent.
Maybe, as a member of a western democracy, you don’t think this matters. But what if you’re a member of a minority group in an oppressive regime? What if you’re arrested because a computer algorithm cant separate humour from intent to harm?
On the other hand, maybe you trust the intentions of your government, but how much faith do you have in them to keep your data private? The recent hack of the SEC shows that even government systems aren’t safe from attackers. When a business database is breached, maybe your credit card details become public, when a government database that has aggregated millions of data points on every aspect of your online life is hacked, you’ve lost all control of your ability to selectively reveal yourself to the world. Just as Lyndon Johnson sought to control physical clouds, he who controls the modern cloud, will rule the world.
Perhaps you think that even this doesn’t matter, if it allows the government to protect us from those that intend to cause harm then its worth the loss of privacy. The trouble with indiscriminate surveillance is that with so much data you see everything but paradoxically, still know nothing.
Intelligence is the strategic collection of pertinent facts, bulk data collection cannot therefore be intelligent. As Bill Binney puts it “bulk data kills people” because technicians are so overwhelmed that they cant isolate whats useful. Data collection as it is can only focus on retribution rather than reduction.
Granted, GDPR is a big step forward for individual consent but will it stop corporations handing over your data to the government? Depending on how cynical you are, you might think that GDPR is just a tool to clean up and create more reliable deterministic data anyway. The nothing to hide, nothing to fear mentality renders us passive supplicants in the removal of our civil liberties. We should be thinking about how we relate to one another and to our Governments and how much power we want to have in that relationship.
To paraphrase Edward Snowden, saying you don’t care about privacy because you’ve got nothing to hide is analogous to saying you don’t care about freedom of speech because you have nothing to say.
Experts from SANS presented the five most dangerous new cyber attack techniques in their annual RSA Conference 2018 keynote session in San Francisco, and shared their views on how they work, how they can be stopped or at least slowed, and how businesses and consumers can prepare.
The five threats outlined are:
1. Repositories and cloud storage data leakage 2. Big Data analytics, de-anonymization, and correlation 3. Attackers monetize compromised systems using crypto coin miners 4. Recognition of hardware flaws 5. More malware and attacks disrupting ICS and utilities instead of seeking profit.
Repositories and cloud storage data leakage
Ed Skoudis, lead for the SANS Penetration Testing Curriculum, talked about the data leakage threats facing us from the increased use of repositories and cloud storage:
“Software today is built in a very different way than it was 10 or even 5 years ago, with vast online code repositories for collaboration and cloud data storage hosting mission-critical applications. However, attackers are increasingly targeting these kinds of repositories and cloud storage infrastructures, looking for passwords, crypto keys, access tokens, and terabytes of sensitive data.”
He continued: “Defenders need to focus on data inventories, appointing a data curator for their organization and educating system architects and developers about how to secure data assets in the cloud. Additionally, the big cloud companies have each launched an AI service to help classify and defend data in their infrastructures. And finally, a variety of free tools are available that can help prevent and detect leakage of secrets through code repositories.”
Big Data analytics, de-anonymization, and correlation
Skoudis went on to talk about the threat of Big Data Analytics and how attackers are using data from several sources to de-anonymise users:
“In the past, we battled attackers who were trying to get access to our machines to steal data for criminal use. Now the battle is shifting from hacking machines to hacking data — gathering data from disparate sources and fusing it together to de-anonymise users, find business weaknesses and opportunities, or otherwise undermine an organisation’s mission. We still need to prevent attackers from gaining shell on targets to steal data. However, defenders also need to start analysing risks associated with how their seemingly innocuous data can be combined with data from other sources to introduce business risk, all while carefully considering the privacy implications of their data and its potential to tarnish a brand or invite regulatory scrutiny.”
Attackers monetize compromised systems using crypto coin miners
Johannes Ullrich, is Dean of Research, SANS Institute and Director of SANS Internet Storm Center. He has been looking at the increasing use of crypto coin miners by cyber criminals:
“Last year, we talked about how ransomware was used to sell data back to its owner and crypto-currencies were the tool of choice to pay the ransom. More recently, we have found that attackers are no longer bothering with data. Due to the flood of stolen data offered for sale, the value of most commonly stolen data like credit card numbers of PII has dropped significantly. Attackers are instead installing crypto coin miners. These attacks are more stealthy and less likely to be discovered and attackers can earn tens of thousands of dollars a month from crypto coin miners. Defenders therefore need to learn to detect these coin miners and to identify the vulnerabilities that have been exploited in order to install them.”
Recognition of hardware flaws
Ullrich then went on to say that software developers often assume that hardware is flawless and that this is a dangerous assumption. He explains why and what needs to be done:
“Hardware is no less complex then software and mistakes have been made in developing hardware just as they are made by software developers. Patching hardware is a lot more difficult and often not possible without replacing entire systems or suffering significant performance penalties. Developers therefore need to learn to create software without relying on hardware to mitigate any security issues. Similar to the way in which software uses encryption on untrusted networks, software needs to authenticate and encrypt data within the system. Some emerging homomorphic encryption algorithms may allow developers to operate on encrypted data without having to decrypt it first.”
More malware and attacks disrupting ICS and utilities instead of seeking profit
Finally, Head of R&D, SANS Institute, James Lyne, discussed the growing trend in malware and attacks that aren’t profit centred as we have largely seen in the past, but instead are focused on disrupting Industrial Control Systems (ICS) and utilities:
“Day to day the grand majority of malicious code has undeniably been focused on fraud and profit. Yet, with the relentless deployment of technology in our societies, the opportunity for political or even military influence only grows. And rare publicly visible attacks like Triton/TriSYS show the capability and intent of those who seek to compromise some of the highest risk components of industrial environments, i.e. the safety systems which have historically prevented critical security and safety meltdowns.”
He continued: “ICS systems are relatively immature and easy to exploit in comparison to the mainstream computing world. Many ICS systems lack the mitigations of modern operating systems and applications. The reliance on obscurity or isolation (both increasingly untrue) do not position them well to withstand a heightened focus on them, and we need to address this as an industry. More worrying is that attackers have demonstrated they have the inclination and resources to diversify their attacks, targeting the sensors that are used to provide data to the industrial controllers themselves. The next few years are likely to see some painful lessons being learned as this attack domain grows, since the mitigations are inconsistent and quite embryonic.”
FANG (Facebook, Amazon, Netflix, Google/YouTube) is about to take a huge bite out of traditional network TV (ABC, NBC, CBS and Fox), and the media business will never be the same.
To understand the profound implications of the recently announced NFL on Amazon Prime or YouTube TV, it may help to understand the economic engine that drives traditional commercial television.
The goal of the commercial TV business is to package a specific, targeted audience and sell it to the highest bidder. The more precise the targeting, the higher the fee; the bigger the targeted audience, the bigger the fee.
TV is data-poor
Because the broadcast television industry is data poor (it only offers metrics about itself), this model has never been a complete solution for brand or lifestyle advertisers. In practice, an advertiser needs to translate ratings and demographic information from Nielsen into knowledge and insights it can link to its key performance indicators (KPIs). Because content is distributed across so many non-TV platforms, this process gets more difficult every day. How effective was your broadcast TV buy? Was there an increase in sales that could be attributed to it? Could we have spent this portion of our advertising budget differently?
FANG is data-rich
There are four data sets that help define each of us: attention, consumption, passion and intention. While traditional broadcast TV tries to measure or attribute some of these to TV viewership, FANG has actionable data that drives KPIs.
Facebook knows what you are paying attention to. You post and share the things you care about, and your Facebook profile makes your attention actionable.
Amazon knows what you consume and what you’re thinking about consuming. If you’ve bought it or are planning to buy it, Amazon knows it and can act on that data.
Netflix knows your passions. You demonstrate how you can be reached on an emotional level every time you watch a video. Netflix knows more about the kind of entertainment that ignites your passions than you do. It continually acts on that data.
Google/YouTube knows your intentions. You never intend to go to Google and stay there; you search for what you intend to do. Your Google profile indicates, with a very high degree of accuracy, what you are likely to do in the near-term future. This is some of the clearest, most actionable data in the world.
We’ll still have four major networks, just not the familiar four
People often reminisce about the „good ol‘ days“ when there were four major networks: ABC, NBC, CBS and Fox. We are transitioning to a world where there will still be four networks, just not the four networks you’re used to. FANG is delivering actionable data to advertisers in ways that traditional broadcasters simply can’t.
The power of Amazon Prime to a fast-moving consumer goods company may be less significant than the power of Amazon Prime to a consumer electronics manufacturer, but Amazon is becoming a complete solution for all types of b-to-c — and many types of b-to-b — advertisers. Its size, scale and efficacy are truly stunning.
If YouTube TV and other over-the-top skinny bundles start to get traction, we are going to see a dramatic shift toward the data-rich, brand-safe, internet giants. (Yes, Facebook and Google will deal with their current content adjacency and brand safety problems, and you will forget they had them.) FANG will not be alone. Apple is going to get into this game, and there are international powerhouses like Alibaba and QQ that are already well on their way.
What does this really mean?
For today: Advertisers are spending, traditional networks are making money and all of this sounds like stuff you’ve heard before. But we’re only talking about timing. Traditional (linear) TV audiences are declining at a significant rate, and they are practically aged out of key demographics. Cable customers are also declining. So, the question is when this shift will make a difference, not if.
For consumers: More choice, more fun. Consumers don’t care about content transport mechanisms or broadcast business models, they just want their content.
For advertisers: Brands have never wanted to buy CPMs (cost per thousand impressions) or GRPs (gross rating points); they want to sell stuff. The data-rich FANG and other tech giants are offering data that can be turned directly into sales.
For networks: It’s just a matter of time before media without actionable data will be impossible to monetize. Can traditional TV catch up? Adapt or die!