Archiv der Kategorie: Privacy

Steve Rymell Head of Technology, Airbus CyberSecurity answers What Should Frighten us about AI-Based Malware?

 

Of all the cybersecurity industry’s problems, one of the most striking is the way attackers are often able to stay one step ahead of defenders without working terribly hard. It’s an issue whose root causes are mostly technical: the prime example are software vulnerabilities which cyber-criminals have a habit of finding out about before vendors and their customers, leading to the almost undefendable zero-day phenomenon which has propelled many famous cyber-attacks.

A second is that organizations struggling with the complexity of unfamiliar and new technologies make mistakes, inadvertently leaving vulnerable ports and services exposed. Starkest of all, perhaps, is the way techniques, tools, and infrastructure set up to help organizations defend themselves (Shodan, for example but also numerous pen-test tools) are now just as likely to be turned against businesses by attackers who tear into networks with the aggression of red teams gone rogue.

Add to this the polymorphic nature of modern malware, and attackers can appear so conceptually unstoppable that it’s no wonder security vendors increasingly emphasize the need not to block attacks but instead respond to them as quickly as possible.

The AI fightback
Some years back, a list of mostly US-based start-ups started a bit of a counter-attack against the doom and gloom with a brave new idea – AI machine learning (ML) security powered by algorithms. In an age of big data, this makes complete sense and the idea has since been taken up by all manner of systems used to for anti-spam, malware detection, threat analysis and intelligence, and Security Operations Centre (SoC) automation where it has been proposed to help patch skills shortages.

I’d rate these as useful advances, but there’s no getting away from the controversial nature of the theory, which has been branded by some as the ultimate example of technology as a ‘black box’ nobody really understands. How do we know that machine learning is able to detect new and unknown types of attack that conventional systems fail to spot? In some cases, it could be because the product brochure says so.

Then the even bigger gotcha hits you – what’s stopping attackers from outfoxing defensive ML with even better ML of their own? If this were possible, even some of the time, the industry would find itself back at square one.

This is pure speculation, of course, because to date nobody has detected AI being used in a cyber-attack, which is why our understanding of how it might work remains largely based around academic research such as IBM’s proof-of-concept DeepLocker malware project.

What might malicious ML look like?
It would be unwise to ignore the potential for trouble. One of the biggest hurdles faced by attackers is quickly understanding what works, for example when sending spam, phishing and, increasingly, political disinformation.

It’s not hard to imagine that big data techniques allied to ML could hugely improve the efficiency of these threats by analyzing how targets react to and share them in real time. This implies the possibility that such campaigns might one day evolve in a matter of hours or minutes; a timescale defender would struggle to counter using today’s technologies.

A second scenario is one that defenders would even see: that cyber-criminals might simulate the defenses of a target using their own ML to gauge the success of different attacks (a technique already routinely used to evade anti-virus). Once again, this exploits the advantage that attackers always have sight of the target, while defenders must rely on good guesses.

Or perhaps ML could simply be used to crank out vast quantities of new and unique malware than is possible today. Whichever of these approaches is taken – and this is only a sample of the possibilities – it jumps out at you how awkward it would be to defend against even relatively simple ML-based attacks. About the only consolation is that if ML-based AI really is a black box that nobody understands then, logically, the attackers won’t understand it either and will waste time experimenting.

Unintended consequences
If we should fear anything it’s precisely this black box effect. There are two parts to this, the biggest of which is the potential for ML-based malware to cause something unintended to happen, especially when targeting critical infrastructure.

This phenomenon has already come to pass with non-AI malware – Stuxnet in 2010 and NotPetya in 2017 are the obvious examples – both of which infected thousands of organizations not on their original target list after unexpectedly ‘escaping’ into the wild.

When it comes to powerful malware exploiting multiple zero days there’s no such thing as a reliably contained attack. Once released, this kind of malware remains pathogenically dangerous until every system it can infect is patched or taken offline, which might be years or decades down the line.

Another anxiety is that because the expertise to understand ML is still thin on the ground, there’s a danger that engineers could come to rely on it without fully understanding its limitations, both for defense and by over-estimating its usefulness in attack. The mistake, then, might be that too many over-invest in it based on marketing promises that end up consuming resources better deployed elsewhere.  Once a more realistic assessment takes hold, ML could end up as just another tool that is good at solving certain very specific problems.

Conclusion
My contradictory-sounding conclusion is that perhaps ML and AI makes no fundamental difference at all. It’s just another stop on a journey computer security has been making since the beginning of digital time. The problem is overcoming our preconceptions about what it is and what it means. Chiefly, we must overcome the tendency to think of ML and AI as mysteriously ‘other’ because we don’t understand it and therefore find it difficult to process the concept of machines making complex decisions.

It’s not as if attackers aren’t breaching networks already with today’s pre-ML technology or that well-prepared defenders aren’t regularly stopping them using the same technology. What AI reminds us is that the real difference is how organizations are defended, not whether they or their attackers use ML and AI or not. That has always been what separates survivors from victims. Cybersecurity remains a working demonstration of how the devil takes the hindmost.

Source: https://www.infosecurity-magazine.com/opinions/frighten-ai-malware-1/

Werbeanzeigen

Do you know who your iPhone is talking to?

 

https://www.washingtonpost.com/technology/2019/05/28/its-middle-night-do-you-know-who-your-iphone-is-talking/?noredirect=on

Yet these days, we spend more time in apps. Apple is strict about requiring apps to get permission to access certain parts of the iPhone, including your camera, microphone, location, health information, photos and contacts. (You can check and change those permissions under privacy settings.) But Apple turns more of a blind eye to what apps do with data we provide them or they generate about us — witness the sorts of tracking I found by looking under the covers for a few days.

“For the data and services that apps create on their own, our App Store Guidelines require developers to have clearly posted privacy policies and to ask users for permission to collect data before doing so. When we learn that apps have not followed our Guidelines in these areas, we either make apps change their practice or keep those apps from being on the store,” Apple says.

Yet very few apps I found using third-party trackers disclosed the names of those companies or how they protect my data. And what good is burying this information in privacy policies, anyway? What we need is accountability.

Getting more deeply involved in app data practices is complicated for Apple. Today’s technology frequently is built on third-party services, so Apple couldn’t simply ban all connections to outside servers. And some companies are so big they don’t even need the help of outsiders to track us.

The result shouldn’t be to increase Apple’s power. “I would like to make sure they’re not stifling innovation,” says Andrés Arrieta, the director of consumer privacy engineering at the Electronic Frontier Foundation. If Apple becomes the Internet’s privacy police, it could shut down rivals.

Jackson suggests Apple could also add controls into iOS like the ones built into Privacy Pro to give everyone more visibility.

Or perhaps Apple could require apps to label when they’re using third-party trackers. If I opened the DoorDash app and saw nine tracker notices, it might make me think twice about using it.

I don’t mind letting your trackers see my private data as long as I get something useful in exchange.

Forget privacy: you’re terrible at targeting anyway

I don’t mind letting your programs see my private data as long as I get something useful in exchange. But that’s not what happens.

A former co-worker told me once: „Everyone loves collecting data, but nobody loves analyzing it later.“ This claim is almost shocking, but people who have been involved in data collection and analysis have all seen it. It starts with a brilliant idea: we’ll collect information about every click someone makes on every page in our app! And we’ll track how long they hesitate over a particular choice! And how often they use the back button! How many seconds they watch our intro video before they abort! How many times they reshare our social media post!

And then they do track all that. Tracking it all is easy. Add some log events, dump them into a database, off we go.

But then what? Well, after that, we have to analyze it. And as someone who has analyzed a lot of data about various things, let me tell you: being a data analyst is difficult and mostly unrewarding (except financially).

See, the problem is there’s almost no way to know if you’re right. (It’s also not clear what the definition of „right“ is, which I’ll get to in a bit.) There are almost never any easy conclusions, just hard ones, and the hard ones are error prone. What analysts don’t talk about is how many incorrect charts (and therefore conclusions) get made on the way to making correct ones. Or ones we think are correct. A good chart is so incredibly persuasive that it almost doesn’t even matter if it’s right, as long as what you want is to persuade someone… which is probably why newpapers, magazines, and lobbyists publish so many misleading charts.

But let’s leave errors aside for the moment. Let’s assume, very unrealistically, that we as a profession are good at analyzing things. What then?

Well, then, let’s get rich on targeted ads and personalized recommendation algorithms. It’s what everyone else does!

Or do they?

The state of personalized recommendations is surprisingly terrible. At this point, the top recommendation is always a clickbait rage-creating article about movie stars or whatever Trump did or didn’t do in the last 6 hours. Or if not an article, then a video or documentary. That’s not what I want to read or to watch, but I sometimes get sucked in anyway, and then it’s recommendation apocalypse time, because the algorithm now thinks I like reading about Trump, and now everything is Trump. Never give positive feedback to an AI.

This is, by the way, the dirty secret of the machine learning movement: almost everything produced by ML could have been produced, more cheaply, using a very dumb heuristic you coded up by hand, because mostly the ML is trained by feeding it examples of what humans did while following a very dumb heuristic. There’s no magic here. If you use ML to teach a computer how to sort through resumes, it will recommend you interview people with male, white-sounding names, because it turns out that’s what your HR department already does. If you ask it what video a person like you wants to see next, it will recommend some political propaganda crap, because 50% of the time 90% of the people do watch that next, because they can’t help themselves, and that’s a pretty good success rate.

(Side note: there really are some excellent uses of ML out there, for things traditional algorithms are bad at, like image processing or winning at strategy games. That’s wonderful, but chances are good that your pet ML application is an expensive replacement for a dumb heuristic.)

Someone who works on web search once told me that they already have an algorithm that guarantees the maximum click-through rate for any web search: just return a page full of porn links. (Someone else said you can reverse this to make a porn detector: any link which has a high click-through rate, regardless of which query it’s answering, is probably porn.)

Now, the thing is, legitimate-seeming businesses can’t just give you porn links all the time, because that’s Not Safe For Work, so the job of most modern recommendation algorithms is to return the closest thing to porn that is still Safe For Work. In other words, celebrities (ideally attractive ones, or at least controversial ones), or politics, or both. They walk that line as closely as they can, because that’s the local maximum for their profitability. Sometimes they accidentally cross that line, and then have to apologize or pay a token fine, and then go back to what they were doing.

This makes me sad, but okay, it’s just math. And maybe human nature. And maybe capitalism. Whatever. I might not like it, but I understand it.

My complaint is that none of the above had anything to do with hoarding my personal information.

The hottest recommendations have nothing to do with me

Let’s be clear: the best targeted ads I will ever see are the ones I get from a search engine when it serves an ad for exactly the thing I was searching for. Everybody wins: I find what I wanted, the vendor helps me buy their thing, and the search engine gets paid for connecting us. I don’t know anybody who complains about this sort of ad. It’s a good ad.

And it, too, had nothing to do with my personal information!

Google was serving targeted search ads decades ago, before it ever occurred to them to ask me to log in. Even today you can still use every search engine web site without logging in. They all still serve ads targeted to your search keyword. It’s an excellent business.

There’s another kind of ad that works well on me. I play video games sometimes, and I use Steam, and sometimes I browse through games on Steam and star the ones I’m considering buying. Later, when those games go on sale, Steam emails me to tell me they are on sale, and sometimes then I buy them. Again, everybody wins: I got a game I wanted (at a discount!), the game maker gets paid, and Steam gets paid for connecting us. And I can disable the emails if I want, but I don’t want, because they are good ads.

But nobody had to profile me to make that happen! Steam has my account, and I told it what games I wanted and then it sold me those games. That’s not profiling, that’s just remembering a list that I explicitly handed to you.

Amazon shows a box that suggests I might want to re-buy certain kinds of consumable products that I’ve bought in the past. This is useful too, and requires no profiling other than remembering the transactions we’ve had with each other in the past, which they kinda have to do anyway. And again, everybody wins.

Now, Amazon also recommends products like the ones I’ve bought before, or looked at before. That’s, say, 20% useful. If I just bought a computer monitor, and you know I did because I bought it from you, then you might as well stop selling them to me. But for a few days after I buy any electronics they also keep offering to sell me USB cables, and they’re probably right. So okay, 20% useful targeting is better than 0% useful. I give Amazon some credit for building a useful profile of me, although it’s specifically a profile of stuff I did on their site and which they keep to themselves. That doesn’t seem too invasive. Nobody is surprised that Amazon remembers what I bought or browsed on their site.

Worse is when (non-Amazon) vendors get the idea that I might want something. (They get this idea because I visited their web site and looked at it.) So their advertising partner chases me around the web trying to sell me the same thing. They do that, even if I already bought it. Ironically, this is because of a half-hearted attempt to protect my privacy. The vendor doesn’t give information about me or my transactions to their advertising partner (because there’s an excellent chance it would land them in legal trouble eventually), so the advertising partner doesn’t know that I bought it. All they know (because of the advertising partner’s tracker gadget on the vendor’s web site) is that I looked at it, so they keep advertising it to me just in case.

But okay, now we’re starting to get somewhere interesting. The advertiser has a tracker that it places on multiple sites and tracks me around. So it doesn’t know what I bought, but it does know what I looked at, probably over a long period of time, across many sites.

Using this information, its painstakingly trained AI makes conclusions about which other things I might want to look at, based on…

…well, based on what? People similar to me? Things my Facebook friends like to look at? Some complicated matrix-driven formula humans can’t possibly comprehend, but which is 10% better?

Probably not. Probably what it does is infer my gender, age, income level, and marital status. After that, it sells me cars and gadgets if I’m a guy, and fashion if I’m a woman. Not because all guys like cars and gadgets, but because some very uncreative human got into the loop and said „please sell my car mostly to men“ and „please sell my fashion items mostly to women.“ Maybe the AI infers the wrong demographic information (I know Google has mine wrong) but it doesn’t really matter, because it’s usually mostly right, which is better than 0% right, and advertisers get some mostly demographically targeted ads, which is better than 0% targeted ads.

You know this is how it works, right? It has to be. You can infer it from how bad the ads are. Anyone can, in a few seconds, think of some stuff they really want to buy which The Algorithm has failed to offer them, all while Outbrain makes zillions of dollars sending links about car insurance to non-car-owning Manhattanites. It might as well be a 1990s late-night TV infomercial, where all they knew for sure about my demographic profile is that I was still awake.

You tracked me everywhere I go, logging it forever, begging for someone to steal your database, desperately fearing that some new EU privacy regulation might destroy your business… for this?

Statistical Astrology

Of course, it’s not really as simple as that. There is not just one advertising company tracking me across every web site I visit. There are… many advertising companies tracking me across every web site I visit. Some of them don’t even do advertising, they just do tracking, and they sell that tracking data to advertisers who supposedly use it to do better targeting.

This whole ecosystem is amazing. Let’s look at online news web sites. Why do they load so slowly nowadays? Trackers. No, not ads – trackers. They only have a few ads, which mostly don’t take that long to load. But they have a lot of trackers, because each tracker will pay them a tiny bit of money to be allowed to track each page view. If you’re a giant publisher teetering on the edge of bankruptcy and you have 25 trackers on your web site already, but tracker company #26 calls you and says they’ll pay you $50k a year if you add their tracker too, are you going to say no? Your page runs like sludge already, so making it 1/25th more sludgy won’t change anything, but that $50k might.

(„Ad blockers“ remove annoying ads, but they also speed up the web, mostly because they remove trackers. Embarrassingly, the trackers themselves don’t even need to cause a slowdown, but they always do, because their developers are invariably idiots who each need to load thousands of lines of javascript to do what could be done in two. But that’s another story.)

Then the ad sellers, and ad networks, buy the tracking data from all the trackers. The more tracking data they have, the better they can target ads, right? I guess.

The brilliant bit here is that each of the trackers has a bit of data about you, but not all of it, because not every tracker is on every web site. But on the other hand, cross-referencing individuals between trackers is kinda hard, because none of them wants to give away their secret sauce. So each ad seller tries their best to cross-reference the data from all the tracker data they buy, but it mostly doesn’t work. Let’s say there are 25 trackers each tracking a million users, probably with a ton of overlap. In a sane world we’d guess that there are, at most, a few million distinct users. But in an insane world where you can’t prove if there’s an overlap, it could be as many as 25 million distinct users! The more tracker data your ad network buys, the more information you have! Probably! And that means better targeting! Maybe! And so you should buy ads from our network instead of the other network with less data! I guess!

None of this works. They are still trying to sell me car insurance for my subway ride.

It’s not just ads

That’s a lot about profiling for ad targeting, which obviously doesn’t work, if anyone would just stop and look at it. But there are way too many people incentivized to believe otherwise. Meanwhile, if you care about your privacy, all that matters is they’re still collecting your personal information whether it works or not.

What about content recommendation algorithms though? Do those work?

Obviously not. I mean, have you tried them. Seriously.

That’s not quite fair. There are a few things that work. Pandora’s music recommendations are surprisingly good, but they are doing it in a very non-obvious way. The obvious way is to take the playlist of all the songs your users listen to, blast it all into an ML training dataset, and then use that to produce a new playlist for new users based on… uh… their… profile? Well, they don’t have a profile yet because they just joined. Perhaps based on the first few songs they select manually? Maybe, but they probably started with either a really popular song, which tells you nothing, or a really obscure song to test the thoroughness of your library, which tells you less than nothing.

(I’m pretty sure this is how Mixcloud works. After each mix, it tries to find the „most similar“ mix to continue with. Usually this is someone else’s upload of the exact same mix. Then the „most similar“ mix to that one is the first one, so it does that. Great job, machine learning, keep it up.)

That leads us to the „random song followed by thumbs up/down“ system that everyone uses. But everyone sucks, except Pandora. Why? Apparently because Pandora spent a lot of time hand-coding a bunch of music characteristics and writing a „real algorithm“ (as opposed to ML) that tries to generate playlists based on the right combinations of those characteristics.

In that sense, Pandora isn’t pure ML. It often converges on a playlist you’ll like within one or two thumbs up/down operations, because you’re navigating through a multidimensional interconnected network of songs that people encoded the hard way, not a massive matrix of mediocre playlists scraped from average people who put no effort into generating those playlists in the first place. Pandora is bad at a lot of things (especially „availability in Canada“) but their music recommendations are top notch.

Just one catch. If Pandora can figure out a good playlist based on a starter song and one or two thumbs up/down clicks, then… I guess it’s not profiling you. They didn’t need your personal information either.

Netflix

While we’re here, I just want to rant about Netflix, which is an odd case of starting off with a really good recommendation algorithm and then making it worse on purpose.

Once upon a time, there was the Netflix prize, which granted $1 million to the best team that could predict people’s movie ratings, based on their past ratings, with better accuracy than Netflix could themselves. (This not-so-shockingly resulted in a privacy fiasco when it turned out you could de-anonymize the data set that they publicly released, oops. Well, that’s what you get when you long-term store people’s personal information in a database.)

Netflix believed their business depended on a good recommendation algorithm. It was already pretty good: I remember using Netflix around 10 years ago and getting several recommendations for things I would never have discovered, but which I turned out to like. That hasn’t happened to me on Netflix in a long, long time.

As the story goes, once upon a time Netflix was a DVD-by-mail service. DVD-by-mail is really slow, so it was absolutely essential that at least one of this week’s DVDs was good enough to entertain you for your Friday night movie. Too many Fridays with only bad movies, and you’d surely unsubscribe. A good recommendation system was key. (I guess there was also some interesting math around trying to make sure to rent out as much of the inventory as possible each week, since having a zillion copies of the most recent blockbuster, which would be popular this month and then die out next month, was not really viable.)

Eventually though, Netflix moved online, and the cost of a bad recommendation was much less: just stop watching and switch to a new movie. Moreover, it was perfectly fine if everyone watched the same blockbuster. In fact, it was better, because they could cache it at your ISP and caches always work better if people are boring and average.

Worse, as the story goes, Netflix noticed a pattern: the more hours people watch, the less likely they are to cancel. (This makes sense: the more hours you spend on Netflix, the more you feel like you „need“ it.) And with new people trying the service at a fixed or proportional rate, higher retention translates directly to faster growth.

When I heard this was also when I learned the word „satisficing,“ which essentially means searching through sludge not for the best option, but for a good enough option. Nowadays Netflix isn’t about finding the best movie, it’s about satisficing. If it has the choice between an award-winning movie that you 80% might like or 20% might hate, and a mainstream movie that’s 0% special but you 99% won’t hate, it will recommend the second one every time. Outliers are bad for business.

The thing is, you don’t need a risky, privacy-invading profile to recommend a mainstream movie. Mainstream movies are specially designed to be inoffensive to just about everyone. My Netflix recommendations screen is no longer „Recommended for you,“ it’s „New Releases,“ and then „Trending Now,“ and „Watch it again.“

As promised, Netflix paid out their $1 million prize to buy the winning recommendation algorithm, which was even better than their old one. But they didn’t use it, they threw it away.

Some very expensive A/B testers determined that this is what makes me watch the most hours of mindless TV. Their revenues keep going up. And they don’t even need to invade my privacy to do it.

Who am I to say they’re wrong?

https://apenwarr.ca/log/20190201

Tim Cook: The Genius Who Took Apple to the Next Level

 

 

Excerpted from Tim Cook: The Genius Who Took Apple to the Next Level

 

They knew that they had to respond immediately. The writ would dominate the next day’s news, and Apple had to have a response. “Tim knew that this was a massive decision on his part,” Sewell said. It was a big moment, “a bet-the-company kind of decision.” Cook and the team stayed up all night—a straight 16 hours—working on their response. Cook already knew his position—Apple would refuse—but he wanted to know all the angles: What was Apple’s legal position? What was its legal obligation? Was this the right response? How should it sound? How should it read? What was the right tone?

iOS 8 added much stronger encryption than had been seen before in smartphones. It encrypted all the user’s data—phone call records, messages, photos, contacts, and so on—with the user’s passcode. The encryption was so strong, not even Apple could break it. Security on earlier devices was much weaker, and there were various ways to break into them, but Apple could no longer access locked devices running iOS 8, even if law enforcement had a valid warrant. “Unlike our competitors, Apple cannot bypass your passcode and therefore cannot access this data,” the company wrote on its website. “So it’s not technically feasible for us to respond to government warrants for the extraction of this data from devices in their possession running iOS 8.”

The War Room

For the next two months, the executive floor at One Infinite Loop turned into a 24/7 situation room, with staffers sending out messages and responding to journalists’ queries. One PR rep said that they were sometimes sending out multiple updates a day with up to 700 journalists cc’d on the emails. This is in stark contrast to Apple’s usual PR strategy, which consists of occasional press releases and routinely ignoring reporters’ calls and emails.

Cook also felt he had to rally the troops, to keep morale high at a time when the company was under attack. In an email to Apple employees, titled “Thank you for your support,” he wrote, “This case is about much more than a single phone or a single investigation.” He continued, “At stake is the data security of hundreds of millions of law-abiding people and setting a dangerous precedent that threatens everyone’s civil liberties.” It worked. Apple employees trusted their leader to make the decision that was right not only for them but also for the general public.

Cook was very concerned about how Apple would be perceived throughout this media firestorm. He wanted very much to use it as an opportunity to educate the public about personal security, privacy, and encryption. “I think a lot of reporters saw a new version, a new face of Apple,” said the PR person, who asked to remain anonymous. “And it was Tim’s decision to act in this fashion. Very different from what we have done in the past. We were sometimes sending out emails to reporters three times a day on keeping them updated.”

Outside Apple’s walls, Cook went on a charm offensive. Eight days after publishing his privacy letter, he sat down for a prime-time interview with ABC News. Sitting in his office at One Infinite Loop, he sincerely explained Apple’s position. It was the “most important [interview] he’s given as Apple’s CEO,” said the Washington Post. “Cook responded to questions with a raw conviction that was even more emphatic than usual,” wrote the paper. “He used sharp and soaring language, calling the request the ‘software equivalent of cancer’ and talking about ‘fundamental’ civil liberties.

https://www.wired.com/story/the-time-tim-cook-stood-his-ground-against-fbi/

Germany bans Facebook from combining user data without permission

Germany’s Federal Cartel Office, or Bundeskartellamt, on Thursday banned Facebook from combining user data from its various platforms such as WhatsApp and Instagram without explicit user permission.

The decision, which comes as the result of a nearly three-year antitrust investigation into Facebook’s data gathering practices, also bans the social media company from gleaning user data from third-party sites unless they voluntarily consent.

“With regard to Facebook’s future data processing policy, we are carrying out what can be seen as an internal divestiture of Facebook’s data,” Bundeskartellamt President Andreas Mundt said in a release. “In [the] future, Facebook will no longer be allowed to force its users to agree to the practically unrestricted collection and assigning of non-Facebook data to their Facebook user accounts.”

Mundt noted that combining user data from various sources “substantially contributed to the fact that Facebook was able to build a unique database for each individual user and thus to gain market power.”

Experts agreed with the decision. “It is high time to regulate the internet giants effectively!” said Marc Al-Hames, general manager of German data protection technologies developer Cliqz GmbH. “Unregulated data capitalism inevitably creates unfair conditions.”

Al-Hames noted that apps like WhatsApp have become “indispensable for many young people,” who feel compelled to join if they want to be part of the social scene. “Social media create social pressure,” he said. “And Facebook exploits this mercilessly: Give me your data or you’re an outsider.”

He called the practice an abuse of dominant market position. “But that’s not all: Facebook monitors our activities regardless of whether we are a member of one of its networks or not. Even those who consciously renounce the social networks for the sake of privacy will still be spied out,” he said, adding that Cliqz and Ghostery stats show that “every fourth of our website visits are monitored by Facebook’s data collection technologies, so-called trackers.”

The Bundeskartellamt’s decision will prevent Facebook from collecting and using data without restriction. “Voluntary consent means that the use of Facebook’s services must [now] be subject to the users’ consent to their data being collected and combined in this way,” said Mundt. “If users do not consent, Facebook may not exclude them from its services and must refrain from collecting and merging data from different sources.”

The ban drew support and calls for it to be expanded to other companies.

“This latest move by Germany’s competition regulator is welcome,” said Morten Brøgger, CEO of secure collaboration platform Wire. “Compromising user privacy for profit is a risk no exec should be willing to take.”

Brøgger contends that Facebook has not fully understood digital privacy’s importance. “From emails suggesting cashing in on user data for money, to the infamous Cambridge Analytica scandal, the company is taking steps back in a world which is increasingly moving towards the protection of everyone’s data,” he said.

“The lesson here is that you cannot simply trust firms that rely on the exchange of data as its main offering, Brøgger added, “and firms using Facebook-owned applications should have a rethink about the platforms they use to do business.”

Al-Hames said regulators shouldn’t stop with Facebook, which he called the number-two offender. “By far the most important data monopolist is Alphabet. With Google search, the Android operating system, the Play Store app sales platform and the Chrome browser, the internet giant collects data on virtually everyone in the Western world,” Al-Hames said. “And even those who want to get free by using alternative services stay trapped in Alphabet’s clutches: With a tracker reach of nearly 80 percent of all page loads Alphabet probably knows more about them than their closest friends or relatives. When it comes to our data, the top priority of the market regulators shouldn’t be Facebook, it should be Alphabet!”

Source: https://www.scmagazine.com/home/network-security/germany-bans-facebook-from-combining-user-data-without-permission/

Apple Glassboxes IOS Apps to remove screen recording code

Pedestrians pass in front of a billboard advertising Apple Inc. iPhone security during the 2019 Consumer Electronics Show (CES) in Las Vegas, Nevada, U.S., on Monday, Jan. 7, 2019. Apple made its presence felt at CES 2019 with a massive billboard highlighting the iPhone’s privacy features. Source: Photographer: David Paul Morris/Bloomberg via Getty Images

Apple is telling app developers to remove or properly disclose their use of analytics code that allows them to record how a user interacts with their iPhone apps — or face removal from the app store, TechCrunch can confirm.

In an email, an Apple spokesperson said: “Protecting user privacy is paramount in the Apple ecosystem. Our App Store Review Guidelines require that apps request explicit user consent and provide a clear visual indication when recording, logging, or otherwise making a record of user activity.”

“We have notified the developers that are in violation of these strict privacy terms and guidelines, and will take immediate action if necessary,” the spokesperson added.

It follows an investigation by TechCrunch that revealed major companies, like Expedia, Hollister and Hotels.com, were using a third-party analytics tool to record every tap and swipe inside the app. We found that none of the apps we tested asked the user for permission, and none of the companies said in their privacy policies that they were recording a user’s app activity.

Even though sensitive data is supposed to be masked, some data — like passport numbers and credit card numbers — was leaking.

Glassbox is a cross-platform analytics tool that specializes in session replay technology. It allows companies to integrate its screen recording technology into their apps to replay how a user interacts with the apps. Glassbox says it provides the technology, among many reasons, to help reduce app error rates. But the company “doesn’t enforce its customers” to mention that they use Glassbox’s screen recording tools in their privacy policies.

But Apple expressly forbids apps that covertly collect data without a user’s permission.

TechCrunch began hearing on Thursday that app developers had already been notified that their apps had fallen afoul of Apple’s rules. One app developer was told by Apple to remove code that recorded app activities, citing the company’s app store guidelines.

“Your app uses analytics software to collect and send user or device data to a third party without the user’s consent. Apps must request explicit user consent and provide a clear visual indication when recording, logging, or otherwise making a record of user activity,” Apple said in the email.

Apple gave the developer less than a day to remove the code and resubmit their app or the app would be removed from the app store, the email said.

When asked if Glassbox was aware of the app store removals, a spokesperson for Glassbox said that “the communication with Apple is through our customers.”

Glassbox is also available to Android app developers. Google did not immediately comment if it would also ban the screen recording code. Google Play also expressly prohibits apps from secretly collecting device usage. “Apps must not hide or cloak tracking behavior or attempt to mislead users about such functionality,” the developer rules state. We’ll update if and when we hear back.

It’s the latest privacy debacle that has forced Apple to wade in to protect its customers after apps were caught misbehaving.

Last week, TechCrunch reported that Apple banned Facebook’s “research” app that the social media giant paid teenagers to collect all of their data.

It followed another investigation by TechCrunch that revealed Facebook misused its Apple-issued enterprise developer certificate to build and provide apps for consumers outside Apple’s App Store. Apple temporarily revoked Facebook’s enterprise developer certificate, knocking all of the company’s internal iOS apps offline for close to a day.

Source: https://techcrunch.com/2019/02/07/apple-glassbox-apps/

Pedestrians pass in front of a billboard advertising Apple Inc. iPhone security during the 2019 Consumer Electronics Show (CES) in Las Vegas, Nevada, U.S., on Monday, Jan. 7, 2019. Apple made its presence felt at CES 2019 with a massive billboard highlighting the iPhone’s privacy features. Source: Photographer: David Paul Morris/Bloomberg via Getty Images

Your Apps Know Where You Were Last Night, and They’re Not Keeping It Secret

Dozens of companies use smartphone locations to help advertisers and even hedge funds. They say it’s anonymous, but the data shows how personal it is.

The millions of dots on the map trace highways, side streets and bike trails — each one following the path of an anonymous cellphone user.

One path tracks someone from a home outside Newark to a nearby Planned Parenthood, remaining there for more than an hour. Another represents a person who travels with the mayor of New York during the day and returns to Long Island at night.

Yet another leaves a house in upstate New York at 7 a.m. and travels to a middle school 14 miles away, staying until late afternoon each school day. Only one person makes that trip: Lisa Magrin, a 46-year-old math teacher. Her smartphone goes with her.

An app on the device gathered her location information, which was then sold without her knowledge. It recorded her whereabouts as often as every two seconds, according to a database of more than a million phones in the New York area that was reviewed by The New York Times. While Ms. Magrin’s identity was not disclosed in those records, The Times was able to easily connect her to that dot.

The app tracked her as she went to a Weight Watchers meeting and to her dermatologist’s office for a minor procedure. It followed her hiking with her dog and staying at her ex-boyfriend’s home, information she found disturbing.

“It’s the thought of people finding out those intimate details that you don’t want people to know,” said Ms. Magrin, who allowed The Times to review her location data.

Like many consumers, Ms. Magrin knew that apps could track people’s movements. But as smartphones have become ubiquitous and technology more accurate, an industry of snooping on people’s daily habits has spread and grown more intrusive.

 

At least 75 companies receive anonymous, precise location data from apps whose users enable location services to get local news and weather or other information, The Times found. Several of those businesses claim to track up to 200 million mobile devices in the United States — about half those in use last year. The database reviewed by The Times — a sample of information gathered in 2017 and held by one company — reveals people’s travels in startling detail, accurate to within a few yards and in some cases updated more than 14,000 times a day.

[Learn how to stop apps from tracking your location.]

These companies sell, use or analyze the data to cater to advertisers, retail outlets and even hedge funds seeking insights into consumer behavior. It’s a hot market, with sales of location-targeted advertising reaching an estimated $21 billion this year. IBM has gotten into the industry, with its purchase of the Weather Channel’s apps. The social network Foursquare remade itself as a location marketing company. Prominent investors in location start-ups include Goldman Sachs and Peter Thiel, the PayPal co-founder.

Businesses say their interest is in the patterns, not the identities, that the data reveals about consumers. They note that the information apps collect is tied not to someone’s name or phone number but to a unique ID. But those with access to the raw data — including employees or clients — could still identify a person without consent. They could follow someone they knew, by pinpointing a phone that regularly spent time at that person’s home address. Or, working in reverse, they could attach a name to an anonymous dot, by seeing where the device spent nights and using public records to figure out who lived there.

Many location companies say that when phone users enable location services, their data is fair game. But, The Times found, the explanations people see when prompted to give permission are often incomplete or misleading. An app may tell users that granting access to their location will help them get traffic information, but not mention that the data will be shared and sold. That disclosure is often buried in a vague privacy policy.

“Location information can reveal some of the most intimate details of a person’s life — whether you’ve visited a psychiatrist, whether you went to an A.A. meeting, who you might date,” said Senator Ron Wyden, Democrat of Oregon, who has proposed bills to limit the collection and sale of such data, which are largely unregulated in the United States.

“It’s not right to have consumers kept in the dark about how their data is sold and shared and then leave them unable to do anything about it,” he added.

Mobile Surveillance Devices

After Elise Lee, a nurse in Manhattan, saw that her device had been tracked to the main operating room at the hospital where she works, she expressed concern about her privacy and that of her patients.

“It’s very scary,” said Ms. Lee, who allowed The Times to examine her location history in the data set it reviewed. “It feels like someone is following me, personally.”

The mobile location industry began as a way to customize apps and target ads for nearby businesses, but it has morphed into a data collection and analysis machine.

Retailers look to tracking companies to tell them about their own customers and their competitors’. For a web seminar last year, Elina Greenstein, an executive at the location company GroundTruth, mapped out the path of a hypothetical consumer from home to work to show potential clients how tracking could reveal a person’s preferences. For example, someone may search online for healthy recipes, but GroundTruth can see that the person often eats at fast-food restaurants.

“We look to understand who a person is, based on where they’ve been and where they’re going, in order to influence what they’re going to do next,” Ms. Greenstein said.

Financial firms can use the information to make investment decisions before a company reports earnings — seeing, for example, if more people are working on a factory floor, or going to a retailer’s stores.

 

Health care facilities are among the more enticing but troubling areas for tracking, as Ms. Lee’s reaction demonstrated. Tell All Digital, a Long Island advertising firm that is a client of a location company, says it runs ad campaigns for personal injury lawyers targeting people anonymously in emergency rooms.

“The book ‘1984,’ we’re kind of living it in a lot of ways,” said Bill Kakis, a managing partner at Tell All.

Jails, schools, a military base and a nuclear power plant — even crime scenes — appeared in the data set The Times reviewed. One person, perhaps a detective, arrived at the site of a late-night homicide in Manhattan, then spent time at a nearby hospital, returning repeatedly to the local police station.

Two location firms, Fysical and SafeGraph, mapped people attending the 2017 presidential inauguration. On Fysical’s map, a bright red box near the Capitol steps indicated the general location of President Trump and those around him, cellphones pinging away. Fysical’s chief executive said in an email that the data it used was anonymous. SafeGraph did not respond to requests for comment.

 

More than 1,000 popular apps contain location-sharing code from such companies, according to 2018 data from MightySignal, a mobile analysis firm. Google’s Android system was found to have about 1,200 apps with such code, compared with about 200 on Apple’s iOS.

The most prolific company was Reveal Mobile, based in North Carolina, which had location-gathering code in more than 500 apps, including many that provide local news. A Reveal spokesman said that the popularity of its code showed that it helped app developers make ad money and consumers get free services.

To evaluate location-sharing practices, The Times tested 20 apps, most of which had been flagged by researchers and industry insiders as potentially sharing the data. Together, 17 of the apps sent exact latitude and longitude to about 70 businesses. Precise location data from one app, WeatherBug on iOS, was received by 40 companies. When contacted by The Times, some of the companies that received that data described it as “unsolicited” or “inappropriate.”

WeatherBug, owned by GroundTruth, asks users’ permission to collect their location and tells them the information will be used to personalize ads. GroundTruth said that it typically sent the data to ad companies it worked with, but that if they didn’t want the information they could ask to stop receiving it.

The Times also identified more than 25 other companies that have said in marketing materials or interviews that they sell location data or services, including targeted advertising.

[Read more about how The Times analyzed location tracking companies.]

The spread of this information raises questions about how securely it is handled and whether it is vulnerable to hacking, said Serge Egelman, a computer security and privacy researcher affiliated with the University of California, Berkeley.

“There are really no consequences” for companies that don’t protect the data, he said, “other than bad press that gets forgotten about.”

A Question of Awareness

Companies that use location data say that people agree to share their information in exchange for customized services, rewards and discounts. Ms. Magrin, the teacher, noted that she liked that tracking technology let her record her jogging routes.

Brian Wong, chief executive of Kiip, a mobile ad firm that has also sold anonymous data from some of the apps it works with, says users give apps permission to use and share their data. “You are receiving these services for free because advertisers are helping monetize and pay for it,” he said, adding, “You would have to be pretty oblivious if you are not aware that this is going on.”

But Ms. Lee, the nurse, had a different view. “I guess that’s what they have to tell themselves,” she said of the companies. “But come on.”

Ms. Lee had given apps on her iPhone access to her location only for certain purposes — helping her find parking spaces, sending her weather alerts — and only if they did not indicate that the information would be used for anything else, she said. Ms. Magrin had allowed about a dozen apps on her Android phone access to her whereabouts for services like traffic notifications.

But it is easy to share information without realizing it. Of the 17 apps that The Times saw sending precise location data, just three on iOS and one on Android told users in a prompt during the permission process that the information could be used for advertising. Only one app, GasBuddy, which identifies nearby gas stations, indicated that data could also be shared to “analyze industry trends.”

More typical was theScore, a sports app: When prompting users to grant access to their location, it said the data would help “recommend local teams and players that are relevant to you.” The app passed precise coordinates to 16 advertising and location companies.

A spokesman for theScore said that the language in the prompt was intended only as a “quick introduction to certain key product features” and that the full uses of the data were described in the app’s privacy policy.

The Weather Channel app, owned by an IBM subsidiary, told users that sharing their locations would let them get personalized local weather reports. IBM said the subsidiary, the Weather Company, discussed other uses in its privacy policy and in a separate “privacy settings” section of the app. Information on advertising was included there, but a part of the app called “location settings” made no mention of it.

The app did not explicitly disclose that the company had also analyzed the data for hedge funds — a pilot program that was promoted on the company’s website. An IBM spokesman said the pilot had ended. (IBM updated the app’s privacy policy on Dec. 5, after queries from The Times, to say that it might share aggregated location data for commercial purposes such as analyzing foot traffic.)

Even industry insiders acknowledge that many people either don’t read those policies or may not fully understand their opaque language. Policies for apps that funnel location information to help investment firms, for instance, have said the data is used for market analysis, or simply shared for business purposes.

“Most people don’t know what’s going on,” said Emmett Kilduff, the chief executive of Eagle Alpha, which sells data to financial firms and hedge funds. Mr. Kilduff said responsibility for complying with data-gathering regulations fell to the companies that collected it from people.

Many location companies say they voluntarily take steps to protect users’ privacy, but policies vary widely.

For example, Sense360, which focuses on the restaurant industry, says it scrambles data within a 1,000-foot square around the device’s approximate home location. Another company, Factual, says that it collects data from consumers at home, but that its database doesn’t contain their addresses.

Some companies say they delete the location data after using it to serve ads, some use it for ads and pass it along to data aggregation companies, and others keep the information for years.

Several people in the location business said that it would be relatively simple to figure out individual identities in this kind of data, but that they didn’t do it. Others suggested it would require so much effort that hackers wouldn’t bother.

It “would take an enormous amount of resources,” said Bill Daddi, a spokesman for Cuebiq, which analyzes anonymous location data to help retailers and others, and raised more than $27 million this year from investors including Goldman Sachs and Nasdaq Ventures. Nevertheless, Cuebiq encrypts its information, logs employee queries and sells aggregated analysis, he said.

There is no federal law limiting the collection or use of such data. Still, apps that ask for access to users’ locations, prompting them for permission while leaving out important details about how the data will be used, may run afoul of federal rules on deceptive business practices, said Maneesha Mithal, a privacy official at the Federal Trade Commission.

“You can’t cure a misleading just-in-time disclosure with information in a privacy policy,” Ms. Mithal said.

Following the Money

Apps form the backbone of this new location data economy.

The app developers can make money by directly selling their data, or by sharing it for location-based ads, which command a premium. Location data companies pay half a cent to two cents per user per month, according to offer letters to app makers reviewed by The Times.

Targeted advertising is by far the most common use of the information.

Google and Facebook, which dominate the mobile ad market, also lead in location-based advertising. Both companies collect the data from their own apps. They say they don’t sell it but keep it for themselves to personalize their services, sell targeted ads across the internet and track whether the ads lead to sales at brick-and-mortar stores. Google, which also receives precise location information from apps that use its ad services, said it modified that data to make it less exact.

Smaller companies compete for the rest of the market, including by selling data and analysis to financial institutions. This segment of the industry is small but growing, expected to reach about $250 million a year by 2020, according to the market research firm Opimas.

Apple and Google have a financial interest in keeping developers happy, but both have taken steps to limit location data collection. In the most recent version of Android, apps that are not in use can collect locations “a few times an hour,” instead of continuously.

Apple has been stricter, for example requiring apps to justify collecting location details in pop-up messages. But Apple’s instructions for writing these pop-ups do not mention advertising or data sale, only features like getting “estimated travel times.”

A spokesman said the company mandates that developers use the data only to provide a service directly relevant to the app, or to serve advertising that met Apple’s guidelines.

Apple recently shelved plans that industry insiders say would have significantly curtailed location collection. Last year, the company said an upcoming version of iOS would show a blue bar onscreen whenever an app not in use was gaining access to location data.

The discussion served as a “warning shot” to people in the location industry, David Shim, chief executive of the location company Placed, said at an industry event last year.

After examining maps showing the locations extracted by their apps, Ms. Lee, the nurse, and Ms. Magrin, the teacher, immediately limited what data those apps could get. Ms. Lee said she told the other operating-room nurses to do the same.

“I went through all their phones and just told them: ‘You have to turn this off. You have to delete this,’” Ms. Lee said. “Nobody knew.”

Source: https://www.nytimes.com/interactive/2018/12/10/business/location-data-privacy-apps.html?action=click&module=Top%20Stories&pgtype=Homepage

Planned Parenthood
Records show a device entering Gracie Mansion, the mayor’s residence, before traveling to a Y.M.C.A. in Brooklyn that the mayor frequents.
It travels to an event on Staten Island that the mayor attended. Later, it returns to a home on Long Island.

An app on Lisa Magrin’s cellphone collected her location information, which was then shared with other companies. The data revealed her daily habits, including hikes with her dog, Lulu. Nathaniel Brooks for The New York Times

A notice that Android users saw when theScore, a sports app, asked for access to their location data.

The Weather Channel app showed iPhone users this message when it first asked for their location data.

Nuclear plant

In the data set reviewed by The Times, phone locations are recorded in sensitive areas including the Indian Point nuclear plant near New York City. By Michael H. Keller | Satellite imagery by Mapbox and DigitalGlobe

Megachurch