Archiv der Kategorie: Artificial Intelligence

Why robots will soon be picking soft fruits and salad

London (CNN Business)

It takes a certain nimbleness to pick a strawberry or a salad. While crops like wheat and potatoes have been harvested mechanically for decades, many fruits and vegetables have proved resistant to automation. They are too easily bruised, or too hard for heavy farm machinery to locate.

But recently, technological developments and advances in machine learning have led to successful trials of more sensitive and dexterous robots, which use cameras and artificial intelligence to locate ripe fruit and handle it with care and precision.
Developed by engineers at the University of Cambridge, the Vegebot is the first robot that can identify and harvest iceberg lettuce — bringing hope to farmers that one of the most demanding crops for human pickers could finally be automated.
First, a camera scans the lettuce and, with the help of a machine learning algorithm trained on more than a thousand lettuce images, decides if it is ready for harvest. Then a second camera guides the picking cage on top of the plant without crushing it. Sensors feel when it is in the right position, and compressed air drives a blade through the stalk at a high force to get a clean cut.

The Vegebot uses machine learning to identify ripe, immature and diseased lettuce heads

Its success rate is high, with 91% of the crop accurately classified, according to a study published in July. But the robot is still much slower than humans, taking 31 seconds on average to pick one lettuce. Researchers say this could easily be sped up by using lighter materials.
Such adjustments would need to be made if the robot was used commercially. „Our goal was to prove you can do it, and we’ve done it,“ Simon Birrell, co-author of the study, tells CNN Business. „Now it depends on somebody taking the baton and running forward,“ he says.

More mouths to feed, but less manual labor

With the world’s population expected to climb to 9.7 billion in 2050 from 7.7 billion today — meaning roughly 80 million more mouths to feed each year — agriculture is under pressure to meet rising demand for food production.
Added pressures from climate change, such as extreme weather, shrinking agricultural lands and the depletion of natural resources, make innovation and efficiency all the more urgent.
This is one reason behind the industry’s drive to develop robotics. The global market for agricultural drones and robots is projected to grow from $2.5 billion in 2018 to $23 billion in 2028, according to a report from market intelligence firm BIS Research.
„Agriculture robots are expected to have a higher operating speed and accuracy than traditional agriculture machinery, which shall lead to significant improvements in production efficiency,“ Rakhi Tanwar, principal analyst of BIS Research, tells CNN Business.

Fruit picking robots like this one, developed by Fieldwork Robotics, operate for more than 20 hours a day

On top of this, growers are facing a long-term labor shortage. According to the World Bank, the share of total employment in agriculture in the world has declined from 43% in 1991 to 28% in 2018.
Tanwar says this is partly due to a lack of interest from younger generations. „The development of robotics in agriculture could lead to a massive relief to the growers who suffer from economic losses due to labor shortage,“ she says.
Robots can work all day and night, without stopping for breaks, and could be particularly useful during intense harvest periods.
„The main benefit is durability,“ says Martin Stoelen, a lecturer in robotics at the University of Plymouth and founder of Fieldwork Robotics, which has developed a raspberry-picking robot in partnership with Hall Hunter, one of the UK’s major berry growers.
Their robots, expected to go into production next year, will operate more than 20 hours a day and seven days a week during busy periods, „which human pickers obviously can’t do,“ says Stoelen.

Octinion's robot picks one strawberry every five seconds

Sustainable farming and food waste

Robots could also lead to more sustainable farming practices. They could enable growers to use less water, less fuel, and fewer pesticides, as well as producing less waste, says Tanwar.
At the moment, a field is typically harvested once, and any unripe fruits or vegetables are left to rot. Whereas, a robot could be trained to pick only ripe vegetables and, working around the clock, it could come back to the same field multiple times to pick any stragglers.
Birrell says that this will be the most important impact of robot pickers. „Right now, between a quarter and a third of food just rots in the field, and this is often because you don’t have humans ready at the right time to pick them,“ he says.
A successful example of this is the strawberry-picking robot developed by Octinion, a Belgium-based engineering startup.
The robot — which launched this year and is being used by growers in the UK and the Netherlands — is mounted on a self-driving trolley to serve table top strawberry production.
It uses 3D vision to locate the ripe berry, softly grips it with a pair of plastic pincers, and — just like a human — turns it 90 degrees to snap it from the stalk, before dropping it gently into a punnet.
„Robotics have the potential to convert the market from (being) supply-driven to demand-driven,“ says Tom Coen, CEO and founder of Octinion. „That will then help to reduce food waste and increase prices,“ he adds.

Harsh conditions

One major challenge with agricultural robots is adapting them for all-weather conditions. Farm machinery tends to be heavy-duty so that it can withstand rain, snow, mud, dust and heat.
„Building robots for agriculture is very different to building it for factories,“ says Birrell. „Until you’re out in the field, you don’t realize how robust it needs to be — it gets banged and crashed, you go over uneven surfaces, you get rained on, you get dust, you get lightning bolts.“
California-based Abundant Robotics has built an apple robot to endure the full range of farm conditions. It consists of an apple-sucking tube on a tractor-like contraption, which drives itself down an orchard row, while using computer vision to locate ripe fruit.
This spells the start of automation for orchard crops, says Dan Steere, CEO of Abundant Robotics. „Automation has steadily improved agricultural productivity for centuries,“ he says. „[We] have missed out on much of those benefits until now.“

Cybersecurity is one of the fastest-growing segments of the technology industry

Source: https://www.fool.com/investing/the-10-biggest-cybersecurity-stocks.aspx

The 10 Biggest Cybersecurity Stocks

When looking to invest in this high-growth tech industry, start with the biggest names on the cybersecurity block.

Cybersecurity is one of the fastest-growing segments of the technology industry. As more people around the globe connect to the internet and hundreds of millions of devices get connected to a network every year, the need to keep all of that data secure is on the rise.

In fact, according to research firm Global Market Insights, cybersecurity is expected to go from a $120 billion-a-year endeavor in 2017 to more than $300 billion in 2024, good for an average 12% annual growth rate. It’s no wonder, then, that so many businesses are getting in on the movement. Old tech titans like Microsoft (NASDAQ:MSFT), Cisco (NASDAQ:CSCO), and Oracle (NYSE:ORCL) all offer cybersecurity as part of their service suites. Other names are investing in the action, too. Old smartphone maker BlackBerry (NYSE:BB), for example, bought small cybersecurity outfit Cylance in early 2019 to further its transformation as a software company.

A silhouette of a person filled in with digital data, signifying artificial intelligence.

Image source: Getty Images.

As the world goes digital, managing new digital-first business operations and keeping information safe and secure will continue to evolve and grow in importance. For those wanting to invest in the cybersecurity industry, researching the biggest names in the business is a good place to get started (after brushing up on the basics here). Here are the 10 largest companies that make cybersecurity their primary concern based on market capitalization (the value of the company calculated by number of shares outstanding multiplied by price per share).

Company Market Capitalization as of July 2019 What the Company Does
1. Palo Alto Networks (NYSE:PANW) $21.3 billion A diversified provider of security solutions, with an increasing focus on cloud software
2. Splunk (NASDAQ:SPLK) $20.5 billion Big data analytics, including security orchestration and automated response
3. Check Point Software (NASDAQ:CHKP) $17.9 billion A diversified provider of security software and hardware
4. CrowdStrike (NASDAQ:CRWD) $17.5 billion Cloud-based endpoint security
5. Okta (NASDAQ:OKTA) $15.4 billion Cloud-based identity and privileged-access management software
6. Fortinet (NASDAQ:FTNT) $14.9 billion A diversified provider of security software and hardware
7. Symantec (NASDAQ:SYMC) $14.0 billion Largest security provider by revenue; owner of LifeLock and Norton Antivirus
8. Akamai Technologies (NASDAQ:AKAM) $13.6 billion Internet content delivery and security
9. Zscaler (NASDAQ:ZS) $10.4 billion Diversified provider of cloud-based security
10. F5 Networks (NASDAQ:FFIV) $8.7 billion Internet and application content delivery and security
Bonus: Proofpoint (NASDAQ:PFPT) $7.0 billion Employee communications and internet security

Data as of July 23, 2019. Data source: YCharts and company-specific investor relations.

Types of cybersecurity stocks

„Cybersecurity“ is the umbrella term, but there are different types of security firms tackling various problems in today’s connected age.

Broad-focus cybersecurity companies

For example, the larger outfits have been angling themselves to cover a wide range of needs, becoming one-stop security shops. Palo Alto Networks and Fortinet are two such companies, covering everything from firewalls (a network feature, sometimes a piece of hardware but more often software, that decides what data to let in and out) to artificial intelligence-based software that automates tasks and monitors an organization’s digital activity.

Endpoint security providers

These companies focus on securing remote devices connected to a network. The number of devices hooked up to the internet has been growing by the hundreds of millions every year, and that trend is expected to continue. Businesses are leading the charge, and everything from employee smartphones and tablets to assets in transit to connected machinery is in need of safekeeping. Endpoint protection software handles that specific need. Startup CrowdStrike, among others, is a specialist in this space.

Specialized security services

These niche companies include Okta, which provides privileged-access management — basically, only allowing users access to the sensitive data that they’re supposed to see. Then there’s security for the cloud, or computing and software that is offered remotely by way of a data center. Zscaler concerns itself with keeping cloud connections and data safe for businesses and organizations.

Regardless of the security need, digital-based operations and communications are on the rise across the board, which means all of the top cybersecurity companies are experiencing growth of some sort. That creates an opportunity for investors to cash in on the movement. Here is a breakdown of each of the top cybersecurity companies and how their stocks are valued.

The top 10 biggest cybersecurity stocks

1. Palo Alto Networks: The largest cybersecurity stock

Sitting atop the cybersecurity pure-play list is Palo Alto Networks. The company has built itself into the leader in the security space, offering a broad range of services for its customers from firewalls to automated threat response to cloud security. The largest player in the cybersecurity niche by market cap, Palo Alto has managed to outpace the industry’s average growth rate in spite of its size.

Part of the story behind Palo Alto’s growth is the company’s acquisition spree of smaller competitors. In May 2019, the company announced its intent to purchase two cloud-based cybersecurity outfits, one for $410 million and the other for a smaller undisclosed sum. Both were added to a new cloud security service segment called Prisma, aimed at continuously updating Palo Alto’s offerings as needs of customers evolve over time. CEO Nikesh Arora, a former executive at Alphabet’s (NASDAQ:GOOGL) (NASDAQ:GOOG) Google, has indicated that strategic acquisitions will continue to play an integral part in his company’s strategy to remain relevant.

The sums of money paid for acquisitions have been substantial (at least $1 billion spent since 2018), and they’re among the reasons Palo Alto is not yet a profitable business. However, when backing out one-time nonrecurring expenses and noncash items, the company still manages to post positive free cash flow (money left over after basic operating expenses and capital expenditures). In short, that means the company can afford its aggressive buying spree.

The free cash flow generation is important, because it gives the leader in pure-play network security the wiggle room it needs to invest heavily in cloud computing, AI, and other technology as customer needs change over time. Global cloud spending is expected to grow an average of about 16% a year through 2022, according to technology research group Gartner. Sitting at the intersection of two double-digit growth industries, that long-term trend should give Palo Alto Networks an enduring outlet to sustain double-digit sales growth and help it maintain its pole position within the world of cybersecurity.

2. Splunk: Big data and securing business operations

Splunk started out as a big data monitoring company. Its software suite allows organizations to analyze and make sense of information being generated from their digital systems, from websites to connected equipment to payment processing networks, among other things. If it’s an electronic system, it creates data; and if it creates data, Splunk can help monitor it and give customers the ability to make sense of trends and other behavior of digital systems. Incidentally, one of the primary use cases for the data parsing and analytics platform is cybersecurity.

To increase its capabilities in that department, Splunk has also embarked on an aggressive acquisition spree. As a result, the big data company is now a leader in the fast-growing security orchestration, automation, and response (SOAR) segment of the cybersecurity industry. SOAR utilizes artificial intelligence (a software system that mimics how the human brain works and learns and adapts to changing circumstances) to sift through information in real time, detect potential threats, and take action to keep things on lockdown. With data breaches a constant threat, the ability to automate aspects of the workload holds appeal for large organizations.

Despite its size, Splunk has still been growing quickly. The downside is that Splunk is spending lots of cash to foster further expansion, which keeps the company in the red. Specifically, research and development of new software capabilities and sales and marketing to acquire new customers are the biggest line items affecting the bottom line. However, much like Palo Alto Networks, Splunk is free cash flow positive; profits will be a bigger consideration later on as the company matures.

That’s because Splunk’s primary industry, big data analytics, should grow an average of 13% a year and surpass $274 billion in size by 2023 — according to researcher IDC. Along the way, Splunk will also benefit from the booming and fast-changing cybersecurity industry, making it one of the best plays on the trend. The company’s expertise in monitoring and making sense of large and complex sets of data particularly lends itself to keeping business information locked up, and its recent takeovers of smaller peers have helped bolster its position in network security. Splunk’s prospects and chances at continued industry leadership look especially good.

3. Check Point Software: Adjusting to a new technology

Check Point Software, as its name implies, offers software security along with hardware to keep business networks secure. Much like Palo Alto Networks, the company has a diversified mix of solutions covering on-premeses computer networks, cloud, and endpoint protection.

Though it’s one of the largest and oldest cybersecurity companies around (founded in 1993), Check Point has not been growing at the breakneck speed of some of its peers. Low-single-digit sales growth has been the norm for some time. The reason? New technologies like the cloud have made some of Check Point’s legacy services like hardware-based security less compelling. The company is trailing some of its competitors, so spending to update the business model for today’s security needs has been a top priority. It isn’t paying off yet, and Check Point’s sluggish pace could mean its younger peers will bypass it in the years ahead.

There is one thing that makes Check Point different from other companies on this list, though. As an older, well-established company, it does turn a profit. Thus, traditional valuation metrics (without the need to make adjustments for things like stock-based compensation, shares a company pays to employees as an extra perk) work for the stock. However, heavy spending to transform the business into a more relevant one for the times has the bottom line stuck in a rut. Until that changes, there’s little compelling reason to consider the stock.

Check Point has been working hard to update its offerings for more modern needs, but the sheer number of newer start-ups could mean this established cybersecurity business will continue to get disrupted. That’s not an enviable situation to be in, especially when the industry overall is growing by double digits.

4. CrowdStrike: The newest stock on the top-10 list

Endpoint security company CrowdStrike more than doubled in value after it had its IPO (sold shares to raise money, making it available to the general investing public for the first time) in June 2019. That easily puts the firm among the largest in the cybersecurity business by market cap.

The stock has years‘ worth of double-digit sales growth baked into it, but momentum could be on CrowdStrike’s side. Revenues more than doubled in 2018. The number of connected devices around the globe is increasing every year — by the hundreds of millions — which plays right into the hands of this security company and its endpoint-protection software suite. Since many of those devices are not tethered to an office or other physical location, CrowdStrike’s cloud computing-native system lends itself to this type of security particularly well.

Because it is cloud based, CrowdStrike also boasts the ability to make near-instant system updates when a threat is detected, and its software can learn and adapt from uploaded customer data. Paired with millions of new connections getting added to an internet-connected network every year, it adds up to lots of new customer sign-ups and expanding relationships with existing ones. Dollar-based net expansion (which measures how much money existing clients spend each year) has been over 100% for years, indicating customers spend more with CrowdStrike as time passes. It’s a powerful business model, one that CrowdStrike plans on putting to use in other security disciplines as it begins to expand beyond endpoint security. With the cloud and the number of endpoints increasing dramatically, it’s no wonder this stock is off to a hot start and looks like it has years‘ worth of growth left ahead of it.

5. Okta: Keeping data on a need-to-know basis

Another upstart security company, Okta has only been around since 2009, but the identity-protection specialist has been growing like a weed. The company ensures that employees and others with privileged access within an organization get connected to the apps and data they need — and keeps everyone else out. The number of digital systems and software being utilized by organizations continues to rise, increasing the complexity and difficulty in keeping systems secure from intruders. Thus, the need for Okta’s identity services has been booming.

In just a few years‘ time, Okta has become one of the largest cybersecurity pure plays around, with sales consistently growing north of 50% in the past. Management expects that trajectory will moderate to somewhere in the mid-30% range for the foreseeable future — still nothing to balk at. And that rate of expansion could be sustainable, too. According to the Global Market Insights cybersecurity report, identity, authentication, and access management services are expected to be an especially fast-growing subset of cybersecurity, with the potential for services to increase an average of 17% a year through 2024. At the forefront of the movement, Okta is primed to gobble up market share as identity and access management increases in importance.

Here’s the downside: Okta is not a profitable business as of this writing. The company is funneling cash into marketing and research to maximize its sales growth now. Profits will be a concern later. The good news, though, is that gross profit margin (the amount of money the company keeps after producing a service and then selling it but before paying other operating expenses) is on the rise as the company grows.

That bodes well for the future of this cybersecurity leader. Identity security/privileged data access rights is expected to be a high-growth segment of network security for the next few years, and Okta is a leader in the space.

6. Fortinet: Successfully bridging legacy security with the new

Another diversified provider of firewalls, cloud and endpoint security, and identity management, Fortinet took a hit amid worries that the trade war between the U.S. and China would dampen growth in the company’s important international markets — Asia and Europe specifically. Newer security upstarts have also disrupted some of Fortinet’s legacy offerings like hardware-based network security for on-premises protection. Economic and industry headwinds or not, though, this cybersecurity outfit is doing just fine.

Revenues and adjusted earnings were up 20% and 77%, respectively, in 2018. Fortinet has been adding dozens of new deals worth more than $1 million every quarter, winning customers over with its new and improved software suite aimed at keeping all parts of an organization safe. Although less aggressive in its acquisition strategy than Palo Alto Networks or Splunk, Fortinet continues to invest heavily in updating its offerings to keep its customers secure. The cloud has been an area of focus, as well as increasing the number of subscription-based software deals. The investments in new technology have been paying off and yielding results for shareholders, even as other legacy cybersecurity companies have been failing to make the cut.

As a result of its less aggressive nature, Fortinet also runs a profitable business where some of its competitors don’t — and the bottom line has been rising faster than sales as the company’s investments have started to yield results. Ample cash means this security business can continue to invest in its new high-octane segments like cloud, endpoint, and identity security, which bodes well for it being able to maintain its two-figure top-line growth rate for some time even as legacy lines of business fade. With a well-established presence in the industry and a successful business update strategy well underway and paying off, Fortinet is one of the best cybersecurity stocks around.

7. Symantec: The biggest cybersecurity company by revenue

Symantec is the world leader in cybersecurity services when using sales figures as the metric. With nearly $5 billion in revenue in the last year, it is nearly double the size of its younger peers like Palo Alto Networks. Yet despite Symantec’s leadership, its market cap lags. One of the oldest network security players around and owner of recognizable software names like LifeLock and Norton Antivirus, Symantec has had to deal with disruption and shifting technology that have left growth near nonexistent and profitability underwhelming.

Though Symantec has been updating its operations — it recently announced a new comprehensive cloud-based security suite covering everything from email to application login protection — results have been sluggish. Fiscal 2019 sales fell 2%. The company’s legacy operations are holding it back, and bloated operating expenses have meant paltry bottom-line earnings. Not exactly what investors should be looking for from the leader of a high-flying industry.

There could be hope of a rebound, though, as Symantec continues to work through its transition. Chipmaker Broadcom (NASDAQ:AVGO) thought there was value in Symantec and was reportedly interested in acquiring the old security company to add it to its growing software division. However, negotiations fell through, and Symantec will have to go it alone for now. Until the company can demonstrate a strategy that can gain some traction in the growing world of cybersecurity, Symantec will continue to struggle in the wake of younger and more nimble peers that started investing earlier in the shifting landscape.

8. Akamai: Guarding the security of the internet itself

The next security outfit on the list handles a different piece of the industry than any of the others covered thus far. Akamai (NASDAQ:AKAM) helps deliver and secure web content as it travels from its source to the end user, from live and streaming video to traditional web page text and pictures. The internet’s continual expansion has been a boon for Akamai, which has launched new services to cover new web applications (like video streaming) and new mobile device types to keep the internet connection to them secure.

Akamai’s traditional web business is a low- to mid-single-digit growth story, but its newer cloud security services have been growing well into the double digits. New services are still a small fraction of the whole, but they are a high-margin endeavor. Akamai’s bottom line has been getting a big double-digit boost as Akamai’s investment and spending on new web delivery applications subside and past spending starts to yield results.

Akamai has grown into one of the internet’s primary content delivery platforms, responsible for handling as much as a third of global web traffic. As such, this company will be slower moving than other security businesses, but Akamai still has growth prospects ahead of it. Internet infrastructure company Cisco expects web traffic — led by video content — to grow an average of 26% a year through 2022. That means Akamai’s newer business should continue to move the needle for some time; plus the overall operation is solidly in profitable territory. In short, the leading internet content delivery and security company should be a slow-and-steady play for the foreseeable future.

9. Zscaler: Another investment in the cloud

Back to small but up-and-coming cybersecurity. Zscaler has its sights set on securing cloud computing and thus built itself from the ground up as a cloud-only software suite. The world is going mobile, and so are business operations. With fewer centralized locations and more remotely connected devices popping up, Zscaler helps keep newer business networks safe for its customers and their employees.

With a business model similar to those of CrowdStrike and Okta, Zscaler plays in a new multibillion-dollar industry that will only continue to grow larger, and the company has been frank in saying it is all about maximizing growth right now. And no wonder, as Gartner says in its cloud research that annual spending will nearly double from 2018 to 2022 to more than $330 billion a year. Sales at Zscaler have been growing north of 60% year over year for some time, but what’s a few hundred million in annual sales when the whole market is worth hundreds of billions? The downside is that in spite of massive growth and a rosy outlook for the good times to continue, operating losses are still substantial. With Zscaler all about nurturing sales as fast as possible, the red ink is unlikely to disappear anytime soon.

Much like its start-up peers, though, Zscaler takes those losses by design as it keeps its foot on the gas. Gross profit margin was an enviable 81% at last report, one of the best in the industry. With profit potential like that in a fast-expanding cloud computing sandbox, it makes sense Zscaler is all about growth now and profit later. With the world going mobile, this security stock looks like an especially promising one in the years ahead as it takes advantage of its early cloud-based security lead.

10. F5 Networks: Lagging behind the cybersecurity growth average

F5 Networks provides hardware and software solutions that help companies keep their applications and app delivery secure. Similar to Akamai, the company’s legacy business isn’t exactly lighting the world on fire. However, newer services, particularly those aimed at cloud computing-based apps, are on a tear. To that end, F5 recently acquired app optimization and security peer NGINX for $670 million.

It’s a sizable sum but likely a prudent move for F5. The company has been reporting low-single-digit revenue growth the last few years — nearly all of which has been driven by big expansion in its software service segment. While the top line has been sluggish, the upside is that new software and security offerings are a much more profitable concern. As a result, earnings are up nearly 40% over the last trailing three-year stretch.

During its transition phase to more modern app security and delivery, F5’s stock has taken a beating. There’s worry that the transition will continue to be a bumpy one, thus making this stock among the cheapest in the cybersecurity industry. However, though the low valuation reflects the fact that F5 has fallen behind the curve in the digital age, F5 is an inexpensive play on digital security and delivery. With internet traffic and content delivery still a slow-and-steady endeavor, F5 can continue to thrive — albeit at a much slower rate than elsewhere in cybersecurity.

Bonus. Proofpoint: An up-and-coming communications security specialist

One of the smaller outfits in the security space, Proofpoint is worth a mention as a bonus number 11 on the top-10 list. The company specifically helps organizations keep their employees safe. Email attacks are a key pain point for many businesses, and securing communications in that department — as well as on social media, cloud applications, and mobile devices — is a specialty at Proofpoint.

Though a niche offering within the greater cybersecurity industry, Proofpoint is expanding fast. After the company grew 38% in 2018, management forecasted full-year 2019 revenue to be up at least another 22%. However, as with its high-powered sales-oriented peers, the company does run up big losses. As with many other cybersecurity plays we’ve been discussing, though, that’s due to Proofpoint reinvesting in itself to foster more growth.

Nevertheless, when we adjust the bottom line for one-time items and other noncash expenses, Proofpoint is free cash flow positive, a metric that has been steadily on the rise. That should help Proofpoint keep up its double-digit growth trajectory as employee access points via remote computers, smartphones, and other devices continue to boom in the States and especially overseas. It’s a much smaller business than the top 10 companies are, but this cybersecurity concern still offers a compelling growth story worth keeping an eye on as it keeps communications safe and secure.

Proofpoint will also likely see long-term benefit from the explosion in devices hooked up to a network in the years ahead. The workforce’s increasing mobility means keeping employee communications on lockdown will be an increasingly complex problem, one that this small security company can help solve.

An illustrated shield displayed on top of a wall of digital data.

Image source: Getty Images.

Choosing the right cybersecurity stock to invest in

Taking a high-level look at the biggest companies in the cybersecurity market is only the start to choosing an investment. Some of the stocks are buys, others not so much. As the industry is still in high-growth mode and adapting fast to technological developments, investors would be best off picking the companies posting the fastest revenue expansion rates and those that carry the highest gross profit margins. Click here for a discussion on the top cybersecurity stocks and an introduction on how to pick the best companies in the industry.

Before investing, though, it’s important to remember a few things. Though cybersecurity is one of the fastest-expanding industries around, with high growth expectations comes a high level of volatility. Stock prices can run higher very quickly — and reverse course just as fast. Only investors who have a long-term perspective (no less than a few years) and the ability to purchase a position over time (buying a few shares at a time on a set schedule, like monthly, quarterly, or whenever the stock dips in price by at least double digits) should consider buying.

For those with the time to wait, though, investing in cybersecurity should be a profitable endeavor. In a decade’s time, this top-10 list will no doubt look very different, but a few of these names will still be around and will likely be much larger than they are today.

 

the combination of repressive regimes with IT monopolies endows those regimes with a built-in advantage over open societies

Source: https://www.wired.com/story/mortal-danger-chinas-push-into-ai/

Governments and companies worldwide are investing heavily in artificial intelligence in hopes of new profits, smarter gadgets, and better health care. Financier and philanthropist George Soros told the World Economic Forum in Davos Thursday that the technology may also undermine free societies and create a new era of authoritarianism.

“I want to call attention to the mortal danger facing open societies from the instruments of control that machine learning and artificial intelligence can put in the hands of repressive regimes,” Soros said. He made an example of China, repeatedly calling out the country’s president, Xi Jinping.

China’s government issued a broad AI strategy in 2017, asserting that it would surpass US prowess in the technology by 2030. As in the US, much of the leading work on AI in China takes place inside a handful of large tech companies, such as search engine Baidu and retailer and payments company Alibaba.

Soros argued that AI-centric tech companies like those can become enablers of authoritarianism. He pointed to China’s developing “social credit” system, aimed at tracking citizens’ reputations by logging financial activity, online interactions, and even energy use, among other things. The system is still taking shape, but depends on data and cooperation from companies like payments firm Ant Financial, a spinout of Alibaba. “The social credit system, if it became operational, would give Xi Jinping total control over the people,” Soros said.

Soros argued that synergy like that between corporate and government AI projects creates a more potent threat than was posed by Cold War–era autocrats, many of whom spurned corporate innovation. “The combination of repressive regimes with IT monopolies endows those regimes with a built-in advantage over open societies,” Soros said. “They pose a mortal threat to open societies.”

Soros is far from the first to raise an alarm about the dangers of AI technology. It’s a favorite topic of Elon Musk, and last year Henry Kissinger called for a US government commission to examine the technology’s risks. Google cofounder Sergey Brin warned in Alphabet’s most recent annual shareholder letter that AI technology had downsides, including the potential to manipulate people. Canada and France plan to establish an intergovernmental group to study how AI changes societies.

The financier attempted to draft Donald Trump into his AI vigilance campaign. He advised the president to be tougher on Chinese telecoms manufacturers ZTE and Huawei, to prevent them from dominating the high-bandwidth 5G mobile networks being built around the world. Both companies are already reeling from sanctions by the US and other governments.

Soros also urged the well-heeled attendees of Davos to help forge international mechanisms to prevent AI-enhanced authoritarianism—and that could both include and contain China. He asked them to imagine a technologically oriented version of the treaty signed after World War II that underpins the United Nations, binding countries into common standards for human rights and freedoms.

Here is the text of Soros’s speech:

I want to use my time tonight to warn the world about an unprecedented danger that’s threatening the very survival of open societies.

Last year when I stood before you I spent most of my time analyzing the nefarious role of the IT monopolies. This is what I said: “An alliance is emerging between authoritarian states and the large data rich IT monopolies that bring together nascent systems of corporate surveillance with an already developing system of state sponsored surveillance. This may well result in a web of totalitarian control the likes of which not even George Orwell could have imagined.”

Tonight I want to call attention to the mortal danger facing open societies from the instruments of control that machine learning and artificial intelligence can put in the hands of repressive regimes. I’ll focus on China, where Xi Jinping wants a one-party state to reign supreme.

A lot of things have happened since last year and I’ve learned a lot about the shape that totalitarian control is going to take in China.

All the rapidly expanding information available about a person is going to be consolidated in a centralized database to create a “social credit system.” Based on that data, people will be evaluated by algorithms that will determine whether they pose a threat to the one-party state. People will then be treated accordingly.

The social credit system is not yet fully operational, but it’s clear where it’s heading. It will subordinate the fate of the individual to the interests of the one-party state in ways unprecedented in history.

I find the social credit system frightening and abhorrent. Unfortunately, some Chinese find it rather attractive because it provides information and services that aren’t currently available and can also protect law-abiding citizens against enemies of the state.

China isn’t the only authoritarian regime in the world, but it’s undoubtedly the wealthiest, strongest and most developed in machine learning and artificial intelligence. This makes Xi Jinping the most dangerous opponent of those who believe in the concept of open society. But Xi isn’t alone. Authoritarian regimes are proliferating all over the world and if they succeed, they will become totalitarian.

As the founder of the Open Society Foundations, I’ve devoted my life to fighting totalizing, extremist ideologies, which falsely claim that the ends justify the means. I believe that the desire of people for freedom can’t be repressed forever. But I also recognize that open societies are profoundly endangered at present.

What I find particularly disturbing is that the instruments of control developed by artificial intelligence give an inherent advantage to authoritarian regimes over open societies. For them, instruments of control provide a useful tool; for open societies, they pose a mortal threat.

I use “open society” as shorthand for a society in which the rule of law prevails as opposed to rule by a single individual and where the role of the state is to protect human rights and individual freedom. In my personal view, an open society should pay special attention to those who suffer from discrimination or social exclusion and those who can’t defend themselves.

By contrast, authoritarian regimes use whatever instruments of control they possess to maintain themselves in power at the expense of those whom they exploit and suppress.

How can open societies be protected if these new technologies give authoritarian regimes a built-in advantage? That’s the question that preoccupies me. And it should also preoccupy all those who prefer to live in an open society.

Open societies need to regulate companies that produce instruments of control, while authoritarian regimes can declare them “national champions.” That’s what has enabled some Chinese state-owned companies to catch up with and even surpass the multinational giants.

This, of course, isn’t the only problem that should concern us today. For instance, man-made climate change threatens the very survival of our civilization. But the structural disadvantage that confronts open societies is a problem which has preoccupied me and I’d like to share with you my ideas on how to deal with it.

My deep concern for this issue arises out of my personal history. I was born in Hungary in 1930 and I’m Jewish. I was 13 years old when the Nazis occupied Hungary and started deporting Jews to extermination camps.

I was very fortunate because my father understood the nature of the Nazi regime and arranged false identity papers and hiding places for all members of his family, and for a number of other Jews as well. Most of us survived.

The year 1944 was the formative experience of my life. I learned at an early age how important it is what kind of political regime prevails. When the Nazi regime was replaced by Soviet occupation I left Hungary as soon as I could and found refuge in England.

At the London School of Economics I developed my conceptual framework under the influence of my mentor, Karl Popper. That framework proved to be unexpectedly useful when I found myself a job in the financial markets. The framework had nothing to do with finance, but it is based on critical thinking. This allowed me to analyze the deficiencies of the prevailing theories guiding institutional investors. I became a successful hedge fund manager and I prided myself on being the best paid critic in the world.

Running a hedge fund was very stressful. When I had made more money than I needed for myself or my family, I underwent a kind of midlife crisis. Why should I kill myself to make more money? I reflected long and hard on what I really cared about and in 1979 I set up the Open Society Fund. I defined its objectives as helping to open up closed societies, reducing the deficiencies of open societies and promoting critical thinking.

My first efforts were directed at undermining the apartheid system in South Africa. Then I turned my attention to opening up the Soviet system. I set up a joint venture with the Hungarian Academy of Science, which was under Communist control, but its representatives secretly sympathized with my efforts. This arrangement succeeded beyond my wildest dreams. I got hooked on what I like to call “political philanthropy.” That was in 1984.

In the years that followed, I tried to replicate my success in Hungary and in other Communist countries. I did rather well in the Soviet empire, including the Soviet Union itself, but in China it was a different story.

My first effort in China looked rather promising. It involved an exchange of visits between Hungarian economists who were greatly admired in the Communist world, and a team from a newly established Chinese think tank which was eager to learn from the Hungarians.

Based on that initial success, I proposed to Chen Yizi, the leader of the think tank, to replicate the Hungarian model in China. Chen obtained the support of Premier Zhao Ziyang and his reform-minded policy secretary Bao Tong.

A joint venture called the China Fund was inaugurated in October 1986. It was an institution unlike any other in China. On paper, it had complete autonomy.

Bao Tong was its champion. But the opponents of radical reforms, who were numerous, banded together to attack him. They claimed that I was a CIA agent and asked the internal security agency to investigate. To protect himself, Zhao Ziyang replaced Chen Yizi with a high-ranking official in the external security police. The two organizations were co-equal and they couldn’t interfere in each other’s affairs.

I approved this change because I was annoyed with Chen Yizi for awarding too many grants to members of his own institute and I was unaware of the political infighting behind the scenes. But applicants to the China Fund soon noticed that the organization had come under the control of the political police and started to stay away. Nobody had the courage to explain to me the reason for it.

Eventually, a Chinese grantee visited me in New York and told me, at considerable risk to himself. Soon thereafter, Zhao Ziyang was removed from power and I used that excuse to close the foundation. This happened just before the Tiananmen Square massacre in 1989 and it left a “black spot” on the record of the people associated with the foundation. They went to great length to clear their names and eventually they succeeded.

In retrospect, it’s clear that I made a mistake in trying to establish a foundation which operated in ways that were alien to people in China. At that time, giving a grant created a sense of mutual obligation between the donor and recipient and obliged both of them to remain loyal to each other forever.

So much for history. Let me now turn to the events that occurred in the last year, some of which surprised me.

When I first started going to China, I met many people in positions of power who were fervent believers in the principles of open society. In their youth they had been deported to the countryside to be re-educated, often suffering hardships far greater than mine in Hungary. But they survived and we had much in common. We had all been on the receiving end of a dictatorship.

They were eager to learn about Karl Popper’s thoughts on the open society. While they found the concept very appealing, their interpretation remained somewhat different from mine. They were familiar with Confucian tradition, but there was no tradition of voting in China. Their thinking remained hierarchical and carried a built-in respect for high office. I, on the other hand I was more egalitarian and wanted everyone to have a vote.

So, I wasn’t surprised when Xi Jinping ran into serious opposition at home; but I was surprised by the form it took. At last summer’s leadership convocation at the seaside resort of Beidaihe, Xi Jinping was apparently taken down a peg or two. Although there was no official communique, rumor had it that the convocation disapproved of the abolition of term limits and the cult of personality that Xi had built around himself.

It’s important to realize that such criticisms were only a warning to Xi about his excesses, but did not reverse the lifting of the two-term limit. Moreover, “The Thought of Xi Jinping,” which he promoted as his distillation of Communist theory was elevated to the same level as the “Thought of Chairman Mao.” So Xi remains the supreme leader, possibly for lifetime. The ultimate outcome of the current political infighting remains unresolved.

I’ve been concentrating on China, but open societies have many more enemies, Putin’s Russia foremost among them. And the most dangerous scenario is when these enemies conspire with, and learn from, each other on how to better oppress their people.

The question poses itself, what can we do to stop them?

The first step is to recognize the danger. That’s why I’m speaking out tonight. But now comes the difficult part. Those of us who want to preserve the open society must work together and form an effective alliance. We have a task that can’t be left to governments.

History has shown that even governments that want to protect individual freedom have many other interests and they also give precedence to the freedom of their own citizens over the freedom of the individual as a general principle.

My Open Society Foundations are dedicated to protecting human rights, especially for those who don’t have a government defending them. When we started four decades ago there were many governments which supported our efforts but their ranks have thinned out. The US and Europe were our strongest allies, but now they’re preoccupied with their own problems.

Therefore, I want to focus on what I consider the most important question for open societies: what will happen in China?

The question can be answered only by the Chinese people. All we can do is to draw a sharp distinction between them and Xi Jinping. Since Xi has declared his hostility to open society, the Chinese people remain our main source of hope.

And there are, in fact, grounds for hope. As some China experts have explained to me, there is a Confucian tradition, according to which advisors of the emperor are expected to speak out when they strongly disagree with one of his actions or decrees, even that may result in exile or execution.

This came as a great relief to me when I had been on the verge of despair. The committed defenders of open society in China, who are around my age, have mostly retired and their places have been taken by younger people who are dependent on Xi Jinping for promotion. But a new political elite has emerged that is willing to uphold the Confucian tradition. This means that Xi will continue to have a political opposition at home.

Xi presents China as a role model for other countries to emulate, but he’s facing criticism not only at home but also abroad. His Belt and Road Initiative has been in operation long enough to reveal its deficiencies.

It was designed to promote the interests of China, not the interests of the recipient countries; its ambitious infrastructure projects were mainly financed by loans, not by grants, and foreign officials were often bribed to accept them. Many of these projects proved to be uneconomic.

The iconic case is in Sri Lanka. China built a port that serves its strategic interests. It failed to attract sufficient commercial traffic to service the debt and enabled China to take possession of the port. There are several similar cases elsewhere and they’re causing widespread resentment.

Malaysia is leading the pushback. The previous government headed by Najib Razak sold out to China but in May 2018 Razak was voted out of office by a coalition led by Mahathir Mohamed. Mahathir immediately stopped several big infrastructure projects and is currently negotiating with China how much compensation Malaysia will still have to pay.

The situation is not as clear-cut in Pakistan, which has been the largest recipient of Chinese investments. The Pakistani army is fully beholden to China but the position of Imran Khan who became prime minister last August is more ambivalent. At the beginning of 2018, China and Pakistan announced grandiose plans in military cooperation. By the end of the year, Pakistan was in a deep financial crisis. But one thing became evident: China intends to use the Belt and Road Initiative for military purposes as well.

All these setbacks have forced Xi Jinping to modify his attitude toward the Belt and Road Initiative. In September, he announced that “vanity projects” will be shunned in favor of more carefully conceived initiatives and in October, the People’s Daily warned that projects should serve the interests of the recipient countries.

Customers are now forewarned and several of them, ranging from Sierra Leone to Ecuador, are questioning or renegotiating projects.

Most importantly, the US government has now identified China as a “strategic rival.” President Trump is notoriously unpredictable, but this decision was the result of a carefully prepared plan. Since then, the idiosyncratic behavior of Trump has been largely superseded by a China policy adopted by the agencies of the administration and overseen by Asian affairs advisor of the National Security Council Matt Pottinger and others. The policy was outlined in a seminal speech by Vice President Mike Pence on October 4th.

Even so, declaring China a strategic rival is too simplistic. China is an important global actor. An effective policy towards China can’t be reduced to a slogan.

It needs to be far more sophisticated, detailed and practical; and it must include an American economic response to the Belt and Road Initiative. The Pottinger plan doesn’t answer the question whether its ultimate goal is to level the playing field or to disengage from China altogether.

Xi Jinping fully understood the threat that the new US policy posed for his leadership. He gambled on a personal meeting with President Trump at the G20 meeting in Buenos Aires. In the meantime, the danger of global trade war escalated and the stock market embarked on a serious sell-off in December. This created problems for Trump who had concentrated all his efforts on the 2018 midterm elections. When Trump and Xi met, both sides were eager for a deal. No wonder that they reached one, but it’s very inconclusive: a ninety-day truce.

In the meantime, there are clear indications that a broad based economic decline is in the making in China, which is affecting the rest of the world. A global slowdown is the last thing the market wants to see.

The unspoken social contract in China is built on steadily rising living standards. If the decline in the Chinese economy and stock market is severe enough, this social contract may be undermined and even the business community may turn against Xi Jinping. Such a downturn could also sound the death knell of the Belt and Road Initiative, because Xi may run out of resources to continue financing so many lossmaking investments.

On the question of global internet governance, there’s an undeclared struggle between the West and China. China wants to dictate rules and procedures that govern the digital economy by dominating the developing world with its new platforms and technologies. This is a threat to the freedom of the Internet and indirectly open society itself.

Last year I still believed that China ought to be more deeply embedded in the institutions of global governance, but since then Xi Jinping’s behavior has changed my opinion. My present view is that instead of waging a trade war with practically the whole world, the US should focus on China. Instead of letting ZTE and Huawei off lightly, it needs to crack down on them. If these companies came to dominate the 5G market, they would present an unacceptable security risk for the rest of the world.

Regrettably, President Trump seems to be following a different course: make concessions to China and declare victory while renewing his attacks on US allies. This is liable to undermine the US policy objective of curbing China’s abuses and excesses.

To conclude, let me summarize the message I’m delivering tonight. My key point is that the combination of repressive regimes with IT monopolies endows those regimes with a built-in advantage over open societies. The instruments of control are useful tools in the hands of authoritarian regimes, but they pose a mortal threat to open societies.

China is not the only authoritarian regime in the world but it is the wealthiest, strongest and technologically most advanced. This makes Xi Jinping the most dangerous opponent of open societies. That’s why it’s so important to distinguish Xi Jinping’s policies from the aspirations of the Chinese people. The social credit system, if it became operational, would give Xi total control over the people. Since Xi is the most dangerous enemy of the open society, we must pin our hopes on the Chinese people, and especially on the business community and a political elite willing to uphold the Confucian tradition.

This doesn’t mean that those of us who believe in the open society should remain passive. The reality is that we are in a Cold War that threatens to turn into a hot one. On the other hand, if Xi and Trump were no longer in power, an opportunity would present itself to develop greater cooperation between the two cyber-superpowers.

It is possible to dream of something similar to the United Nations Treaty that arose out of the Second World War. This would be the appropriate ending to the current cycle of conflict between the US and China. It would reestablish international cooperation and allow open societies to flourish. That sums up my message.

Steve Rymell Head of Technology, Airbus CyberSecurity answers What Should Frighten us about AI-Based Malware?

Of all the cybersecurity industry’s problems, one of the most striking is the way attackers are often able to stay one step ahead of defenders without working terribly hard. It’s an issue whose root causes are mostly technical: the prime example are software vulnerabilities which cyber-criminals have a habit of finding out about before vendors and their customers, leading to the almost undefendable zero-day phenomenon which has propelled many famous cyber-attacks.

A second is that organizations struggling with the complexity of unfamiliar and new technologies make mistakes, inadvertently leaving vulnerable ports and services exposed. Starkest of all, perhaps, is the way techniques, tools, and infrastructure set up to help organizations defend themselves (Shodan, for example but also numerous pen-test tools) are now just as likely to be turned against businesses by attackers who tear into networks with the aggression of red teams gone rogue.

Add to this the polymorphic nature of modern malware, and attackers can appear so conceptually unstoppable that it’s no wonder security vendors increasingly emphasize the need not to block attacks but instead respond to them as quickly as possible.

The AI fightback
Some years back, a list of mostly US-based start-ups started a bit of a counter-attack against the doom and gloom with a brave new idea – AI machine learning (ML) security powered by algorithms. In an age of big data, this makes complete sense and the idea has since been taken up by all manner of systems used to for anti-spam, malware detection, threat analysis and intelligence, and Security Operations Centre (SoC) automation where it has been proposed to help patch skills shortages.

I’d rate these as useful advances, but there’s no getting away from the controversial nature of the theory, which has been branded by some as the ultimate example of technology as a ‘black box’ nobody really understands. How do we know that machine learning is able to detect new and unknown types of attack that conventional systems fail to spot? In some cases, it could be because the product brochure says so.

Then the even bigger gotcha hits you – what’s stopping attackers from outfoxing defensive ML with even better ML of their own? If this were possible, even some of the time, the industry would find itself back at square one.

This is pure speculation, of course, because to date nobody has detected AI being used in a cyber-attack, which is why our understanding of how it might work remains largely based around academic research such as IBM’s proof-of-concept DeepLocker malware project.

What might malicious ML look like?
It would be unwise to ignore the potential for trouble. One of the biggest hurdles faced by attackers is quickly understanding what works, for example when sending spam, phishing and, increasingly, political disinformation.

It’s not hard to imagine that big data techniques allied to ML could hugely improve the efficiency of these threats by analyzing how targets react to and share them in real time. This implies the possibility that such campaigns might one day evolve in a matter of hours or minutes; a timescale defender would struggle to counter using today’s technologies.

A second scenario is one that defenders would even see: that cyber-criminals might simulate the defenses of a target using their own ML to gauge the success of different attacks (a technique already routinely used to evade anti-virus). Once again, this exploits the advantage that attackers always have sight of the target, while defenders must rely on good guesses.

Or perhaps ML could simply be used to crank out vast quantities of new and unique malware than is possible today. Whichever of these approaches is taken – and this is only a sample of the possibilities – it jumps out at you how awkward it would be to defend against even relatively simple ML-based attacks. About the only consolation is that if ML-based AI really is a black box that nobody understands then, logically, the attackers won’t understand it either and will waste time experimenting.

Unintended consequences
If we should fear anything it’s precisely this black box effect. There are two parts to this, the biggest of which is the potential for ML-based malware to cause something unintended to happen, especially when targeting critical infrastructure.

This phenomenon has already come to pass with non-AI malware – Stuxnet in 2010 and NotPetya in 2017 are the obvious examples – both of which infected thousands of organizations not on their original target list after unexpectedly ‘escaping’ into the wild.

When it comes to powerful malware exploiting multiple zero days there’s no such thing as a reliably contained attack. Once released, this kind of malware remains pathogenically dangerous until every system it can infect is patched or taken offline, which might be years or decades down the line.

Another anxiety is that because the expertise to understand ML is still thin on the ground, there’s a danger that engineers could come to rely on it without fully understanding its limitations, both for defense and by over-estimating its usefulness in attack. The mistake, then, might be that too many over-invest in it based on marketing promises that end up consuming resources better deployed elsewhere.  Once a more realistic assessment takes hold, ML could end up as just another tool that is good at solving certain very specific problems.

Conclusion
My contradictory-sounding conclusion is that perhaps ML and AI makes no fundamental difference at all. It’s just another stop on a journey computer security has been making since the beginning of digital time. The problem is overcoming our preconceptions about what it is and what it means. Chiefly, we must overcome the tendency to think of ML and AI as mysteriously ‘other’ because we don’t understand it and therefore find it difficult to process the concept of machines making complex decisions.

It’s not as if attackers aren’t breaching networks already with today’s pre-ML technology or that well-prepared defenders aren’t regularly stopping them using the same technology. What AI reminds us is that the real difference is how organizations are defended, not whether they or their attackers use ML and AI or not. That has always been what separates survivors from victims. Cybersecurity remains a working demonstration of how the devil takes the hindmost.

Source: https://www.infosecurity-magazine.com/opinions/frighten-ai-malware-1/

45 Techniques Used by Data Scientists

These techniques cover most of what data scientists and related practitioners are using in their daily activities, whether they use solutions offered by a vendor, or whether they design proprietary tools. When you click on any of the 45 links below, you will find a selection of articles related to the entry in question. Most of these articles are hard to find with a Google search, so in some ways this gives you access to the hidden literature on data science, machine learning, and statistical science. Many of these articles are fundamental to understanding the technique in question, and come with further references and source code.

Starred techniques (marked with a *) belong to what I call deep data science, a branch of data science that has little if any overlap with closely related fields such as machine learning, computer science, operations research, mathematics, or statistics. Even classical machine learning and statistical techniques such as clustering, density estimation,  or tests of hypotheses, have model-free, data-driven, robust versions designed for automated processing (as in machine-to-machine communications), and thus also belong to deep data science. However, these techniques are not starred here, as the standard versions of these techniques are more well known (and unfortunately more used) than the deep data science equivalent.

To learn more about deep data science,  click here. Note that unlike deep learning, deep data science is not the intersection of data science and artificial intelligence; however, the analogy between deep data science and deep learning is not completely meaningless, in the sense that both deal with automation.

Also, to discover in which contexts and applications the 40 techniques below are used, I invite you to read the following articles:

Finally, when using a technique, you need to test its performance. Read this article about 11 Important Model Evaluation Techniques Everyone Should Know.

The 40 data science techniques

  1. Linear Regression
  2. Logistic Regression
  3. Jackknife Regression *
  4. Density Estimation
  5. Confidence Interval
  6. Test of Hypotheses
  7. Pattern Recognition
  8. Clustering – (aka Unsupervised Learning)
  9. Supervised Learning
  10. Time Series
  11. Decision Trees
  12. Random Numbers
  13. Monte-Carlo Simulation
  14. Bayesian Statistics
  15. Naive Bayes
  16. Principal Component Analysis – (PCA)
  17. Ensembles
  18. Neural Networks
  19. Support Vector Machine – (SVM)
  20. Nearest Neighbors – (k-NN)
  21. Feature Selection – (aka Variable Reduction)
  22. Indexation / Cataloguing *
  23. (Geo-) Spatial Modeling
  24. Recommendation Engine *
  25. Search Engine *
  26. Attribution Modeling *
  27. Collaborative Filtering *
  28. Rule System
  29. Linkage Analysis
  30. Association Rules
  31. Scoring Engine
  32. Segmentation
  33. Predictive Modeling
  34. Graphs
  35. Deep Learning
  36. Game Theory
  37. Imputation
  38. Survival Analysis
  39. Arbitrage
  40. Lift Modeling
  41. Yield Optimization
  42. Cross-Validation
  43. Model Fitting
  44. Relevancy Algorithm *
  45. Experimental Design

Source: https://www.datasciencecentral.com/profiles/blogs/40-techniques-used-by-data-scientists

Beapy uses NSA’s DoublePulsar EternalBlue & Mimikatz to collect and use passwords to mine for cryptocurrency following Coinhive

Two years after highly classified exploits built by the National Security Agency were stolen and published, hackers are still using the tools for nefarious reasons.

Security researchers at Symantec say they’ve seen a recent spike in a new malware, dubbed Beapy, which uses the leaked hacking tools to spread like wildfire across corporate networks to enslave computers into running mining code to generate cryptocurrency.

Beapy was first spotted in January but rocketed to more than 12,000 unique infections across 732 organizations since March, said Alan Neville, Symantec’s lead researcher on Beapy, in an email to TechCrunch. The malware almost exclusively targets enterprises, host to large numbers of computers, which when infected with cryptocurrency mining malware can generate sizable sums of money.

The malware relies on someone in the company opening a malicious email. Once opened, the malware drops the NSA-developed DoublePulsar malware to create a persistent backdoor on the infected computer, and uses the NSA’s EternalBlue exploit to spread laterally throughout the network. These are the same exploits that helped spread the WannaCry ransomware in 2017. Once the computers on the network are backdoored, the Beapy malware is pulled from the hacker’s command and control server to infect each computer with the mining software.

Not only does Beapy use the NSA’s exploits to spread, it also uses Mimikatz, an open-source credential stealer, to collect and use passwords from infected computers to navigate its way across the network.

According to the researchers, more than 80 percent of Beapy’s infections are in China.

Hijacking computers to mine for cryptocurrency — known as cryptojacking — has been on the decline in recent months, partially following the shutdown of Coinhive, a popular mining tool. Hackers are finding the rewards fluctuate greatly depending on the value of the cryptocurrency. But cryptojacking remains a more stable source of revenue than the hit-and-miss results of ransomware.

In September, some 919,000 computers were vulnerable to EternalBlue attacks — many of which were exploited for mining cryptocurrency. Today, that figure has risen to more than a million.

Typically cryptojackers exploit vulnerabilities in websites, which, when opened on a user’s browser, uses the computer’s processing power to generate cryptocurrency. But file-based cryptojacking is far more efficient and faster, allowing the hackers to make more money.

In a single month, file-based mining can generate up to $750,000, Symantec researchers estimate, compared to just $30,000 from a browser-based mining operation.

Cryptojacking might seem like a victimless crime — no data is stolen and files aren’t encrypted, but Symantec says the mining campaigns can slow down computers and cause device degradation.

A new cryptocurrency mining malware uses leaked NSA exploits to spread across enterprise networks

Tim Cook: The Genius Who Took Apple to the Next Level

 

 

Excerpted from Tim Cook: The Genius Who Took Apple to the Next Level

 

They knew that they had to respond immediately. The writ would dominate the next day’s news, and Apple had to have a response. “Tim knew that this was a massive decision on his part,” Sewell said. It was a big moment, “a bet-the-company kind of decision.” Cook and the team stayed up all night—a straight 16 hours—working on their response. Cook already knew his position—Apple would refuse—but he wanted to know all the angles: What was Apple’s legal position? What was its legal obligation? Was this the right response? How should it sound? How should it read? What was the right tone?

iOS 8 added much stronger encryption than had been seen before in smartphones. It encrypted all the user’s data—phone call records, messages, photos, contacts, and so on—with the user’s passcode. The encryption was so strong, not even Apple could break it. Security on earlier devices was much weaker, and there were various ways to break into them, but Apple could no longer access locked devices running iOS 8, even if law enforcement had a valid warrant. “Unlike our competitors, Apple cannot bypass your passcode and therefore cannot access this data,” the company wrote on its website. “So it’s not technically feasible for us to respond to government warrants for the extraction of this data from devices in their possession running iOS 8.”

The War Room

For the next two months, the executive floor at One Infinite Loop turned into a 24/7 situation room, with staffers sending out messages and responding to journalists’ queries. One PR rep said that they were sometimes sending out multiple updates a day with up to 700 journalists cc’d on the emails. This is in stark contrast to Apple’s usual PR strategy, which consists of occasional press releases and routinely ignoring reporters’ calls and emails.

Cook also felt he had to rally the troops, to keep morale high at a time when the company was under attack. In an email to Apple employees, titled “Thank you for your support,” he wrote, “This case is about much more than a single phone or a single investigation.” He continued, “At stake is the data security of hundreds of millions of law-abiding people and setting a dangerous precedent that threatens everyone’s civil liberties.” It worked. Apple employees trusted their leader to make the decision that was right not only for them but also for the general public.

Cook was very concerned about how Apple would be perceived throughout this media firestorm. He wanted very much to use it as an opportunity to educate the public about personal security, privacy, and encryption. “I think a lot of reporters saw a new version, a new face of Apple,” said the PR person, who asked to remain anonymous. “And it was Tim’s decision to act in this fashion. Very different from what we have done in the past. We were sometimes sending out emails to reporters three times a day on keeping them updated.”

Outside Apple’s walls, Cook went on a charm offensive. Eight days after publishing his privacy letter, he sat down for a prime-time interview with ABC News. Sitting in his office at One Infinite Loop, he sincerely explained Apple’s position. It was the “most important [interview] he’s given as Apple’s CEO,” said the Washington Post. “Cook responded to questions with a raw conviction that was even more emphatic than usual,” wrote the paper. “He used sharp and soaring language, calling the request the ‘software equivalent of cancer’ and talking about ‘fundamental’ civil liberties.

https://www.wired.com/story/the-time-tim-cook-stood-his-ground-against-fbi/

Alexa do you work for the NSA ;-)

Tens of millions of people use smart speakers and their voice software to play games, find music or trawl for trivia. Millions more are reluctant to invite the devices and their powerful microphones into their homes out of concern that someone might be listening.

Sometimes, someone is.

Amazon.com Inc. employs thousands of people around the world to help improve the Alexa digital assistant powering its line of Echo speakers. The team listens to voice recordings captured in Echo owners’ homes and offices. The recordings are transcribed, annotated and then fed back into the software as part of an effort to eliminate gaps in Alexa’s understanding of human speech and help it better respond to commands.

The Alexa voice review process, described by seven people who have worked on the program, highlights the often-overlooked human role in training software algorithms. In marketing materials Amazon says Alexa “lives in the cloud and is always getting smarter.” But like many software tools built to learn from experience, humans are doing some of the teaching.

The team comprises a mix of contractors and full-time Amazon employees who work in outposts from Boston to Costa Rica, India and Romania, according to the people, who signed nondisclosure agreements barring them from speaking publicly about the program. They work nine hours a day, with each reviewer parsing as many as 1,000 audio clips per shift, according to two workers based at Amazon’s Bucharest office, which takes up the top three floors of the Globalworth building in the Romanian capital’s up-and-coming Pipera district. The modern facility stands out amid the crumbling infrastructure and bears no exterior sign advertising Amazon’s presence.

The work is mostly mundane. One worker in Boston said he mined accumulated voice data for specific utterances such as “Taylor Swift” and annotated them to indicate the searcher meant the musical artist. Occasionally the listeners pick up things Echo owners likely would rather stay private: a woman singing badly off key in the shower, say, or a child screaming for help. The teams use internal chat rooms to share files when they need help parsing a muddled word—or come across an amusing recording.

 Amazon in Bucharest
Amazon has offices in this Bucharest building.
Photographer: Irina Vilcu/Bloomberg

Sometimes they hear recordings they find upsetting, or possibly criminal. Two of the workers said they picked up what they believe was a sexual assault. When something like that happens, they may share the experience in the internal chat room as a way of relieving stress. Amazon says it has procedures in place for workers to follow when they hear something distressing, but two Romania-based employees said that, after requesting guidance for such cases, they were told it wasn’t Amazon’s job to interfere.

“We take the security and privacy of our customers’ personal information seriously,” an Amazon spokesman said in an emailed statement. “We only annotate an extremely small sample of Alexa voice recordings in order [to] improve the customer experience. For example, this information helps us train our speech recognition and natural language understanding systems, so Alexa can better understand your requests, and ensure the service works well for everyone.

“We have strict technical and operational safeguards, and have a zero tolerance policy for the abuse of our system. Employees do not have direct access to information that can identify the person or account as part of this workflow. All information is treated with high confidentiality and we use multi-factor authentication to restrict access, service encryption and audits of our control environment to protect it.”

Amazon, in its marketing and privacy policy materials, doesn’t explicitly say humans are listening to recordings of some conversations picked up by Alexa. “We use your requests to Alexa to train our speech recognition and natural language understanding systems,” the company says in a list of frequently asked questions.

In Alexa’s privacy settings, Amazon gives users the option of disabling the use of their voice recordings for the development of new features. The company says people who opt out of that program might still have their recordings analyzed by hand over the regular course of the review process. A screenshot reviewed by Bloomberg shows that the recordings sent to the Alexa reviewers don’t provide a user’s full name and address but are associated with an account number, as well as the user’s first name and the device’s serial number.

The Intercept reported earlier this year that employees of Amazon-owned Ring manually identify vehicles and people in videos captured by the company’s doorbell cameras, an effort to better train the software to do that work itself.

“You don’t necessarily think of another human listening to what you’re telling your smart speaker in the intimacy of your home,” said Florian Schaub, a professor at the University of Michigan who has researched privacy issues related to smart speakers. “I think we’ve been conditioned to the [assumption] that these machines are just doing magic machine learning. But the fact is there is still manual processing involved.”

“Whether that’s a privacy concern or not depends on how cautious Amazon and other companies are in what type of information they have manually annotated, and how they present that information to someone,” he added.

When the Echo debuted in 2014, Amazon’s cylindrical smart speaker quickly popularized the use of voice software in the home. Before long, Alphabet Inc. launched its own version, called Google Home, followed by Apple Inc.’s HomePod. Various companies also sell their own devices in China. Globally, consumers bought 78 million smart speakers last year, according to researcher Canalys. Millions more use voice software to interact with digital assistants on their smartphones.

Alexa software is designed to continuously record snatches of audio, listening for a wake word. That’s “Alexa” by default, but people can change it to “Echo” or “computer.” When the wake word is detected, the light ring at the top of the Echo turns blue, indicating the device is recording and beaming a command to Amazon servers.

 Inside An Amazon 4-Star Store
An Echo smart speaker inside an Amazon 4-star store in Berkeley, California.
Photographer: Cayce Clifford/Bloomberg

Most modern speech-recognition systems rely on neural networks patterned on the human brain. The software learns as it goes, by spotting patterns amid vast amounts of data. The algorithms powering the Echo and other smart speakers use models of probability to make educated guesses. If someone asks Alexa if there’s a Greek place nearby, the algorithms know the user is probably looking for a restaurant, not a church or community center.

But sometimes Alexa gets it wrong—especially when grappling with new slang, regional colloquialisms or languages other than English. In French, avec sa, “with his” or “with her,” can confuse the software into thinking someone is using the Alexa wake word. Hecho, Spanish for a fact or deed, is sometimes misinterpreted as Echo. And so on. That’s why Amazon recruited human helpers to fill in the gaps missed by the algorithms.

Apple’s Siri also has human helpers, who work to gauge whether the digital assistant’s interpretation of requests lines up with what the person said. The recordings they review lack personally identifiable information and are stored for six months tied to a random identifier, according to an Apple security white paper. After that, the data is stripped of its random identification information but may be stored for longer periods to improve Siri’s voice recognition.

At Google, some reviewers can access some audio snippets from its Assistant to help train and improve the product, but it’s not associated with any personally identifiable information and the audio is distorted, the company says.

A recent Amazon job posting, seeking a quality assurance manager for Alexa Data Services in Bucharest, describes the role humans play: “Every day she [Alexa] listens to thousands of people talking to her about different topics and different languages, and she needs our help to make sense of it all.” The want ad continues: “This is big data handling like you’ve never seen it. We’re creating, labeling, curating and analyzing vast quantities of speech on a daily basis.”

Amazon’s review process for speech data begins when Alexa pulls a random, small sampling of customer voice recordings and sends the audio files to the far-flung employees and contractors, according to a person familiar with the program’s design.

 Amazon.com Inc. Holds Product Reveal Launch
The Echo Spot
Photographer: Daniel Berman/Bloomberg

Some Alexa reviewers are tasked with transcribing users’ commands, comparing the recordings to Alexa’s automated transcript, say, or annotating the interaction between user and machine. What did the person ask? Did Alexa provide an effective response?

Others note everything the speaker picks up, including background conversations—even when children are speaking. Sometimes listeners hear users discussing private details such as names or bank details; in such cases, they’re supposed to tick a dialog box denoting “critical data.” They then move on to the next audio file.

According to Amazon’s website, no audio is stored unless Echo detects the wake word or is activated by pressing a button. But sometimes Alexa appears to begin recording without any prompt at all, and the audio files start with a blaring television or unintelligible noise. Whether or not the activation is mistaken, the reviewers are required to transcribe it. One of the people said the auditors each transcribe as many as 100 recordings a day when Alexa receives no wake command or is triggered by accident.

In homes around the world, Echo owners frequently speculate about who might be listening, according to two of the reviewers. “Do you work for the NSA?” they ask. “Alexa, is someone else listening to us?”

— With assistance by Gerrit De Vynck, Mark Gurman, and Irina Vilcu

Source: https://www.bloomberg.com/news/articles/2019-04-10/is-anyone-listening-to-you-on-alexa-a-global-team-reviews-audio

Smart firewall iPhone app promises to put your privacy before profits

you-asked-am-i-addicted-to-my-phone

For weeks, a small team of security researchers and developers have been putting the finishing touches on a new privacy app, which its founder says can nix some of the hidden threats that mobile users face — often without realizing.

Phones track your location, apps siphon off our data, and aggressive ads try to grab your attention. Your phone has long been a beacon of data, broadcasting to ad networks and data trackers, trying to build up profiles on you wherever you go to sell you things you’ll never want.

Will Strafach  knows that all too well. A security researcher and former iPhone jailbreaker, Strafach has shifted his time digging into apps for insecure, suspicious and unethical behavior. Last year, he found AccuWeather was secretly sending precise location data without a user’s permission. And just a few months ago, he revealed a list of dozens of apps that were sneakily siphoning off their users’ tracking data to data monetization firms without their users’ explicit consent.

Now his team — including co-founder Joshua Hill and chief operating officer Chirayu Patel — will soon bake those findings into its new “smart firewall” app, which he says will filter and block traffic that invades a user’s privacy.

“We’re in a ‘wild west’ of data collection,” he said, “where data is flying out from your phone under the radar — not because people don’t care but there’s no real visibility and people don’t know it’s happening,” he told me in a call last week.

At its heart, the Guardian Mobile Firewall — currently in a closed beta — funnels all of an iPhone or iPad’s internet traffic through an encrypted virtual private network (VPN) tunnel to Guardian’s servers, outsourcing all of the filtering and enforcement to the cloud to help reduce performance issues on the device’s battery. It means the Guardian app can near-instantly spot if another app is secretly sending a device’s tracking data to a tracking firm, warning the user or giving the option to stop it in its tracks. The aim isn’t to prevent a potentially dodgy app from working properly, but to give users’ awareness and choice over what data leaves their device.

Strafach described the app as “like a junk email filter for your web traffic,” and you can see from of the app’s dedicated tabs what data gets blocked and why. A future version plans to allow users to modify or block their precise geolocation from being sent to certain servers. Strafach said the app will later tell a user how many times an app accesses device data, like their contact lists.

But unlike other ad and tracker blockers, the app doesn’t use overkill third-party lists that prevent apps from working properly. Instead, taking a tried-and-tested approach from the team’s own research. The team periodically scans a range of apps in the App Store to help identify problematic and privacy-invasive issues that are fed to the app to help improve over time. If an app is known to have security issues, the Guardian app can alert a user to the threat. The team plans to continue building machine learning models that help to identify new threats — including so-called “aggressive ads” — that hijack your mobile browser and redirect you to dodgy pages or apps.

Screenshots of the Guardian app, set to be released in December (Image: supplied)

Strafach said that the app will “err on the side of usability” by warning users first — with the option of blocking it. A planned future option will allow users to go into a higher, more restrictive privacy level — “Lockdown mode” — which will deny bad traffic by default until the user intervenes.

What sets the Guardian app from its distant competitors is its anti-data collection.

Whenever you use a VPN — to evade censorship, site blocks or surveillance — you have to put more trust in the VPN server to keep all of your internet traffic safe than your internet provider or cell carrier. Strafach said that neither he nor the team wants to know who uses the app. The less data they have, the less they know, and the safer and more private its users are.

“We don’t want to collect data that we don’t need,” said Strafach. “We consider data a liability. Our rule is to collect as little as possible. We don’t even use Google Analytics or any kind of tracking in the app — or even on our site, out of principle.”

The app works by generating a random set of VPN credentials to connect to the cloud. The connection uses IPSec (IKEv2) with a strong cipher suite, he said. In other words, the Guardian app isn’t a creepy VPN app like Facebook’s Onavo, which Apple pulled from the App Store for collecting data it shouldn’t have been. “On the server side, we’ll only see a random device identifier, because we don’t have accounts so you can’t be attributable to your traffic,” he said.

“We don’t even want to say ‘you can trust us not to do anything,’ because we don’t want to be in a position that we have to be trusted,” he said. “We really just want to run our business the old fashioned way. We want people to pay for our product and we provide them service, and we don’t want their data or send them marketing.”

“It’s a very hard line,” he said. “We would shut down before we even have to face that kind of decision. It would go against our core principles.”

I’ve been using the app for the past week. It’s surprisingly easy to use. For a semi-advanced user, it can feel unnatural to flip a virtual switch on the app’s main screen and allow it to run its course. Anyone who cares about their security and privacy are often always aware of their “opsec” — one wrong move and it can blow your anonymity shield wide open. Overall, the app works well. It’s non-intrusive, it doesn’t interfere, but with the “VPN” icon lit up at the top of the screen, there’s a constant reminder that the app is working in the background.

It’s impressive how much the team has kept privacy and anonymity so front of mind throughout the app’s design process — even down to allowing users to pay by Apple Pay and through in-app purchases so that no billing information is ever exchanged.

The app doesn’t appear to slow down the connection when browsing the web or scrolling through Twitter or Facebook, on neither LTE or a Wi-Fi network. Even streaming a medium-quality live video stream didn’t cause any issues. But it’s still early days, and even though the closed beta has a few hundred users — myself included — as with any bandwidth-intensive cloud service, the quality could fluctuate over time. Strafach said that the backend infrastructure is scalable and can plug-and-play with almost any cloud service in the case of outages.

In its pre-launch state, the company is financially healthy, scoring a round of initial seed funding to support getting the team together, the app’s launch, and maintaining its cloud infrastructure. Steve Russell, an experienced investor and board member, said he was “impressed” with the team’s vision and technology.

“Quality solutions for mobile security and privacy are desperately needed, and Guardian distinguishes itself both in its uniqueness and its effectiveness,” said Russell in an email.

He added that the team is “world class,” and has built a product that’s “sorely needed.”

Strafach said the team is running financially conservatively ahead of its public reveal, but that the startup is looking to raise a Series A to support its anticipated growth — but also the team’s research that feeds the app with new data. “There’s a lot we want to look into and we want to put out more reports on quite a few different topics,” he said.

As the team continue to find new threats, the better the app will become.

The app’s early adopter program is open, including its premium options. The app is expected to launch fully in December.

Source: https://techcrunch.com/2018/10/24/smart-firewall-guardian-iphone-app-privacy-before-profits/

How Companies Learn Your Secrets

Andrew Pole had just started working as a statistician for Target in 2002, when two colleagues from the marketing department stopped by his desk to ask an odd question: “If we wanted to figure out if a customer is pregnant, even if she didn’t want us to know, can you do that? ”

Pole has a master’s degree in statistics and another in economics, and has been obsessed with the intersection of data and human behavior most of his life. His parents were teachers in North Dakota, and while other kids were going to 4-H, Pole was doing algebra and writing computer programs. “The stereotype of a math nerd is true,” he told me when I spoke with him last year. “I kind of like going out and evangelizing analytics.”

As the marketers explained to Pole — and as Pole later explained to me, back when we were still speaking and before Target told him to stop — new parents are a retailer’s holy grail. Most shoppers don’t buy everything they need at one store. Instead, they buy groceries at the grocery store and toys at the toy store, and they visit Target only when they need certain items they associate with Target — cleaning supplies, say, or new socks or a six-month supply of toilet paper. But Target sells everything from milk to stuffed animals to lawn furniture to electronics, so one of the company’s primary goals is convincing customers that the only store they need is Target. But it’s a tough message to get across, even with the most ingenious ad campaigns, because once consumers’ shopping habits are ingrained, it’s incredibly difficult to change them.

There are, however, some brief periods in a person’s life when old routines fall apart and buying habits are suddenly in flux. One of those moments — the moment, really — is right around the birth of a child, when parents are exhausted and overwhelmed and their shopping patterns and brand loyalties are up for grabs. But as Target’s marketers explained to Pole, timing is everything. Because birth records are usually public, the moment a couple have a new baby, they are almost instantaneously barraged with offers and incentives and advertisements from all sorts of companies. Which means that the key is to reach them earlier, before any other retailers know a baby is on the way. Specifically, the marketers said they wanted to send specially designed ads to women in their second trimester, which is when most expectant mothers begin buying all sorts of new things, like prenatal vitamins and maternity clothing. “Can you give us a list?” the marketers asked.

“We knew that if we could identify them in their second trimester, there’s a good chance we could capture them for years,” Pole told me. “As soon as we get them buying diapers from us, they’re going to start buying everything else too. If you’re rushing through the store, looking for bottles, and you pass orange juice, you’ll grab a carton. Oh, and there’s that new DVD I want. Soon, you’ll be buying cereal and paper towels from us, and keep coming back.”

Continue reading the main story

The desire to collect information on customers is not new for Target or any other large retailer, of course. For decades, Target has collected vast amounts of data on every person who regularly walks into one of its stores. Whenever possible, Target assigns each shopper a unique code — known internally as the Guest ID number — that keeps tabs on everything they buy. “If you use a credit card or a coupon, or fill out a survey, or mail in a refund, or call the customer help line, or open an e-mail we’ve sent you or visit our Web site, we’ll record it and link it to your Guest ID,” Pole said. “We want to know everything we can.”

Also linked to your Guest ID is demographic information like your age, whether you are married and have kids, which part of town you live in, how long it takes you to drive to the store, your estimated salary, whether you’ve moved recently, what credit cards you carry in your wallet and what Web sites you visit. Target can buy data about your ethnicity, job history, the magazines you read, if you’ve ever declared bankruptcy or got divorced, the year you bought (or lost) your house, where you went to college, what kinds of topics you talk about online, whether you prefer certain brands of coffee, paper towels, cereal or applesauce, your political leanings, reading habits, charitable giving and the number of cars you own. (In a statement, Target declined to identify what demographic information it collects or purchases.) All that information is meaningless, however, without someone to analyze and make sense of it. That’s where Andrew Pole and the dozens of other members of Target’s Guest Marketing Analytics department come in.

Almost every major retailer, from grocery chains to investmentbanks to the U.S. Postal Service, has a “predictive analytics” department devoted to understanding not just consumers’ shopping habits but also their personal habits, so as to more efficiently market to them. “But Target has always been one of the smartest at this,” says Eric Siegel, a consultant and the chairman of a conference called Predictive Analytics World. “We’re living through a golden age of behavioral research. It’s amazing how much we can figure out about how people think now.”

The reason Target can snoop on our shopping habits is that, over the past two decades, the science of habit formation has become a major field of research in neurology and psychology departments at hundreds of major medical centers and universities, as well as inside extremely well financed corporate labs. “It’s like an arms race to hire statisticians nowadays,” said Andreas Weigend, the former chief scientist at Amazon.com. “Mathematicians are suddenly sexy.” As the ability to analyze data has grown more and more fine-grained, the push to understand how daily habits influence our decisions has become one of the most exciting topics in clinical research, even though most of us are hardly aware those patterns exist. One study from Duke University estimated that habits, rather than conscious decision-making, shape 45 percent of the choices we make every day, and recent discoveries have begun to change everything from the way we think about dieting to how doctors conceive treatments for anxiety, depression and addictions.

This research is also transforming our understanding of how habits function across organizations and societies. A football coach named Tony Dungy propelled one of the worst teams in the N.F.L. to the Super Bowl by focusing on how his players habitually reacted to on-field cues. Before he became Treasury secretary, Paul O’Neill overhauled a stumbling conglomerate, Alcoa, and turned it into a top performer in the Dow Jones by relentlessly attacking one habit — a specific approach to worker safety — which in turn caused a companywide transformation. The Obama campaign has hired a habit specialist as its “chief scientist” to figure out how to trigger new voting patterns among different constituencies.

Researchers have figured out how to stop people from habitually overeating and biting their nails. They can explain why some of us automatically go for a jog every morning and are more productive at work, while others oversleep and procrastinate. There is a calculus, it turns out, for mastering our subconscious urges. For companies like Target, the exhaustive rendering of our conscious and unconscious patterns into data sets and algorithms has revolutionized what they know about us and, therefore, how precisely they can sell.

Inside the brain-and-cognitive-sciences department of the Massachusetts Institute of Technology are what, to the casual observer, look like dollhouse versions of surgical theaters. There are rooms with tiny scalpels, small drills and miniature saws. Even the operating tables are petite, as if prepared for 7-year-old surgeons. Inside those shrunken O.R.’s, neurologists cut into the skulls of anesthetized rats, implanting tiny sensors that record the smallest changes in the activity of their brains.

An M.I.T. neuroscientist named Ann Graybiel told me that she and her colleagues began exploring habits more than a decade ago by putting their wired rats into a T-shaped maze with chocolate at one end. The maze was structured so that each animal was positioned behind a barrier that opened after a loud click. The first time a rat was placed in the maze, it would usually wander slowly up and down the center aisle after the barrier slid away, sniffing in corners and scratching at walls. It appeared to smell the chocolate but couldn’t figure out how to find it. There was no discernible pattern in the rat’s meanderings and no indication it was working hard to find the treat.

The probes in the rats’ heads, however, told a different story. While each animal wandered through the maze, its brain was working furiously. Every time a rat sniffed the air or scratched a wall, the neurosensors inside the animal’s head exploded with activity. As the scientists repeated the experiment, again and again, the rats eventually stopped sniffing corners and making wrong turns and began to zip through the maze with more and more speed. And within their brains, something unexpected occurred: as each rat learned how to complete the maze more quickly, its mental activity decreased. As the path became more and more automatic — as it became a habit — the rats started thinking less and less.

This process, in which the brain converts a sequence of actions into an automatic routine, is called “chunking.” There are dozens, if not hundreds, of behavioral chunks we rely on every day. Some are simple: you automatically put toothpaste on your toothbrush before sticking it in your mouth. Some, like making the kids’ lunch, are a little more complex. Still others are so complicated that it’s remarkable to realize that a habit could have emerged at all.

Take backing your car out of the driveway. When you first learned to drive, that act required a major dose of concentration, and for good reason: it involves peering into the rearview and side mirrors and checking for obstacles, putting your foot on the brake, moving the gearshift into reverse, removing your foot from the brake, estimating the distance between the garage and the street while keeping the wheels aligned, calculating how images in the mirrors translate into actual distances, all while applying differing amounts of pressure to the gas pedal and brake.

Now, you perform that series of actions every time you pull into the street without thinking very much. Your brain has chunked large parts of it. Left to its own devices, the brain will try to make almost any repeated behavior into a habit, because habits allow our minds to conserve effort. But conserving mental energy is tricky, because if our brains power down at the wrong moment, we might fail to notice something important, like a child riding her bike down the sidewalk or a speeding car coming down the street. So we’ve devised a clever system to determine when to let a habit take over. It’s something that happens whenever a chunk of behavior starts or ends — and it helps to explain why habits are so difficult to change once they’re formed, despite our best intentions.

To understand this a little more clearly, consider again the chocolate-seeking rats. What Graybiel and her colleagues found was that, as the ability to navigate the maze became habitual, there were two spikes in the rats’ brain activity — once at the beginning of the maze, when the rat heard the click right before the barrier slid away, and once at the end, when the rat found the chocolate. Those spikes show when the rats’ brains were fully engaged, and the dip in neural activity between the spikes showed when the habit took over. From behind the partition, the rat wasn’t sure what waited on the other side, until it heard the click, which it had come to associate with the maze. Once it heard that sound, it knew to use the “maze habit,” and its brain activity decreased. Then at the end of the routine, when the reward appeared, the brain shook itself awake again and the chocolate signaled to the rat that this particular habit was worth remembering, and the neurological pathway was carved that much deeper.

The process within our brains that creates habits is a three-step loop. First, there is a cue, a trigger that tells your brain to go into automatic mode and which habit to use. Then there is the routine, which can be physical or mental or emotional. Finally, there is a reward, which helps your brain figure out if this particular loop is worth remembering for the future. Over time, this loop — cue, routine, reward; cue, routine, reward — becomes more and more automatic. The cue and reward become neurologically intertwined until a sense of craving emerges. What’s unique about cues and rewards, however, is how subtle they can be. Neurological studies like the ones in Graybiel’s lab have revealed that some cues span just milliseconds. And rewards can range from the obvious (like the sugar rush that a morning doughnut habit provides) to the infinitesimal (like the barely noticeable — but measurable — sense of relief the brain experiences after successfully navigating the driveway). Most cues and rewards, in fact, happen so quickly and are so slight that we are hardly aware of them at all. But our neural systems notice and use them to build automatic behaviors.

Habits aren’t destiny — they can be ignored, changed or replaced. But it’s also true that once the loop is established and a habit emerges, your brain stops fully participating in decision-making. So unless you deliberately fight a habit — unless you find new cues and rewards — the old pattern will unfold automatically.

“We’ve done experiments where we trained rats to run down a maze until it was a habit, and then we extinguished the habit by changing the placement of the reward,” Graybiel told me. “Then one day, we’ll put the reward in the old place and put in the rat and, by golly, the old habit will re-emerge right away. Habits never really disappear.”

Luckily, simply understanding how habits work makes them easier to control. Take, for instance, a series of studies conducted a few years ago at Columbia University and the University of Alberta. Researchers wanted to understand how exercise habits emerge. In one project, 256 members of a health-insurance plan were invited to classes stressing the importance of exercise. Half the participants received an extra lesson on the theories of habit formation (the structure of the habit loop) and were asked to identify cues and rewards that might help them develop exercise routines.

The results were dramatic. Over the next four months, those participants who deliberately identified cues and rewards spent twice as much time exercising as their peers. Other studies have yielded similar results. According to another recent paper, if you want to start running in the morning, it’s essential that you choose a simple cue (like always putting on your sneakers before breakfast or leaving your running clothes next to your bed) and a clear reward (like a midday treat or even the sense of accomplishment that comes from ritually recording your miles in a log book). After a while, your brain will start anticipating that reward — craving the treat or the feeling of accomplishment — and there will be a measurable neurological impulse to lace up your jogging shoes each morning.

Our relationship to e-mail operates on the same principle. When a computer chimes or a smartphone vibrates with a new message, the brain starts anticipating the neurological “pleasure” (even if we don’t recognize it as such) that clicking on the e-mail and reading it provides. That expectation, if unsatisfied, can build until you find yourself moved to distraction by the thought of an e-mail sitting there unread — even if you know, rationally, it’s most likely not important. On the other hand, once you remove the cue by disabling the buzzing of your phone or the chiming of your computer, the craving is never triggered, and you’ll find, over time, that you’re able to work productively for long stretches without checking your in-box.

Some of the most ambitious habit experiments have been conducted by corporate America. To understand why executives are so entranced by this science, consider how one of the world’s largest companies, Procter & Gamble, used habit insights to turn a failing product into one of its biggest sellers. P.& G. is the corporate behemoth behind a whole range of products, from Downy fabric softener to Bounty paper towels to Duracell batteries and dozens of other household brands. In the mid-1990s, P.& G.’s executives began a secret project to create a new product that could eradicate bad smells. P.& G. spent millions developing a colorless, cheap-to-manufacture liquid that could be sprayed on a smoky blouse, stinky couch, old jacket or stained car interior and make it odorless. In order to market the product — Febreze — the company formed a team that included a former Wall Street mathematician named Drake Stimson and habit specialists, whose job was to make sure the television commercials, which they tested in Phoenix, Salt Lake City and Boise, Idaho, accentuated the product’s cues and rewards just right.

Video

TimesCast | Retailers‘ Predictions

February 16, 2012 – In a preview of this Sunday’s New York Times Magazine, Charles Duhigg details how some retailers profit by predicting major changes in your life.

By Kassie Bracken on Publish Date February 17, 2012. . Watch in Times Video »

The first ad showed a woman complaining about the smoking section of a restaurant. Whenever she eats there, she says, her jacket smells like smoke. A friend tells her that if she uses Febreze, it will eliminate the odor. The cue in the ad is clear: the harsh smell of cigarette smoke. The reward: odor eliminated from clothes. The second ad featured a woman worrying about her dog, Sophie, who always sits on the couch. “Sophie will always smell like Sophie,” she says, but with Febreze, “now my furniture doesn’t have to.” The ads were put in heavy rotation. Then the marketers sat back, anticipating how they would spend their bonuses. A week passed. Then two. A month. Two months. Sales started small and got smaller. Febreze was a dud.

The panicked marketing team canvassed consumers and conducted in-depth interviews to figure out what was going wrong, Stimson recalled. Their first inkling came when they visited a woman’s home outside Phoenix. The house was clean and organized. She was something of a neat freak, the woman explained. But when P.& G.’s scientists walked into her living room, where her nine cats spent most of their time, the scent was so overpowering that one of them gagged.

According to Stimson, who led the Febreze team, a researcher asked the woman, “What do you do about the cat smell?”

“It’s usually not a problem,” she said.

“Do you smell it now?”

“No,” she said. “Isn’t it wonderful? They hardly smell at all!”

A similar scene played out in dozens of other smelly homes. The reason Febreze wasn’t selling, the marketers realized, was that people couldn’t detect most of the bad smells in their lives. If you live with nine cats, you become desensitized to their scents. If you smoke cigarettes, eventually you don’t smell smoke anymore. Even the strongest odors fade with constant exposure. That’s why Febreze was a failure. The product’s cue — the bad smells that were supposed to trigger daily use — was hidden from the people who needed it the most. And Febreze’s reward (an odorless home) was meaningless to someone who couldn’t smell offensive scents in the first place.

P.& G. employed a Harvard Business School professor to analyze Febreze’s ad campaigns. They collected hours of footage of people cleaning their homes and watched tape after tape, looking for clues that might help them connect Febreze to people’s daily habits. When that didn’t reveal anything, they went into the field and conducted more interviews. A breakthrough came when they visited a woman in a suburb near Scottsdale, Ariz., who was in her 40s with four children. Her house was clean, though not compulsively tidy, and didn’t appear to have any odor problems; there were no pets or smokers. To the surprise of everyone, she loved Febreze.

“I use it every day,” she said.

“What smells are you trying to get rid of?” a researcher asked.

“I don’t really use it for specific smells,” the woman said. “I use it for normal cleaning — a couple of sprays when I’m done in a room.”

The researchers followed her around as she tidied the house. In the bedroom, she made her bed, tightened the sheet’s corners, then sprayed the comforter with Febreze. In the living room, she vacuumed, picked up the children’s shoes, straightened the coffee table, then sprayed Febreze on the freshly cleaned carpet.

“It’s nice, you know?” she said. “Spraying feels like a little minicelebration when I’m done with a room.” At the rate she was going, the team estimated, she would empty a bottle of Febreze every two weeks.

When they got back to P.& G.’s headquarters, the researchers watched their videotapes again. Now they knew what to look for and saw their mistake in scene after scene. Cleaning has its own habit loops that already exist. In one video, when a woman walked into a dirty room (cue), she started sweeping and picking up toys (routine), then she examined the room and smiled when she was done (reward). In another, a woman scowled at her unmade bed (cue), proceeded to straighten the blankets and comforter (routine) and then sighed as she ran her hands over the freshly plumped pillows (reward). P.& G. had been trying to create a whole new habit with Febreze, but what they really needed to do was piggyback on habit loops that were already in place. The marketers needed to position Febreze as something that came at the end of the cleaning ritual, the reward, rather than as a whole new cleaning routine.

The company printed new ads showing open windows and gusts of fresh air. More perfume was added to the Febreze formula, so that instead of merely neutralizing odors, the spray had its own distinct scent. Television commercials were filmed of women, having finished their cleaning routine, using Febreze to spritz freshly made beds and just-laundered clothing. Each ad was designed to appeal to the habit loop: when you see a freshly cleaned room (cue), pull out Febreze (routine) and enjoy a smell that says you’ve done a great job (reward). When you finish making a bed (cue), spritz Febreze (routine) and breathe a sweet, contented sigh (reward). Febreze, the ads implied, was a pleasant treat, not a reminder that your home stinks.

And so Febreze, a product originally conceived as a revolutionary way to destroy odors, became an air freshener used once things are already clean. The Febreze revamp occurred in the summer of 1998. Within two months, sales doubled. A year later, the product brought in $230 million. Since then Febreze has spawned dozens of spinoffs — air fresheners, candles and laundry detergents — that now account for sales of more than $1 billion a year. Eventually, P.& G. began mentioning to customers that, in addition to smelling sweet, Febreze can actually kill bad odors. Today it’s one of the top-selling products in the world.

Andrew Pole was hired by Target to use the same kinds of insights into consumers’ habits to expand Target’s sales. His assignment was to analyze all the cue-routine-reward loops among shoppers and help the company figure out how to exploit them. Much of his department’s work was straightforward: find the customers who have children and send them catalogs that feature toys before Christmas. Look for shoppers who habitually purchase swimsuits in April and send them coupons for sunscreen in July and diet books in December. But Pole’s most important assignment was to identify those unique moments in consumers’ lives when their shopping habits become particularly flexible and the right advertisement or coupon would cause them to begin spending in new ways.

In the 1980s, a team of researchers led by a U.C.L.A. professor named Alan Andreasen undertook a study of peoples’ most mundane purchases, like soap, toothpaste, trash bags and toilet paper. They learned that most shoppers paid almost no attention to how they bought these products, that the purchases occurred habitually, without any complex decision-making. Which meant it was hard for marketers, despite their displays and coupons and product promotions, to persuade shoppers to change.

But when some customers were going through a major life event, like graduating from college or getting a new job or moving to a new town, their shopping habits became flexible in ways that were both predictable and potential gold mines for retailers. The study found that when someone marries, he or she is more likely to start buying a new type of coffee. When a couple move into a new house, they’re more apt to purchase a different kind of cereal. When they divorce, there’s an increased chance they’ll start buying different brands of beer.

Consumers going through major life events often don’t notice, or care, that their shopping habits have shifted, but retailers notice, and they care quite a bit. At those unique moments, Andreasen wrote, customers are “vulnerable to intervention by marketers.” In other words, a precisely timed advertisement, sent to a recent divorcee or new homebuyer, can change someone’s shopping patterns for years.

And among life events, none are more important than the arrival of a baby. At that moment, new parents’ habits are more flexible than at almost any other time in their adult lives. If companies can identify pregnant shoppers, they can earn millions.

The only problem is that identifying pregnant customers is harder than it sounds. Target has a baby-shower registry, and Pole started there, observing how shopping habits changed as a woman approached her due date, which women on the registry had willingly disclosed. He ran test after test, analyzing the data, and before long some useful patterns emerged. Lotions, for example. Lots of people buy lotion, but one of Pole’s colleagues noticed that women on the baby registry were buying larger quantities of unscented lotion around the beginning of their second trimester. Another analyst noted that sometime in the first 20 weeks, pregnant women loaded up on supplements like calcium, magnesium and zinc. Many shoppers purchase soap and cotton balls, but when someone suddenly starts buying lots of scent-free soap and extra-big bags of cotton balls, in addition to hand sanitizers and washcloths, it signals they could be getting close to their delivery date.

As Pole’s computers crawled through the data, he was able to identify about 25 products that, when analyzed together, allowed him to assign each shopper a “pregnancy prediction” score. More important, he could also estimate her due date to within a small window, so Target could send coupons timed to very specific stages of her pregnancy.

One Target employee I spoke to provided a hypothetical example. Take a fictional Target shopper named Jenny Ward, who is 23, lives in Atlanta and in March bought cocoa-butter lotion, a purse large enough to double as a diaper bag, zinc and magnesium supplements and a bright blue rug. There’s, say, an 87 percent chance that she’s pregnant and that her delivery date is sometime in late August. What’s more, because of the data attached to her Guest ID number, Target knows how to trigger Jenny’s habits. They know that if she receives a coupon via e-mail, it will most likely cue her to buy online. They know that if she receives an ad in the mail on Friday, she frequently uses it on a weekend trip to the store. And they know that if they reward her with a printed receipt that entitles her to a free cup of Starbucks coffee, she’ll use it when she comes back again.

In the past, that knowledge had limited value. After all, Jenny purchased only cleaning supplies at Target, and there were only so many psychological buttons the company could push. But now that she is pregnant, everything is up for grabs. In addition to triggering Jenny’s habits to buy more cleaning products, they can also start including offers for an array of products, some more obvious than others, that a woman at her stage of pregnancy might need.

Pole applied his program to every regular female shopper in Target’s national database and soon had a list of tens of thousands of women who were most likely pregnant. If they could entice those women or their husbands to visit Target and buy baby-related products, the company’s cue-routine-reward calculators could kick in and start pushing them to buy groceries, bathing suits, toys and clothing, as well. When Pole shared his list with the marketers, he said, they were ecstatic. Soon, Pole was getting invited to meetings above his paygrade. Eventually his paygrade went up.

At which point someone asked an important question: How are women going to react when they figure out how much Target knows?

“If we send someone a catalog and say, ‘Congratulations on your first child!’ and they’ve never told us they’re pregnant, that’s going to make some people uncomfortable,” Pole told me. “We are very conservative about compliance with all privacy laws. But even if you’re following the law, you can do things where people get queasy.”

About a year after Pole created his pregnancy-prediction model, a man walked into a Target outside Minneapolis and demanded to see the manager. He was clutching coupons that had been sent to his daughter, and he was angry, according to an employee who participated in the conversation.

Video

How to Break the Cookie Habit

Charles Duhigg explains the science of habits.

By Random House on Publish Date February 16, 2012. . Watch in Times Video »

“My daughter got this in the mail!” he said. “She’s still in high school, and you’re sending her coupons for baby clothes and cribs? Are you trying to encourage her to get pregnant?”

The manager didn’t have any idea what the man was talking about. He looked at the mailer. Sure enough, it was addressed to the man’s daughter and contained advertisements for maternity clothing, nursery furniture and pictures of smiling infants. The manager apologized and then called a few days later to apologize again.

On the phone, though, the father was somewhat abashed. “I had a talk with my daughter,” he said. “It turns out there’s been some activities in my house I haven’t been completely aware of. She’s due in August. I owe you an apology.”

When I approached Target to discuss Pole’s work, its representatives declined to speak with me. “Our mission is to make Target the preferred shopping destination for our guests by delivering outstanding value, continuous innovation and exceptional guest experience,” the company wrote in a statement. “We’ve developed a number of research tools that allow us to gain insights into trends and preferences within different demographic segments of our guest population.” When I sent Target a complete summary of my reporting, the reply was more terse: “Almost all of your statements contain inaccurate information and publishing them would be misleading to the public. We do not intend to address each statement point by point.” The company declined to identify what was inaccurate. They did add, however, that Target “is in compliance with all federal and state laws, including those related to protected health information.”

When I offered to fly to Target’s headquarters to discuss its concerns, a spokeswoman e-mailed that no one would meet me. When I flew out anyway, I was told I was on a list of prohibited visitors. “I’ve been instructed not to give you access and to ask you to leave,” said a very nice security guard named Alex.

Using data to predict a woman’s pregnancy, Target realized soon after Pole perfected his model, could be a public-relations disaster. So the question became: how could they get their advertisements into expectant mothers’ hands without making it appear they were spying on them? How do you take advantage of someone’s habits without letting them know you’re studying their lives?

Before I met Andrew Pole, before I even decided to write a book about the science of habit formation, I had another goal: I wanted to lose weight.

I had got into a bad habit of going to the cafeteria every afternoon and eating a chocolate-chip cookie, which contributed to my gaining a few pounds. Eight, to be precise. I put a Post-it note on my computer reading “NO MORE COOKIES.” But every afternoon, I managed to ignore that note, wander to the cafeteria, buy a cookie and eat it while chatting with colleagues. Tomorrow, I always promised myself, I’ll muster the willpower to resist.

Tomorrow, I ate another cookie.

When I started interviewing experts in habit formation, I concluded each interview by asking what I should do. The first step, they said, was to figure out my habit loop. The routine was simple: every afternoon, I walked to the cafeteria, bought a cookie and ate it while chatting with friends.

Next came some less obvious questions: What was the cue? Hunger? Boredom? Low blood sugar? And what was the reward? The taste of the cookie itself? The temporary distraction from my work? The chance to socialize with colleagues?

Rewards are powerful because they satisfy cravings, but we’re often not conscious of the urges driving our habits in the first place. So one day, when I felt a cookie impulse, I went outside and took a walk instead. The next day, I went to the cafeteria and bought a coffee. The next, I bought an apple and ate it while chatting with friends. You get the idea. I wanted to test different theories regarding what reward I was really craving. Was it hunger? (In which case the apple should have worked.) Was it the desire for a quick burst of energy? (If so, the coffee should suffice.) Or, as turned out to be the answer, was it that after several hours spent focused on work, I wanted to socialize, to make sure I was up to speed on office gossip, and the cookie was just a convenient excuse? When I walked to a colleague’s desk and chatted for a few minutes, it turned out, my cookie urge was gone.

All that was left was identifying the cue.

Deciphering cues is hard, however. Our lives often contain too much information to figure out what is triggering a particular behavior. Do you eat breakfast at a certain time because you’re hungry? Or because the morning news is on? Or because your kids have started eating? Experiments have shown that most cues fit into one of five categories: location, time, emotional state, other people or the immediately preceding action. So to figure out the cue for my cookie habit, I wrote down five things the moment the urge hit:

Where are you? (Sitting at my desk.)

What time is it? (3:36 p.m.)

What’s your emotional state? (Bored.)

Who else is around? (No one.)

What action preceded the urge? (Answered an e-mail.)

The next day I did the same thing. And the next. Pretty soon, the cue was clear: I always felt an urge to snack around 3:30.

Once I figured out all the parts of the loop, it seemed fairly easy to change my habit. But the psychologists and neuroscientists warned me that, for my new behavior to stick, I needed to abide by the same principle that guided Procter & Gamble in selling Febreze: To shift the routine — to socialize, rather than eat a cookie — I needed to piggyback on an existing habit. So now, every day around 3:30, I stand up, look around the newsroom for someone to talk to, spend 10 minutes gossiping, then go back to my desk. The cue and reward have stayed the same. Only the routine has shifted. It doesn’t feel like a decision, any more than the M.I.T. rats made a decision to run through the maze. It’s now a habit. I’ve lost 21 pounds since then (12 of them from changing my cookie ritual).

After Andrew Pole built his pregnancy-prediction model, after he identified thousands of female shoppers who were most likely pregnant, after someone pointed out that some of those women might be a little upset if they received an advertisement making it obvious Target was studying their reproductive status, everyone decided to slow things down.

The marketing department conducted a few tests by choosing a small, random sample of women from Pole’s list and mailing them combinations of advertisements to see how they reacted.

“We have the capacity to send every customer an ad booklet, specifically designed for them, that says, ‘Here’s everything you bought last week and a coupon for it,’ ” one Target executive told me. “We do that for grocery products all the time.” But for pregnant women, Target’s goal was selling them baby items they didn’t even know they needed yet.

“With the pregnancy products, though, we learned that some women react badly,” the executive said. “Then we started mixing in all these ads for things we knew pregnant women would never buy, so the baby ads looked random. We’d put an ad for a lawn mower next to diapers. We’d put a coupon for wineglasses next to infant clothes. That way, it looked like all the products were chosen by chance.

“And we found out that as long as a pregnant woman thinks she hasn’t been spied on, she’ll use the coupons. She just assumes that everyone else on her block got the same mailer for diapers and cribs. As long as we don’t spook her, it works.”

In other words, if Target piggybacked on existing habits — the same cues and rewards they already knew got customers to buy cleaning supplies or socks — then they could insert a new routine: buying baby products, as well. There’s a cue (“Oh, a coupon for something I need!”) a routine (“Buy! Buy! Buy!”) and a reward (“I can take that off my list”). And once the shopper is inside the store, Target will hit her with cues and rewards to entice her to purchase everything she normally buys somewhere else. As long as Target camouflaged how much it knew, as long as the habit felt familiar, the new behavior took hold.

Soon after the new ad campaign began, Target’s Mom and Baby sales exploded. The company doesn’t break out figures for specific divisions, but between 2002 — when Pole was hired — and 2010, Target’s revenues grew from $44 billion to $67 billion. In 2005, the company’s president, Gregg Steinhafel, boasted to a room of investors about the company’s “heightened focus on items and categories that appeal to specific guest segments such as mom and baby.”

Pole was promoted. He has been invited to speak at conferences. “I never expected this would become such a big deal,” he told me the last time we spoke.

A few weeks before this article went to press, I flew to Minneapolis to try and speak to Andrew Pole one last time. I hadn’t talked to him in more than a year. Back when we were still friendly, I mentioned that my wife was seven months pregnant. We shop at Target, I told him, and had given the company our address so we could start receiving coupons in the mail. As my wife’s pregnancy progressed, I noticed a subtle upswing in the number of advertisements for diapers and baby clothes arriving at our house.

Pole didn’t answer my e-mails or phone calls when I visited Minneapolis. I drove to his large home in a nice suburb, but no one answered the door. On my way back to the hotel, I stopped at a Target to pick up some deodorant, then also bought some T-shirts and a fancy hair gel. On a whim, I threw in some pacifiers, to see how the computers would react. Besides, our baby is now 9 months old. You can’t have too many pacifiers.

When I paid, I didn’t receive any sudden deals on diapers or formula, to my slight disappointment. It made sense, though: I was shopping in a city I never previously visited, at 9:45 p.m. on a weeknight, buying a random assortment of items. I was using a corporate credit card, and besides the pacifiers, hadn’t purchased any of the things that a parent needs. It was clear to Target’s computers that I was on a business trip. Pole’s prediction calculator took one look at me, ran the numbers and decided to bide its time. Back home, the offers would eventually come. As Pole told me the last time we spoke: “Just wait. We’ll be sending you coupons for things you want before you even know you want them.”

Source: https://www.nytimes.com/2012/02/19/magazine/shopping-habits.html February 2012