The 1992 film Sneakers features a scene where Sir Ben Kingsley and Robert Redford’s characters – both activist hackers whose lives have taken divergent paths – discuss the idea of power.
‘It’s not about who’s got the most bullets,’ Kingsley states, ‘It’s about who controls the information. What we see and hear, how we work, what we think… it’s all about the information!’
Over twenty years later those words are increasingly becoming a reality, but the real power isn’t merely the acquisition of information, instead it resides in being able to find the patterns hidden in its midst. Trend analysts have of course been doing this very thing for years, but due to our ever expanding use of the internet for all aspects of modern life there is now so much data being created that it requires a new approach – this is the world of ‘Big Data’.
IBM recently stated that people create a staggering 2.5 quintillion bytes of data every day (that’s roughly equivalent to over half a billion HD movie downloads). This information is generated from a huge variety of sources including social media posts, digital pictures, videos, retail transactions, and even the GPS tracking functions of mobile phones. In fact IBM go on to say that ‘90% of the data created in the world today has been created in the last two years.’
Harnessing this vast array of information is no easy matter, requiring superfast computers that can process colossal amounts of data and produce results in time frames that would have been impossible only a few years ago. Even the way the data itself is stored and arranged is a far stretch from the kind of spreadsheets or databases we have used in the past. But the potential rewards for this substantial investment in cutting edge technology and software are so great that big data research is rapidly becoming the digital equivalent of the 19th century gold rush.
‘Big data marks the beginning of a major transformation,’ explain authors Kenneth Cukier and Viktor Mayer-Schonberger in their book Big Data. ‘Just as the telescope enabled us to comprehend the universe and the microscope allowed us to understand germs, the new techniques for collecting and analyzing huge bodies of data will help us make sense of our world in ways we are just starting to appreciate.’ Financial institutions now pay handsomely for analysis that allows them to predict the future movements of markets, while insurance companies use it to determine the potential risks of each policy holder. The medical profession is discovering how careful examination of large data sets can help them diagnose diseases such as breast cancer faster, and even law enforcement is making use of the predictive power that big data allows.
A recent episode of the Horizon programme on the BBC followed the work of Professor Jeff Brantingham, whose PredPol system was helping lower crime in districts of Los Angeles. The analytic software he and his team developed interrogated historical and current crime records, alongside wider data for the areas in question, looking for complex trends. It then produced a list of predictions on where and when crimes would be most likely to occur, which the police department used to strategically deploy officers in those areas. Now this might sound like something out of Minority Report, but at the end of the six month trial the police confirmed that burglaries were down by 27%, and the areas that used the software had seen a 13% overall decrease in crime as opposed to the 0.4% by those using traditional methods.
‘We all like to think that we’re in control of everything,’ said Brantingham, ‘but in fact all of our behaviour is very regular, very patterned in ways that is often frightening to us. Offenders are no different, they do exactly the same things over and over and over again, and their criminal offending patterns emerge right out of that regularity of their behaviour.’
PredPol has proven so successful that it is being rolled out across several other divisions of the LAPD, and the system has also reached these shores as Kent Police is now trialling the software. But the power of big data isn’t just in predicting the future, it is also a key element in what has become known as Big Science, an area that seeks to discover the very origins of life itself and how our bodies work today.
The Human Genome project, which set out to decode the genetic building blocks of life, took a team of dedicated scientists a decade of intensive work to complete. The same task now, using the new technologies behind big data, would take as little as a day. Astronomers are able to map the heavens with much higher powered telescopes than ever before and collate the terabytes of data that they produce in a fraction of the time. Particle physicists working on the Large Hadron Collider were able to record the behaviour of subatomic particles travelling at unbelievable speeds and process that data to potentially discover the Higgs Boson (or ‘God’) particle, which could change the future of science in a profound way.
All of these advances in data collection and manipulation are exciting and possibly life-changing. None more so than the recently announced ‘Brain Research through Advancing Innovative Nanotechnologies’ programme (or BRAIN for short), whose goal is to map the activity of the human brain. Using state of the art equipment the scientists will be attempting to monitor, record and analyse millions, even billions, of neurons simultaneously. The programme, which was revealed by President Obama in April, hopes to find cures to conditions such as Alzheimer’s, strokes, and autism, with additional research into other areas of mental illness.
There are those who argue that the nature of big data means discoveries can be made without the scientists truly understanding the origin or motive of the problem – so they’ll know how something happens, but not necessarily why. This argument opens up the philosophical nature of research, but some big data proponents respond in more pragmatic terms. ‘As humans we have been conditioned to look for causes,’ Kenneth Cukier and Viktor Mayer-Schonberger explain, ‘even though searching for causality is often difficult and may lead us down the wrong paths. In a big-data world, by contrast, we won’t have to be fixated on causality; instead we can discover patterns and correlations in the data that offer us novel and invaluable insights. The correlations may not tell us precisely why something is happening, but they alert us that it is happening. And in many situations that is good enough.’
This article originally appeared on the PC Advisor website and in the print magazine as part of a monthly section I write entitled News Viewpoint. To see that version please click here, or purchase a copy of the fine magazine from your local newsagent.
Living With The Future 