All topics

Big data for investment management and quant analysis

The use of big data in investment management - a review of recent literature 

“Big Data” is a Big Topic. 

Data is growing exponentially and many investment analysts believe this has the potential to transform modern economics, driving innovation and giving a competitive edge to those who are able to harness its power.

the big data machine - the new approach to investment research

How can investment management firms gain an edge from Big Data analytics? Many asset management companies, hedge funds and proprietary traders are already investing to find innovative ways of using big data for investment research – and you can be sure that the best ideas will be closely guarded secrets. For example, investment bank Goldman Sachs reports the use of satellite images of retail car parks as an indicator of retail sales – at an aggregate level or for individual companies.

Big Data can also be interrogated by fund managers using quantitative analysis - to identify hidden relationships – at a company level or an econometric level – to give investors an edge.

One area receiving wide attention is the processing of media data – analysing news stories, tweets, and posts to Facebook, Instagram, Pinterest et al, and examining the tone of the language to determine the popularity of products. A similar methodology is being applied to use implied employee satisfaction as an indicator of company earnings' surprises.

In many applications of Big Data, harnessing the potential of the datasets is of course a huge challenge, requiring advanced processes such as machine learning, language processing and other technologies in the drive to gain insight and create alpha.

Big Data is already having a Big Impact on factor analysis. “Smart beta” is a relatively straightforward application of data science, and in the space of a few years, the smart beta philosophy has gained widespread acceptance. Smart beta has blurred the lines between active and passive, generating alpha by identifying risk factors which carry an expectation of excess return over time. And the number of risk factors identified is accelerating as more data becomes available and processing techniques improve. The key here, of course, is to process data intelligently in order to establish which risk factors really are likely to generate alpha out-of-sample, and which are the result of spurious data-mining. The same applies, on a much larger scale, when “big data” is used.

Big Data is driving a Big Battle for Talent. With 2.5 quintillion bytes of data being produced every day, the value of Big Data reaches far beyond stock market analysis, investment research and quantitative finance. The potential applications span multiple industries and geographies. Data scientists and quant analysts are in high demand, with Accenture reporting in a recent survey that 69% of large companies are planning to increase investment in data science over the next 12 months.

Browse our full library of white papers on quantitative methods

See our complete PDF list of white papers on alpha trading strategies

Big data – references and recommended reading list (all PDF)

White papers (PDF) on the use of big data in investment management, for investment research or investment marketing: 

Goldman Sachs (2015) The Big Data Revolution

BlackRock (2015) Finding Big Alpha in Big Data: The Evolution of Active Investing

BlackRock (2015) Bond Investors Must Harvest Big Data's Brave New World

KPMG (2015) Data and Analytics: New Drivers of Performance and Valuation

Infosys (2015) Using Big Data Technology in Capital Markets

BNY Mellon (2015) Asset Management: Applying Big Data to product development and client satisfaction

Celent (2014) Big Data in Wealth Management: Searching for Consumer Insight

SimCorp (2015) Big Data: What is it and what might it mean to asset managers?


Join Savvy Investor and keep up-to-date with new research

The flipside to exponential growth of data, of course, is the everyday problem of data overload that we all face. At Savvy Investor, we try to make life easier for institutional investors by curating the best papers, reports, news and blogs from around the web, and then providing a tailored experience to members who specify the topics they wish to follow. Why not register today?


See Andrew’s Linkedin bio at
To contact him, email