Skip to main content
MENU
Subscribe
  • Sign Up Free
  • LOGIN
  • Subscribe
  • Topics
    • Alternatives
    • Consultants
    • Courts
    • Defined Contribution
    • ESG
    • ETFs
    • Face to Face
    • Hedge Funds
    • Industry Voices
    • Investing
    • Money Management
    • Partner Content
    • Pension Funds
    • Private Equity
    • Real Estate
    • Regulation
    • SECURE 2.0
    • Special Reports
    • Washington
    • White Papers
  • Rankings & Awards
    • 1,000 Largest Retirement Plans
    • Top-Performing Managers
    • Largest Money Managers
    • DC Money Managers
    • DC Record Keepers
    • Largest Hedge Fund Managers
    • World's Largest Retirement Funds
    • Best Places to Work in Money Management
    • Excellence & Innovation Awards
    • WPS Innovation Awards
    • Eddy Awards
  • ETFs
    • Latest ETF News
    • Fund Screener
    • Education Center
    • Equities
    • Fixed Income
    • Commodities
    • Actively Managed
    • Alternatives
    • ESG Rated
  • ESG
    • Latest ESG News
    • The Institutional Investor’s Guide to ESG Investing
    • ESG Sustainability - Gaining Momentum
    • ESG Investing | Industry Brief
    • Innovation in ESG Investing
    • 2023 ESG Investing Conference
    • ESG Rated ETFs
  • Defined Contribution
    • Latest DC News
    • The Plan Sponsor's Guide to Retirement Income
    • DC Money Manager Rankings
    • DC Record Keeper Rankings
    • Innovations in DC
    • Trends in DC: Focus on Retirement Income
    • 2023 Defined Contribution East Conference
  • Searches & Hires
    • Latest Searches & Hires News
    • Searches & Hires Database
    • RFPs
  • Research Center
    • The P&I Research Center
    • Earnings Tracker
    • Endowment Returns Tracker
    • Corporate Pension Contribution Tracker
    • Pension Fund Returns Tracker
    • Pension Risk Transfer Database
  • Careers
  • Events
    • View All Conferences
    • View All Webinars
    • 2023 Canadian Pension Risk Strategies
    • 2023 Retirement Income
Breadcrumb
  1. Home
  2. INDUSTRY VOICES
April 19, 2018 01:00 AM

Commentary: Big data analysis is coming to a pension fund near you

Irene Aldridge
  • Tweet
  • Share
  • Share
  • Email
  • More
    Reprints Print

    In the next five years, big data analysis is poised to become one of the most important and competitive skill sets around. Portfolio analysis in particular is where pension funds are focusing their big data investments.

    Big data is a set of techniques embedded in the latest, most sophisticated technologies: social media analytics, digital video recognition, 5G cellular technology and much more. The capabilities of big data are incredibly powerful and extend far beyond traditional systems. Supported as a spying technology in the World War II and later, the Cold War, core big data techniques were developed in the 1940s, 1960s and 1980s and are well tested and established.

    At the heart of big data is the concept of data as a structure that obeys well-defined principles. Regardless of how random, disjointed or even empty your data pool might be — a lot of big data addresses issues with missing values — big data techniques help distill the important drivers and create meaningful inferences.

    Many portfolio managers swim in data. And the more data, the merrier, think modern financial industry stalwarts such as Goldman Sachs Group Inc. Indeed, Goldman recently created a central data "lake" — a repository of all sorts of data accessible to all channels of the organization. Synthesizing data and making inferences quickly and efficiently is what the big data techniques do best. In other words, separating the wheat from the chaff in data with big data analytics takes only seconds, not weeks, as traditionally has been the case.

    As explained in our latest research paper, "Big Data in Portfolio Allocation," the key to successful data processing lies in the concept of eigenvalues, first invented in the 18th century as a tool to help solve differential equations. Eigenvalues, also known as characteristic values, help explain and reduce any data set, no matter how disjointed, random or even empty. Furthermore, the past 50 years have seen an explosion of interest and analysis of eigenvalue behavior. In particular, eigenvalues have been shown to obey strict distributional properties, including how close to each other they may or may not occur.

    Eigenvalue analysis is performed using two, now-common big data techniques: principal component analysis or its cousin, singular value decomposition. Both techniques allow users to identify and label critical and less relevant factors in data so that researchers can replace original data with smaller, more informative, yet equivalent data sets.

    Eigenvalue analysis is, therefore, particularly important to institutional portfolio managers dealing with a large number of financial instruments and an explosive number of exchange-traded funds. The sheer number of financial instruments available today can produce unwieldy inferences in traditional modern portfolio theory management frameworks. In addition, multiple studies have shown that the vanilla MPT framework applied to a large number of instruments creates two major problems for optimal portfolio allocation: unreasonably extreme positions and possibly extreme reallocations, resulting in poor diversification, and high transaction costs in practice. Many portfolio managers resort to "art" — experience and intuition — and strict threshold policies to constrain extreme positions and portfolio composition changes, but that tends to defeat the purpose of mathematical approaches. Big data eigenvalue analysis, on the other hand, allows researchers to reoptimize the MPT optimization framework with scientific precision.

    MPT reallocation involves computation of the correlation matrix of the assets, then inverting the correlation matrix to generate optimal portfolio weights. Several big data studies proposed breaking down the correlation matrix and then picking up the key components in an effort to reduce the matrix' complexity. The straight-on decomposition of the correlation matrix, however, has met with muted results in part because it failed to produce significant portfolio gains.

    Our research paper is the first to study the big data properties of the inverse of the correlation matrix. The paper shows the inverse is much more informative than the correlation matrix itself, from a big data perspective. Subsequently, the paper proposes big data approaches to harness the correlation inverse and to deliver superior out-of-sample returns.

    Our research uses daily closing prices to construct monthly correlation matrices for end-of-month portfolio reallocation. By decomposing the inverse of the correlation matrix using a brand-new technique (even for the big data universe), we show that the performance of the Standard & Poor's 500 portfolio alone can be improved by as much as 400% over the 20 years from 1998 to 2017.

    And this is just one of the applications showcasing the big power of big data. Armed with eigenvalues and other big data techniques, researchers can identify key trends, drivers and even missing or latent variables quickly, seamlessly and with great precision. Huge amounts of data become trivial and information of all sorts is more valuable than ever. Most importantly, researchers, analysts and portfolio managers can save countless hours and ramp up productivity by letting technology do the grunt work. Then, researchers and portfolio managers can focus their efforts on creating inferences from smaller, much more powerful datasets condensed by big data.

    Irene Aldridge is president and head of research at AbleMarkets.com, New York, and is co-author of "Real-Time Risk: What Investors Should Know About Fintech, High-Frequency Trading and Flash Crashes." This content represents the views of the author. It was submitted and edited under P&I guidelines but is not a product of P&I's editorial team.

    Related Articles
    Capabilities key to firms' future success
    Race is on to grab most possible from machine learning
    Broad adoption will mean better beta for all investors
    Commentary: What investors can learn from watching the World Cup final
    Managers should use strong '17 to reinvest in firms: BCG
    Treasury recommends a lighter regulatory touch for fintech companies
    Recommended for You
    Kathryn Kaminski
    With inverted yield curve, shorting bonds can be useful, strategic tool
    Liang Yin
    Private equity can help fill the gap in financing climate-change fixes
    Tamara Close
    Commentary: Is data the problem or the answer for ESG investors?
    OCIO: Steady Hand at the Wheel
    Sponsored Content: OCIO: Steady Hand at the Wheel

    Reader Poll

    May 1, 2023
     
    SEE MORE POLLS >
    Sponsored
    White Papers
    Middle market credit: We’re gonna need a bigger boat
    Alternative Credit: Differences and Opportunities in CLOs and Credit Risk Shari…
    Fixed Income is Attractive, but Beware of "Fake" Yield
    Counting on a Crisis: A Catalyst for Investment Innovation?
    A Strategic Allocator's Guide to Productivity and Profits
    Biodiversity: why investors should care
    View More
    Sponsored Content
    Partner Content
    The Industrialization of ESG Investment
    For institutional investors, ETFs can make meeting liquidity needs easier
    Gold: the most effective commodity investment
    2021 Investment Outlook | Investing Beyond the Pandemic: A Reset for Portfolios
    Ten ways retirement plan professionals add value to plan sponsors
    Gold: an efficient hedge
    View More
    E-MAIL NEWSLETTERS

    Sign up and get the best of News delivered straight to your email inbox, free of charge. Choose your news – we will deliver.

    Subscribe Today
    December 12, 2022 page one

    Get access to the news, research and analysis of events affecting the retirement and institutional money management businesses from a worldwide network of reporters and editors.

    Subscribe
    Connect With Us
    • RSS
    • Twitter
    • Facebook
    • LinkedIn

    Our Mission

    To consistently deliver news, research and analysis to the executives who manage the flow of funds in the institutional investment market.

    About Us

    Main Office
    685 Third Avenue
    Tenth Floor
    New York, NY 10017-4036

    Chicago Office
    130 E. Randolph St.
    Suite 3200
    Chicago, IL 60601

    Contact Us

    Careers at Crain

    About Pensions & Investments

     

    Advertising
    • Media Kit
    • P&I Content Solutions
    • P&I Careers | Post a Job
    • Reprints & Permissions
    Resources
    • Subscribe
    • Newsletters
    • FAQ
    • P&I Research Center
    • Site map
    • Staff Directory
    Legal
    • Privacy Policy
    • Terms and Conditions
    • Privacy Request
    Pensions & Investments
    Copyright © 1996-2023. Crain Communications, Inc. All Rights Reserved.
    • Topics
      • Alternatives
      • Consultants
      • Courts
      • Defined Contribution
      • ESG
      • ETFs
      • Face to Face
      • Hedge Funds
      • Industry Voices
      • Investing
      • Money Management
      • Partner Content
      • Pension Funds
      • Private Equity
      • Real Estate
      • Regulation
      • SECURE 2.0
      • Special Reports
      • Washington
      • White Papers
    • Rankings & Awards
      • 1,000 Largest Retirement Plans
      • Top-Performing Managers
      • Largest Money Managers
      • DC Money Managers
      • DC Record Keepers
      • Largest Hedge Fund Managers
      • World's Largest Retirement Funds
      • Best Places to Work in Money Management
      • Excellence & Innovation Awards
      • WPS Innovation Awards
      • Eddy Awards
    • ETFs
      • Latest ETF News
      • Fund Screener
      • Education Center
      • Equities
      • Fixed Income
      • Commodities
      • Actively Managed
      • Alternatives
      • ESG Rated
    • ESG
      • Latest ESG News
      • The Institutional Investor’s Guide to ESG Investing
      • ESG Sustainability - Gaining Momentum
      • ESG Investing | Industry Brief
      • Innovation in ESG Investing
      • 2023 ESG Investing Conference
      • ESG Rated ETFs
    • Defined Contribution
      • Latest DC News
      • The Plan Sponsor's Guide to Retirement Income
      • DC Money Manager Rankings
      • DC Record Keeper Rankings
      • Innovations in DC
      • Trends in DC: Focus on Retirement Income
      • 2023 Defined Contribution East Conference
    • Searches & Hires
      • Latest Searches & Hires News
      • Searches & Hires Database
      • RFPs
    • Research Center
      • The P&I Research Center
      • Earnings Tracker
      • Endowment Returns Tracker
      • Corporate Pension Contribution Tracker
      • Pension Fund Returns Tracker
      • Pension Risk Transfer Database
    • Careers
    • Events
      • View All Conferences
      • View All Webinars
      • 2023 Canadian Pension Risk Strategies
      • 2023 Retirement Income