Alternative Data: Emerging Uses in the New Economy

The "new" economy can be defined as the increasingly digitized economic landscape that emerged following COVID-19 restrictions that compelled businesses and individuals to conduct employment, personal, and social transactions online.

Those transactions have created an abundance of data, and that data is fueling the new economy by enabling governments and enterprises to formulate policy, track consumers, and develop future-forward strategies. Termed "alternative data" in economics, this valuable information includes sources outside traditional government and corporate publications. 

Investment firms were among the first enterprises to use alternative data, leading to the growth of agencies and platforms offering real-time information to guide important decisions. These initial adopters were quickly joined by governments and other enterprises looking to leverage the power of big data analytics to develop business strategies and formulate critical government policy.

A Quick Primer on Alternative Data

Alternative data is information used by professionals in the financial, economic, and investment sectors outside the scope of "official" data released by governments and corporations. 

A quick list of sources includes:

• Sensors used in advanced supply chain systems
• Internet of Things (IoT) sensors
• Mobile device applications
• App store analytics
• Satellite imagery
• Public Records
• Geolocation data
• Website usage statistics 
• Shipping trackers
• Price trackers

Mainstream Applications of Alternative Data

Applications of alternative data for business continue to grow as the economy digitizes. Common applications currently include: 

Real Estate

Data extraction from real estate listings, geolocation, and satellite data can provide insights into key factors affecting the property market. Some valuable metrics offered by these forms of alternative data include current supply, rental availability, home prices, local building permits, and energy consumption patterns. 

E-Commerce

E-commerce websites, social media networks, digital marketplaces, search engines, and customer review sites provide critical sales and product information. Some alternative data metrics vital to this sector include prices, product stock levels, customer comments and reviews, SEO trends, brand mentions, and social media sentiment.

Investment Data

Finance managers can now look beyond earnings reports to alternative data to gauge a company's health and investment potential. Some unique sources include mergers and acquisitions intelligence, job postings, credit inquiries, social media posts, and forums.

Commodities 

Alternative data provides a significant competitive advantage for tracking commodity prices. Some sources include satellite images of oil tankers, agricultural activity, mining projects, and supply chain sensors. 

New Uses of Alternative Data

The widespread adoption of GPS, sensors and tracking devices is leading to next-generation uses of alternative data that include:

• Mobility data: using personal location information to assess economic conditions and guide monetary policy
• Future product demand: using financial transaction data to guide production by predicting demand levels
• Inflation: extracting prices from online websites to predict future trends
• COVID-19: using mobile applications to track individuals diagnosed with positive COVID-19 tests, predicting unemployment insurance trends with data from online benefits applications
• Recessions: using flash estimates of key economic indicators to predict recessions 

Case Study: How the Bank of Japan Uses Alternative Data

Since the start of the COVID-19 crisis, the Bank of Japan (BOJ) has actively used alternative data to assess economic conditions and formulate monetary policy. 

Some specific applications include:

• The use of high-frequency data during COVID-19 to capture mobility trends and consumption by tracking cultural site visits, downtown Tokyo pedestrian and vehicle traffic, restaurant visits, credit card transactions, and airport use
• The use of textual data (words and sentences in public documents, reports, social networks, and news media) to gauge consumer sentiment
• The use of granular (low-level) detailed data to obtain insights about specific financial market areas, including the Over-The-Counter (OTC) derivatives market, bank lending rates, and other factors governing market stability

According to a recent BOJ review, alternative data use is projected to increase as economic digitization progresses. In addition, data volume is also expected to grow as firms are increasingly required to disclose information on key areas such as climate change and Environmental, Social, and Governance (ESG) policies. 

How to Obtain Alternative Data

Some primary alternative data sources include published reports, real-time data platforms, and purchased data sets. Many businesses also use web scraping to extract specific information from websites, social media networks, and other public online sources. 

How Web Scraping Works

Web scraping is a process that extracts information from the internet using scripts and HTTP requests. This is similar to regular users tapping or clicking links where the browser requests data from the server. Web scraping operates on the same principle using automatic code that scans and extracts data. The main difference is that web scraping automates the process and can process thousands of web pages per second.

Data received via web scraping is usually formatted as HTML or JSON documents. To make it easy to read, a process called parsing transforms that data into a CSV or other accessible format.

Ready-To-Use Scraping Tools

Web scraping is a complex process that poses many challenges, even for the most experienced developers. To address this issue, some businesses choose to leverage web scraping APIs that allow them to quickly scrape data while forgoing the complexities of the process. That way, they can focus on extracting insights for use in data-driven, precision decision-making.

Web Scraping Makes Alternative Data More Accessible

Alternative data uses are increasing as more parts of the economy are digitized. To gain insights into how fast its use is growing, Oxylabs conducted a survey in cooperation with Censuswide that asked over 1 000 UK and US-based senior data decision-makers how alternative data is being extracted and used at their respective firms.

The survey results confirm that alternative data use is widespread and growing, with over 57% of respondents revealing that web scraping was a primary data extraction method used at their company. Other alternative data sources included third-party providers (42%) and manual extraction techniques (41%).