26 Years of US Market Data in Parquet

What's inside

• Stocks and ETFs from 2000; futures from December 2007 (~130 of the most active contracts, continuous ratio-adjusted)
• One-minute, five-minute, 30-minute, hourly, and daily bars
• Delisted symbols included -- survivorship-bias-free (23,000+ cumulative stocks, 9,500+ ETFs; Enron, Lehman, and every other blow-up your backtest should have seen)
• Apache Parquet, ready for pandas, Polars, and DuckDB -- no CSV parsing, no database load step
• New bars published the same night, typically by midnight ET

Adjustment policy, exactly: prices are split-adjusted; dividends are not embedded in prices. Futures are continuous, back-adjusted by ratio. Zero-volume bars are preserved, not deleted. Full details on the methodology & data quality page.

An actual file, opened

One file per trading day, every symbol in it -- a full-day cross-section in a single read. This is a real query against a real file, output included:

$ python
>>> import duckdb
>>> duckdb.sql("""
...     SELECT timestamp, open, high, low, close, volume
...     FROM 'by_date/stock_1min/2024/2024-01-03.parquet'
...     WHERE symbol = 'AAPL' ORDER BY timestamp
... """).fetchdf()

# 741 rows in ~6 ms. First bars of the premarket:
#            timestamp    open    high     low   close  volume
# 0 2024-01-03 04:00:00  185.00  185.00  185.00  185.00   791.0
# 1 2024-01-03 04:01:00  184.90  184.90  184.90  184.90  1278.0

File layout and schema:

by_date/{asset}_{timeframe}/YYYY/YYYY-MM-DD.parquet

e.g. by_date/stock_1min/2024/2024-03-15.parquet
     ~22 MB, ~1.28M rows, ~7,180 symbols

columns (intraday):
  timestamp    TIMESTAMP[us]  -- bar OPEN time, US/Eastern, tz-naive
  symbol       VARCHAR        -- 'AAPL'; delisted end in '-DELISTED'
  asset_type   VARCHAR
  open/high/low/close  DOUBLE -- split-adjusted
  volume       DOUBLE

columns (daily): same, with a DATE column named 'date'

...plus 5-min, 30-min, and hourly for every asset class -- 15 datasets, 95,834 files total.

one-time purchase, no subscription · free account for delivery (~30s) · no credit card for the sample · secure checkout via Stripe

Five lines to a backtest-ready DataFrame

pandas, one day:

import pandas as pd

bars = pd.read_parquet("2024-03-15.parquet")
aapl = bars[bars.symbol == "AAPL"].set_index("timestamp")
print(aapl[["open", "high", "low", "close", "volume"]].head())

DuckDB, a whole year of files in one glob:

import duckdb

df = duckdb.sql("""
    SELECT symbol, date, close, volume
    FROM 'by_date/stock_daily/2024/*.parquet'
    WHERE symbol = 'AAPL' ORDER BY date
""").df()

Worked examples with real outputs: DuckDB guide · Polars guide · build a local data lake

Why not just use free data?

Free sources give you today's survivors: tickers that went to zero simply vanish, which quietly inflates every backtest. They also rewrite history silently, throttle unpredictably, and rarely offer intraday depth. This archive keeps the losers (suffixed -DELISTED), states its adjustment policy precisely, and ships as files you own and version yourself. The longer write-up: what breaks when you backtest on yfinance.

Free sample vs. archive vs. Keep Current

The archive is a one-time purchase -- perpetual access to everything dated up to it. Keep Current is optional and cancelable anytime. Full details on pricing.

one-time purchase, no subscription · free account for delivery (~30s) · no credit card for the sample · secure checkout via Stripe

How the data is built

Sourced from a commercial vendor, validated and normalized by an automated pipeline: schema and timezone checks, split-adjustment verification, cross-timeframe consistency checks, and a checkpointed merge ledger so partial updates can never corrupt published files. Delisted-ticker history is merged weekly. The complete methodology -- including known limitations, honestly stated -- is on the data quality page.

Not included (so there's no surprise after purchase): real-time or streaming data, tick data, options, fundamentals, and corporate-action event files. This is historical OHLCV, done carefully.

Common questions

Before you buy: data license · refund policy · terms · privacy · more in the full FAQ

one-time purchase, no subscription · free account for delivery (~30s) · no credit card for the sample · secure checkout via Stripe