Download all historical US stock data

"All" here means every US stock, ETF and futures contract in the archive, delisted names included, from January 2000 (futures from December 2007) through 2026-09-16, at 1-minute, 5-minute, 30-minute, 1-hour and daily resolution. This page is the practical version: how big it is, the command that fetches it, where to put it, and how to check it landed.

How big is everything?

One Parquet file per trading day and timeframe, snappy-compressed. Current sizes:

Daily history for every US stock is about a gigabyte; the 1-minute stock set is the bulk of the archive. Most people start with daily plus one intraday timeframe and add the rest later, since the files never change shape.

What is free and what is not

A free account can download roughly the last 365 days of daily files for stocks, ETFs and futures, all symbols included. The Complete Archive ($79 one-time) unlocks every timeframe and the full history; Keep Current adds the nightly files after your purchase date. See pricing. The download mechanics below are identical for both; only the date range the API allows differs.

The download: one loop per dataset

Create an API key on the account page. The manifest endpoint returns up to 400 presigned URLs per call, so loop by year and let xargs fetch eight files at a time:

  export KEY=mp_live_...
  mkdir -p by_date/stock_daily && cd by_date/stock_daily
  for y in $(seq 2000 2026); do
    curl -s -H "Authorization: Bearer $KEY" \
      "https://marketparquet.com/api/v1/manifest/stock_daily?start=$y-01-01&end=$y-12-31" \
    | jq -r '(.files // [])[] | "\(.download_url) -o \(.filename)"' \
    | xargs -P8 -n3 curl -s --retry 3
  done

Change stock_daily to any dataset name from the table above and re-run. The presigned URLs expire after an hour, which is why the loop fetches each year's manifest right before downloading it. Rate limits are 600 requests a minute per IP with a 429 plus Retry-After when exceeded; the loop above stays well under that.

Local layout

Keep the archive's own layout: by_date/{dataset}/{year}/YYYY-MM-DD.parquet. Tools that read globs (DuckDB, Polars, pyarrow datasets) then prune by path, and a re-run of the loop into the same folders is idempotent. The data lake guide shows the full layout and a Python downloader that sorts files into year folders as they arrive.

Verify the download

Count files and read every footer once; a truncated file fails to open:

  import duckdb

  print(duckdb.sql("""
      SELECT count(*) AS files, min(filename) AS first, max(filename) AS last
      FROM read_parquet('by_date/stock_daily/*/*.parquet', filename=true, union_by_name=true)
      LIMIT 1
  """))

  # rows per year, a quick sanity check against the browse page
  print(duckdb.sql("""
      SELECT year(COALESCE(date, CAST(timestamp AS DATE))) AS y, count(*) AS rows
      FROM read_parquet('by_date/stock_daily/*/*.parquet', union_by_name=true)
      GROUP BY 1 ORDER BY 1
  """))

union_by_name=true matters for daily files: those dated before 2026-03-27 carry a timestamp column, newer ones carry date. Intraday files share one schema throughout.

Keeping it current

New files land the same night after each session, typically by midnight ET. With Keep Current, re-run the current year's loop nightly or weekly; files already on disk are skipped by name. The changelog lists the days on which historical files were rewritten (split adjustments, delisting metadata), so you know when to re-fetch a range.

Related

free historical stock data · build a local data lake · API documentation · stock data hub · symbol pages