yfinance limitations for backtesting

yfinance is a great exploration tool and a poor research substrate. This guide covers exactly where it breaks down once a notebook becomes a repeatable backtest -- and which limitations you can live with.

Where yfinance fits

yfinance is free, ubiquitous, and takes one line to fetch a chart's worth of data. It is also a community wrapper around Yahoo Finance's web endpoints -- not a documented market-data service. For eyeballing a handful of large-cap symbols, checking a price on a date, or teaching, it is genuinely the right tool. The problems start when its output becomes the input to a strategy you intend to trust.

Problem 1: prices that change under you

Yahoo's Adj Close folds dividends into the entire price history. That means every historical price your backtest sees changes a little after every dividend -- run the same download in January and July and the 2015 prices differ. Two consequences:

  • Backtests are not reproducible from a fresh download; results drift for reasons unrelated to your strategy.
  • Dividend-smoothed prices are not the prices that traded. Stops, limit fills, and "stock fell 5% today" logic execute at levels that never printed on the tape.

A static file archive inverts this: MarketParquet prices are split-adjusted traded levels, dividends deliberately left out of the series, and a downloaded file never changes underneath you. (Details and a worked AAPL example: methodology.)

Problem 2: survivorship bias

Yahoo serves listed companies. Tickers that were delisted, acquired, or went bankrupt mostly return nothing -- so a universe built from yfinance is a universe of survivors. The effect is not subtle: of the 21,427 stock, ETF, and futures symbols in the MarketParquet daily archive, roughly 10,000 no longer trade. The 2010-01-04 daily file contains 5,876 stock symbols; 3,516 of them -- 60% -- are gone today. A momentum or small-cap strategy backtested only on the 40% that survived will look dramatically better than it was.

Problem 3: intraday history barely exists

Yahoo's endpoints only serve 1-minute bars for roughly the last 30 calendar days, in request windows of about a week, and hourly data for about two years. There is no way to pull 1-minute data for 2015 -- or even for last quarter -- from yfinance at any price. Intraday backtesting over years of history requires an archive that was collected as it happened; MarketParquet's 1-minute files go back to 2000 for stocks and ETFs, and to the end of 2007 for futures.

Problem 4: shape and reliability of bulk access

  • Ticker-at-a-time -- yfinance returns one symbol's history per logical request. Cross-sectional research ("rank all stocks by yesterday's volume") wants the transpose: every symbol for one day. MarketParquet files are already that shape -- one file per trading day, all ~6,500+ active symbols inside.
  • Rate limits and breakage -- Yahoo throttles and reshapes its endpoints without notice; every few months a yfinance version stops working until patched. Fine for a hobby script, painful inside a nightly pipeline.
  • No terms for it -- Yahoo's data is licensed for personal viewing; redistributing or building a product on scraped Yahoo data is a compliance risk vendors and funds will not accept.

What staying reproducible looks like

The fix is boring on purpose: download typed Parquet files once, keep them on disk, and make every backtest read the same bytes. 

  import pandas as pd

  # Same file, same bytes, same backtest -- forever.
  df = pd.read_parquet("stock_daily_2024-01-15.parquet")
  day = df.set_index("symbol")

  # The whole cross-section is already here:
  # 6,500+ active symbols plus that era's now-delisted tickers
  # (suffixed '-DELISTED'), split-adjusted OHLCV, typed columns.
  liquid = df[df["volume"] > 1_000_000]

Column mapping if you are porting yfinance code: yfinance Open/High/Low/Close/Volume map directly to open/high/low/close/volume; there is no Adj Close because prices are already split-adjusted, and the index is a symbol/date pair rather than a per-ticker DatetimeIndex.

Honest comparison

                        yfinance          MarketParquet
  price                 free              $79 one-time archive
  daily history         ~listed life      2000-present incl. delisted
  1-min history         ~30 days          2000-present (stocks/ETFs)
  delisted tickers      mostly absent     ~10,000 preserved
  bulk shape            per ticker        per trading day (all symbols)
  reproducible          no (adj close     yes (static Parquet files)
                        mutates)
  rate limits           opaque, shifting  documented API, bulk download

If your work is a chart or a one-off look-up, keep using yfinance -- it is the better tool for that. The row that usually decides it for backtesters is reproducibility.

FAQ

Can I mix yfinance and MarketParquet data? For research convenience, yes -- but expect systematic price differences on dividend-paying names, since Yahoo's adjusted prices embed dividends and MarketParquet's deliberately do not.

Does yfinance data match MarketParquet on splits? Broadly yes for raw Close on active large caps; both are split-adjusted. Differences concentrate in Adj Close, thinly traded names, and anything delisted.

I only trade the S&P 500 -- do delisted tickers matter? Yes: index constituents change constantly, and the members that were removed (often after crashing) are exactly the ones a survivors-only dataset forgets. Point-in-time universes need the dead tickers too.

Related

MarketParquet vs Yahoo Finance · delisted stock data · data quality & methodology · browse stock daily · DuckDB guide · pricing