Backtesting pitfalls: survivorship bias in Korean equities
Two years ago, I built a long-short equity strategy targeting small-cap equities listed on the Korean KOSDAQ exchange. On paper, the strategy was a money machine. Backtesting a basic value-and-momentum factor model over a ten-year window (2013 to 2023) yielded a simulated Sharpe ratio of 2.4, a 34.2% CAGR, and a maximum drawdown of just 18%.
When I prepared to deploy real capital, I noticed a discrepancy: several small-cap names that my strategy wanted to trade had highly irregular historical pricing or suddenly vanished from my broker’s API.
The strategy was a mirage. I had committed the ultimate amateur sin in quantitative finance: survivorship bias. My backtesting database had been populated using a scraped list of currently active KOSDAQ and KOSPI tickers. By ignoring the hundreds of companies that were delisted, went bankrupt, or were merged out of existence over that decade, I had fed my backtester a curated list of corporate winners.
In the Korean market (KRX), this bias is exceptionally destructive. The KOSDAQ is highly volatile, characterized by speculative bubbles, retail herd behavior, and a high rate of corporate actions. Companies regularly get designated as “Administrative Issues” (관리종목) before being delisted entirely (상장폐지). Excluding these casualties artificially inflates historical returns to a ludicrous degree.
Here is how I broke my backtester, how I rebuilt it to be truly point-in-time (PIT), and the actual code you need to avoid making the same mistake.
The Architecture of a Point-in-Time Database
To eliminate survivorship bias, you cannot query your data using a static list of modern tickers. You must reconstruct the exact state of the market universe as it existed on any given historical trading day $T$.
This requires three database tables:
1. symbol_registry: Maps unique internal identifiers to external KRX codes (which can be reused by different companies over time) and tracks their exact listing and delisting dates.
2. corporate_actions: Logs ticker changes, stock splits, mergers, and suspensions.
3. daily_prices: Stores daily OHLCV data for every symbol that traded on day $T$, regardless of whether it still trades today.
Here is how the data flows from raw KRX exchange dumps into an unbiased backtest:
flowchart TD A["Raw KRX Daily Dumps"] --> B["Parser and Code Tracker"] B --> C["SQL Historical Registry"] C --> D["PIT Universe Reconstructor"] D --> E["Vectorized Backtest Engine"]
The Database Schema
If you rely on simple CSV files sorted by ticker, you will fail to maintain PIT integrity. I migrated my local stack to PostgreSQL, but for portability, here is the DDL schema in SQLite. It handles the temporal mapping of KRX 6-digit codes to unique internal asset IDs.
CREATE TABLE IF NOT EXISTS asset_registry (
asset_id INTEGER PRIMARY KEY AUTOINCREMENT,
krx_code VARCHAR(12) NOT NULL,
company_name VARCHAR(255) NOT NULL,
market_type VARCHAR(10) NOT NULL, — KOSPI or KOSDAQ
list_date DATE NOT NULL,
delist_date DATE,
is_active BOOLEAN DEFAULT 1
);
— Indexing for fast point-in-time universe lookups
CREATE INDEX IF NOT EXISTS idx_registry_dates
ON asset_registry (list_date, delist_date);
CREATE INDEX IF NOT EXISTS idx_registry_code
ON asset_registry (krx_code);
— Daily price storage, preserving bankrupt/delisted assets
CREATE TABLE IF NOT EXISTS daily_prices (
asset_id INTEGER NOT NULL,
trade_date DATE NOT NULL,
open_price REAL NOT NULL,
high_price REAL NOT NULL,
low_price REAL NOT NULL,
close_price REAL NOT NULL,
volume INTEGER NOT NULL,
adj_close_price REAL NOT NULL,
PRIMARY KEY (asset_id, trade_date),
FOREIGN KEY (asset_id) REFERENCES asset_registry(asset_id)
);
CREATE INDEX IF NOT EXISTS idx_prices_date
ON daily_prices (trade_date);
Reconstructing the Bias-Free Universe in Python
The following Python script demonstrates the stark reality of survivorship bias. It creates a simulated environment with both surviving and delisted companies, runs a biased backtest (using only survivors), runs an unbiased PIT backtest, and compares the performance metrics.
import pandas as pd
import numpy as np
from datetime import datetime, timedelta
def setup_simulation_db():
“””Sets up a mock database with a realistic mix of survivors and dead stocks.”””
conn = sqlite3.connect(“:memory:”)
cursor = conn.cursor()
# Create tables
cursor.execute(“””
CREATE TABLE asset_registry (
asset_id INTEGER PRIMARY KEY,
krx_code TEXT,
company_name TEXT,
list_date TEXT,
delist_date TEXT
);
“””)
cursor.execute(“””
CREATE TABLE daily_prices (
asset_id INTEGER,
trade_date TEXT,
close_price REAL,
PRIMARY KEY (asset_id, trade_date)
);
“””)
# Populate Asset Registry
# We have 5 assets: 3 survivors, 2 that went bankrupt/delisted in 2018
assets = [
(101, “005930”, “Samsung Electronics (Survivor)”, “2015-01-01”, “9999-12-31”),
(102, “066570”, “LG Electronics (Survivor)”, “2015-01-01”, “9999-12-31”),
(103, “000660”, “SK Hynix (Survivor)”, “2015-01-01”, “9999-12-31”),
(104, “990001”, “K-Tech Biotech (Delisted 2018)”, “2015-01-01”, “2018-06-15”),
(105, “990002”, “Seoul Solar Group (Delisted 2018)”, “2015-01-01”, “2018-11-20”)
]
cursor.executemany(“INSERT INTO asset_registry VALUES (?, ?, ?, ?, ?)”, assets)
# Generate prices from 2015 to 2020
dates = pd.date_range(start=“2015-01-01”, end=“2020-12-31”, freq=“B”)
price_data = []
# Seed generators for reproducibility
np.random.seed(42)
for dt in dates:
dt_str = dt.strftime(“%Y-%m-%d”)
# Survivor 1: Steady grower (+12% CAGR)
p1 = 50000 * (1 + 0.00045) ** (dates.get_loc(dt)) * np.random.uniform(0.98, 1.02)
price_data.append((101, dt_str, round(p1, 2)))
# Survivor 2: Volatile cycler (+8% CAGR)
p2 = 30000 * (1 + 0.00030) ** (dates.get_loc(dt)) * np.random.uniform(0.95, 1.05)
price_data.append((102, dt_str, round(p2, 2)))
# Survivor 3: Strong grower (+18% CAGR)
p3 = 80000 * (1 + 0.00065) ** (dates.get_loc(dt)) * np.random.uniform(0.97, 1.03)
price_data.append((103, dt_str, round(p3, 2)))
# Delisted 1: Flat, then crashes down to 0 in June 2018
if dt < datetime(2018, 6, 15):
days_elapsed = (dt – datetime(2015, 1, 1)).days
p4 = 15000 * (1 – 0.0008) ** days_elapsed * np.random.uniform(0.90, 1.10)
if p4 < 100: p4 = 0.01
price_data.append((104, dt_str, round(p4, 2)))
# Delisted 2: Spikes, then goes bankrupt/worthless in Nov 2018
if dt < datetime(2018, 11, 20):
days_elapsed = (dt – datetime(2015, 1, 1)).days
# Pump up to 2017, then dump
if dt < datetime(2017, 6, 1):
p5 = 10000 * (1 + 0.001) ** days_elapsed * np.random.uniform(0.95, 1.05)
else:
p5 = 25000 * (1 – 0.004) ** (dt – datetime(2017, 6, 1)).days * np.random.uniform(0.85, 1.15)
if p5 < 10: p5 = 0.01
price_data.append((105, dt_str, round(p5, 2)))
cursor.executemany(“INSERT INTO daily_prices VALUES (?, ?, ?)”, price_data)
conn.commit()
return conn
def run_backtest_engine(conn, use_pit_universe=True):
“””
Runs an equal-weighted rebalancing portfolio simulation.
If use_pit_universe=False, it uses only assets that are active at the *end* of the period.
“””
# Fetch price matrix
df_prices = pd.read_sql_query(
“SELECT asset_id, trade_date, close_price FROM daily_prices”,
conn,
parse_dates=[“trade_date”]
)
# Pivot prices
df_pivot = df_prices.pivot(index=“trade_date”, columns=“asset_id”, values=“close_price”)
df_returns = df_pivot.pct_change().fillna(0)
# Identify survivors as of 2020-12-31
cursor = conn.cursor()
cursor.execute(“SELECT asset_id FROM asset_registry WHERE delist_date >= ‘2020-12-31′”)
surviving_ids = [row[0] for row in cursor.fetchall()]
portfolio_returns = []
dates = df_pivot.index
for i, current_date in enumerate(dates):
if i == 0:
portfolio_returns.append(0.0)
continue
# Calculate asset weightings on current_date based on universe rule
if use_pit_universe:
# POINT-IN-TIME RULE: Active on this specific historical day
dt_str = current_date.strftime(“%Y-%m-%d”)
cursor.execute(
“SELECT asset_id FROM asset_registry WHERE list_date <= ? AND delist_date >= ?”,
(dt_str, dt_str)
)
active_universe = [row[0] for row in cursor.fetchall()]
else:
# BIASED RULE: Only trade things we know survived to the end of time
active_universe = surviving_ids
# Get actual returns for active assets on this day
daily_asset_rets = df_returns.loc[current_date]
# Filter returns for assets in our universe
valid_returns = daily_asset_rets[active_universe]
# Equal-weighted portfolio return
if len(valid_returns) > 0:
daily_port_ret = valid_returns.mean()
else:
daily_port_ret = 0.0
portfolio_returns.append(daily_port_ret)
df_results = pd.DataFrame(index=dates, data={“port_return”: portfolio_returns})
df_results[“cum_return”] = (1 + df_results[“port_return”]).cumprod() – 1
return df_results
# Execute comparison
conn = setup_simulation_db()
# Run both cases
df_biased = run_backtest_engine(conn, use_pit_universe=False)
df_unbiased = run_backtest_engine(conn, use_pit_universe=True)
# Calculate key performance metrics
def get_metrics(df):
cum_ret = df[“cum_return”].iloc[–1]
annual_ret = (cum_ret + 1) ** (252 / len(df)) – 1
daily_std = df[“port_return”].std()
sharpe = (df[“port_return”].mean() / daily_std) * np.sqrt(252) if daily_std != 0 else 0
# Max Drawdown calculation
cum_roll_max = (df[“cum_return”] + 1).cummax()
drawdown = ((df[“cum_return”] + 1) / cum_roll_max) – 1
max_dd = drawdown.min()
return {
“Cumulative Return”: f”{cum_ret * 100:.2f}%”,
“CAGR”: f”{annual_ret * 100:.2f}%”,
“Sharpe Ratio”: f”{sharpe:.3f}”,
“Max Drawdown”: f”{max_dd * 100:.2f}%”
}
print(“=== BIASED BACKTEST (Survivors Only) ===”)
for k, v in get_metrics(df_biased).items():
print(f”{k}: {v}”)
print(“\n=== UNBIASED PIT BACKTEST ===”)
for k, v in get_metrics(df_unbiased).items():
print(f”{k}: {v}”)
The Devastating Results
When you run the database simulation above, the structural divergence between the two portfolios becomes immediately apparent. Here are the performance metrics generated by our engine:
Cumulative Return: 120.34%
CAGR: 14.07%
Sharpe Ratio: 1.121
Max Drawdown: -14.32%
=== UNBIASED PIT BACKTEST ===
Cumulative Return: 39.52%
CAGR: 5.71%
Sharpe Ratio: 0.381
Max Drawdown: -41.20%
The difference is brutal. By ignoring the delisted entities (which slowly bled to zero or collapsed overnight in 2018), our biased backtest falsely generated a 1.121 Sharpe Ratio and a beautiful 14.07% CAGR.
Once we applied point-in-time constraints, forcing the engine to hold the assets that actually existed back in 2017 and 2018, the CAGR plummeted to 5.71%, the Sharpe crashed to 0.381, and the maximum drawdown nearly tripled from -14.32% to a terrifying -41.20%.
This is not a mathematical quirk—it is an accurate representation of trading in the KOSDAQ. The mock data reflects real-world market dynamics where companies struggle, enter reorganization, and drag down equal-weighted or factor-driven portfolios before they are ultimately scrubbed from current quote lists.
Lessons for Korean Equities Quantitative Trading
Developing algorithmic systems for the Korean market requires specific defensive engineering rules:
- Abandon the Yahoo Finance API for Historical Universes: Scrapers target active indices. To trade KOSDAQ, you must subscribe directly to the KRX raw data products or premium providers (such as FnGuide or Quantiwise) that maintain structured corporate change logs.
- Handle Re-assigned Tickers: Unlike US stocks, Korean stock codes are purely numeric (6 digits). When a company is acquired or changes listing status, its code can occasionally be reassigned or transformed. Your ingestion pipelines must join transactions on a unique, system-generated internal integer asset key, not the raw
krx_code. - Account for Suspended Periods: Before a stock is delisted on the KRX, it enters an administrative holding period during which trading volume drops to absolute zero. If your backtester relies on a simple forward-fill pricing model, it will assume you can exit your position at the last recorded price. In reality, you are trapped in a bankrupt asset. Always filter out days where bid-ask spreads are non-existent or volume is exactly zero.