\[ % MathJax has no bm package; redefine \bm in terms of \boldsymbol, which it supports natively \newcommand{\bm}[1]{\boldsymbol{#1}} % --- Operators ----------------------------------------------------------- % Expectation operator \DeclareMathOperator{\E}{E} % Variance operator \DeclareMathOperator{\Var}{Var} % Covariance operator \DeclareMathOperator{\Cov}{Cov} % Correlation operator \DeclareMathOperator{\Corr}{Corr} % Rank operator (Spearman rank correlation) \DeclareMathOperator{\Rank}{Rank} % Skewness operator \DeclareMathOperator{\Skewness}{Skewness} % Kurtosis operator \DeclareMathOperator{\Kurtosis}{Kurtosis} % Difference/differential operator (upright d per ISO 80000-2) \newcommand{\Diff}{\mathrm{d}} % --- Risk Measures ------------------------------------------------------- % Value at Risk \DeclareMathOperator{\VaR}{VaR} % Expected Shortfall \DeclareMathOperator{\ES}{ES} % Marginal Value at Risk \DeclareMathOperator{\MVaR}{MVaR} % Component Value at Risk \DeclareMathOperator{\CompVaR}{CVaR} % Incremental Value at Risk \DeclareMathOperator{\IVaR}{IVaR} % Component Expected Shortfall \DeclareMathOperator{\CompES}{CES} % Weighted sensitivity for risk factor in FRTB SBM bucket \DeclareMathOperator{\WS}{WS} % Hedge-benefit ratio for FRTB DRC bucket \DeclareMathOperator{\HBR}{HBR} % Stress scenario risk measure for FRTB NMRF capital add-on \DeclareMathOperator{\SES}{SES} % Jump-to-default exposure for obligor (FRTB DRC) \DeclareMathOperator{\JTD}{JTD} % Loss given default for obligor (FRTB DRC) \DeclareMathOperator{\LGD}{LGD} % Default risk weight for obligor (FRTB DRC) \DeclareMathOperator{\RW}{RW} % ES ratio function \newcommand{\ESratio}{\lambda} % Arbitrary risk measure function (coherence axioms) \newcommand{\RiskMeasure}{\varphi} % Asset position (coherence axioms) \newcommand{\Asset}{A} % First named asset in examples \newcommand{\AssetA}{A} % Second named asset in examples \newcommand{\AssetB}{B} % Third named asset in examples \newcommand{\AssetC}{C} % Constant (risk measure axioms) \newcommand{\Constant}{c} % Bucket-level aggregate sensitivity in FRTB SBM \newcommand{\BucketAgg}{S} % Cross-bucket correlation in FRTB SBM \newcommand{\CrossBucketCorr}{\gamma} % Notional amount for an instrument (FRTB DRC and RRAO) \newcommand{\Notional}{\text{Notional}} % --- Distributions ------------------------------------------------------- % Binomial distribution \DeclareMathOperator{\Binomial}{Binomial} % Uniform distribution \DeclareMathOperator{\Uniform}{Uniform} % Normal distribution \newcommand{\NormalDist}{\mathcal{N}} % Student-t CDF \newcommand{\StudentCDF}{t} % Student-t PDF \newcommand{\StudentPDF}{f} % --- Returns ------------------------------------------------------------- % Price \newcommand{\Price}{P} % Simple (arithmetic) return \newcommand{\SimpleReturns}{R} % Log (compound) return \newcommand{\CompoundReturns}{y} % Matrix of compound returns \newcommand{\ReturnMatrix}{\bm{y}} % Dividend payment \newcommand{\dividend}{d} % --- Volatility ---------------------------------------------------------- % Realised variance \DeclareMathOperator{\RealisedVar}{RV} % Realised volatility (square root of realised variance) \DeclareMathOperator{\RealisedVol}{RVol} % Volatility \newcommand{\Vol}{\sigma} % GARCH intercept parameter \newcommand{\GARCHconst}{\omega} % ARCH coefficient (weight on past squared returns) \newcommand{\ARCHcoeff}{\alpha} % GARCH coefficient (weight on past variance) \newcommand{\GARCHcoeff}{\beta} % EWMA decay factor (typically 0.94) \newcommand{\EWMAdecay}{\lambda} % Standardized residual / error term \newcommand{\StdNormal}{\epsilon} % Leverage parameter in apARCH \newcommand{\APARCHleverage}{\zeta} % Power parameter in apARCH \newcommand{\APARCHpower}{\delta} % Degrees of freedom (Student-t) \newcommand{\DOF}{\nu} % GJR-GARCH leverage parameter \newcommand{\GJRleverage}{\gamma} % GARCH-X external regressor coefficient \newcommand{\RegressorCoeff}{\xi} % Mean (first moment) \newcommand{\Mean}{\mu} % Lag order in volatility models \newcommand{\Lag}{L} % Dummy variable (indicator in regressions) \newcommand{\DummyVar}{D} % --- Portfolio ----------------------------------------------------------- % Portfolio weight (scalar) \newcommand{\weight}{w} % Portfolio weight vector \newcommand{\weights}{\bm{w}} % Covariance matrix \newcommand{\CovMatrix}{\bm{\Sigma}} % Correlation matrix \newcommand{\CorrMatrix}{\bm{C}} % Correlation coefficient \newcommand{\correlation}{\rho} % Number of assets in portfolio \newcommand{\NumberAssets}{K} % Portfolio value \newcommand{\PortfolioValue}{\vartheta} % Asset beta (CAPM) \newcommand{\AssetBeta}{\beta} % Ledoit-Wolf shrinkage intensity \newcommand{\ShrinkageIntensity}{\delta} % Sample covariance matrix \newcommand{\SampleCov}{\bm{S}} % --- Time Series --------------------------------------------------------- % Sample size \newcommand{\SampleSize}{T} % Generic count (number of lags, payments, etc.) \newcommand{\Count}{N} % Estimation window length \newcommand{\EstWindow}{W_E} % Testing window length \newcommand{\TestWindow}{W_T} % Stress window length \newcommand{\StressWindow}{W_S} % AR coefficient \newcommand{\ARcoeff}{\phi} % MA coefficient \newcommand{\MAcoeff}{\psi} % --- Probability --------------------------------------------------------- % Probability level (e.g. 0.01 for 1% VaR) \newcommand{\probability}{p} % Cumulative sorted scenario weight \newcommand{\CumScenarioWeight}{\Omega} % p-quantile of the P and L distribution \newcommand{\Quantile}{q} % Quantile function \newcommand{\QuantileFunction}{Q} % Probability density function \newcommand{\PDF}{f} % Cumulative distribution function \newcommand{\CDF}{F} % Standard normal CDF \newcommand{\NormalCDF}{\Phi} % Inverse standard normal (quantile function) \newcommand{\NormalQuantile}{\Phi^{-1}} % Standard normal density function \newcommand{\NormalPDF}{\phi} % Likelihood function \newcommand{\lik}{\mathcal{L}} % Log-likelihood \newcommand{\LogLikelihood}{\ell} % Parameter vector (MLE estimation) \newcommand{\ParamSet}{\theta} % Parameter space \newcommand{\ParamSpace}{\Theta} % Fisher information matrix \newcommand{\FisherInfo}{\mathcal{I}} % Statistical power (Type II error rate) \newcommand{\StatPower}{\beta} % Significance level (CI coverage complement) \newcommand{\SignifLevel}{\gamma} % --- Options ------------------------------------------------------------- % Call option label \newcommand{\CallOption}{\text{call}} % Put option label \newcommand{\PutOption}{\text{put}} % Strike price \newcommand{\Strike}{X} % Risk-free interest rate \newcommand{\RiskFree}{r_f} % Option maturity (time to expiration) \newcommand{\OptionMaturity}{\tau} % Option delta \newcommand{\OptionDelta}{\Delta} % Option gamma \newcommand{\OptionGamma}{\Gamma} % Option vega \newcommand{\Vega}{\mathcal{V}} % Holding period \newcommand{\HoldingPeriod}{H} % Simulation profit/loss (signed; negative is a loss) \newcommand{\ProfitLoss}{\Pi} % Futures price \newcommand{\Futures}{F} % Hedge ratio \newcommand{\HedgeRatio}{h} % Units of basic asset held \newcommand{\StockHolding}{x^b} % Units of options held \newcommand{\OptionHolding}{x^o} % --- Interest Rates ------------------------------------------------------ % Bond convexity \newcommand{\Convexity}{C} % Interest rate / yield \newcommand{\Yields}{r} % Key rate change, basis points (scalar tenor component); used as \KeyRateChange_k or \KeyRateChange_{t,k}. EXCEPTION to the no-macro-to-macro-indirection rule, decided 2026-08-09: composed from \Yields on purpose, so a future change to the rate glyph propagates here without a follow-up edit. \Yields is defined earlier in generated output (same interest_rates category, TOML order), so expansion order is safe; verify this still holds if either entry is reordered. \newcommand{\KeyRateChange}{\Delta \Yields^{\mathrm{bp}}} % Key rate changes, basis points (vector); used bare or as \KeyRateChanges_t. EXCEPTION to the no-macro-to-macro-indirection rule, decided 2026-08-09: composed from \Yields on purpose, so a future change to the rate glyph propagates here without a follow-up edit. \bm{\Yields} depends on \bm/\boldsymbol correctly expanding a macro argument rather than a bare letter --- confirm this renders correctly the first time either consumer compiles it. \newcommand{\KeyRateChanges}{\Delta \bm{\Yields}^{\mathrm{bp}}} % Dollar value of a basis point \newcommand{\DV}{\text{DV01}} % Mean reversion speed (interest rate models) \newcommand{\MeanReversion}{\varkappa} % Modified duration \newcommand{\ModDur}{D} % Macaulay duration \newcommand{\MacDur}{D_{\text{Mac}}} % Effective duration \newcommand{\EffDur}{D_{\text{eff}}} % DV01 vector \newcommand{\DVvec}{\bm{d}} % Long-run mean rate (Vasicek/CIR/Hull-White) \newcommand{\LongRunRate}{\theta} % Hull-White time-varying drift \newcommand{\HullWhiteDrift}{\varphi} % Cash flow (coupon/principal payment) \newcommand{\CashFlow}{c} % --- Extreme Value ------------------------------------------------------- % Tail index (EVT, Pareto-type tails) \newcommand{\TailIndex}{\iota} % Shape parameter (xi = 1/iota) \newcommand{\ShapeParam}{\varsigma} % Threshold value (EVT) \newcommand{\Threshold}{u} % Extremal index (Leadbetter 1983) \newcommand{\ExtremalIndex}{\theta} % Multivariate extremal coefficient, theta in [1,d]. theta=1 complete tail dependence; theta=d asymptotic independence. Glyph collides with ExtremalIndex and ParamSet; disambiguated at source by macro name. \newcommand{\ExtremalCoef}{\theta} % GEV distribution function \newcommand{\GEV}{\mathcal{H}} % GPD distribution function \newcommand{\GPD}{\mathcal{G}} % GPD scale parameter \newcommand{\GPDscale}{\beta} % GPD scale parameter at a threshold u, distinct from the generic GPD scale \newcommand{\GPDscaleAtThreshold}{\beta_u} % Number of threshold exceedances (EVT) \newcommand{\TailCount}{C} % Sample maximum (block maxima, EVT) \newcommand{\SampleMax}{M} % Kurtosis value (realized, e.g. sample kurtosis) \newcommand{\KurtosisVal}{\kappa} % Pareto scaling constant (EVT) \newcommand{\ParetoConst}{\mathcal{A}} % Generic constant (EVT asymptotics) \newcommand{\RemainderConst}{\mathcal{C}} % Little-o asymptotic notation \newcommand{\LittleO}{o} % Moment order (EVT) \newcommand{\MomentOrder}{\mathscr{m}} % Doubled sample size (EVT block maxima) \newcommand{\DoubleSample}{D} % GEV normalizing location constant \newcommand{\GEVloc}{a} % GEV normalizing scale constant \newcommand{\GEVscale}{b} % Negative return (loss, EVT context) \newcommand{\NegativeReturn}{L} % --- Copulas ------------------------------------------------------------- % Copula function \newcommand{\Copula}{\mathcal{C}} % Copula density \newcommand{\CopulaDensity}{c} % Generator function (Archimedean copulas) \newcommand{\generator}{\varphi} % Copula dependence parameter \newcommand{\CopulaParam}{\theta} % Copula marginal parameter set \newcommand{\MarginalParamSet}{\eta} % Lower tail dependence coefficient \newcommand{\LowerTailDep}{\lambda_L} % Upper tail dependence coefficient \newcommand{\UpperTailDep}{\lambda_U} % Marginal CDF (second variable) \newcommand{\MarginalCDF}{G} % Joint density function \newcommand{\JointDensity}{h} % Marginal density function \newcommand{\MarginalDensity}{g} % Joint distribution function \newcommand{\JointCDF}{H} % Gaussian copula correlation parameter \newcommand{\CopulaCorr}{\rho} % --- Multivariate -------------------------------------------------------- % Diagonal matrix of conditional volatilities \newcommand{\DiagVolD}{\bm{D}} % Idiosyncratic variance matrix (diagonal, factor models) \newcommand{\IdioVarMatrix}{\bm{\Psi}} % Covariance matrix entry (two indices) \newcommand{\CovElement}{\sigma} % Auxiliary matrix in DCC dynamics \newcommand{\DCCauxQ}{\bm{Q}} % BEKK constant matrix \newcommand{\BEKKconst}{\bm{\Omega}} % BEKK ARCH parameter matrix \newcommand{\BEKKarch}{\bm{A}} % BEKK GARCH parameter matrix \newcommand{\BEKKgarch}{\bm{B}} % DCC rescaling diagonal matrix \newcommand{\DCCrescaleZ}{\bm{Z}} % DCC auxiliary matrix element \newcommand{\DCCelement}{q} % Factor loadings matrix (loadings/eigenvectors of the covariance matrix) \newcommand{\FactorLoadings}{\bm{\Lambda}} % Factor subscript label \newcommand{\factor}{\text{factor}} % PCA eigenvalue \newcommand{\Eigenvalue}{\lambda} % Number of factors in factor model \newcommand{\NumberFactors}{m} % Factor return vector \newcommand{\FactorReturn}{\bm{g}} % PCA factor score (scalar; not an eigenvector) \newcommand{\PCAFactor}{F} % BEKK cross-covariance coefficient \newcommand{\BEKKcross}{\delta} % Factor loading element \newcommand{\FactorLoading}{\Lambda} % DCC correlation persistence parameter \newcommand{\DCCxi}{\xi} % DCC news coefficient (weight on recent shocks) \newcommand{\DCCzeta}{\zeta} % --- Simulation ---------------------------------------------------------- % Number of simulation paths \newcommand{\NumberSims}{B} % Cholesky factor of covariance matrix \newcommand{\Cholesky}{\bm{L}} % Uniform random number \newcommand{\UniformDraw}{\tilde{u}} % --- Backtesting --------------------------------------------------------- % Violation ratio (observed/expected violations) \DeclareMathOperator{\ViolRatio}{VR} % Stressed Value at Risk \DeclareMathOperator{\StressedVaR}{SVaR} % QLIKE loss function for variance forecasts \DeclareMathOperator{\QLIKE}{QLIKE} % Mean squared error (forecast loss function) \DeclareMathOperator{\MSE}{MSE} % Mean absolute error (forecast loss function) \DeclareMathOperator{\MAE}{MAE} % Markov transition probability \newcommand{\TransProb}{\pi} % Exception indicator: 1 if y_t < -VaR_t \newcommand{\ExceptionInd}{\eta} % Exception count \newcommand{\ExceptionCount}{\upsilon} % Probability Integral Transform \newcommand{\ProbIntegralTransform}{\hat{u}} % Quantile score (pinball loss function) \newcommand{\QuantileScore}{\mathcal{S}} % Kolmogorov-Smirnov test statistic \newcommand{\KSstat}{D} % Test statistic (generic base letter) \newcommand{\TestStat}{J} % --- General ------------------------------------------------------------- % Regression intercept \newcommand{\RegressionIntercept}{a} % Regression slope \newcommand{\RegressionSlope}{\beta} % Risk factor \newcommand{\RiskFactor}{x} % Forecast horizon \newcommand{\ForecastHorizon}{h} % Block length (bootstrap) \newcommand{\BlockLength}{\ell} % Capital tau variant \newcommand{\DeliveryTime}{\Upsilon} % Today's calendar time in years (option pricing input; distinct from trading-date index t) \newcommand{\CalendarTime}{t^*} % Probability measure \newcommand{\ProbMeasure}{\mathbb{P}} % Risk-neutral measure \newcommand{\RiskNeutral}{\mathbb{Q}} % Pricing function (Black-Scholes, bond pricing, etc.) \newcommand{\PricingFn}{V} % Indicator function \newcommand{\Indicator}{\bm{1}} % Vector of ones \newcommand{\OnesVector}{\bm{1}} % Risk factor sensitivity (first-order P&L) \newcommand{\Sensitivity}{\delta} % Second-order sensitivity (convexity/gamma) \newcommand{\ConvexitySens}{\gamma} % Output floor percentage (Basel) \newcommand{\OutputFloor}{\alpha} % --- Subscript Labels ---------------------------------------------------- % Subscript label: annualized \newcommand{\Annual}{a} % Subscript label: implied \newcommand{\Implied}{I} % Subscript label: portfolio \newcommand{\Portfolio}{\pi} % --- Text Abbreviations -------------------------------------------- % S&P 500 index \newcommand{\SP}{\text{S\&P-500}} % Student-t distribution (text) \newcommand{\St}{\text{Student-t}} % Geopolitical Risk index (Caldara and Iacoviello) \newcommand{\GPR}{\text{GPR}} \]

2  Financial and economic data

Financial markets generate vast amounts of data, sometimes at nanosecond intervals. Getting that data is expensive.

2.1 Data and libraries

The EODHD client is the only thing needed here, and only for the download examples later in the chapter.

library(eodhdR2)
import pandas as pd
from eodhd import APIClient
using HTTP, JSON3, DataFrames

2.2 Practical issues in market data

Financial data comes in many forms. Stocks, bonds, futures, options, commodities and foreign exchange all trade differently.

2.2.1 Names and tickers

All stocks that are trading in the market are associated with a ticker symbol, serving as an identifier for the specific security. These are often specific to the particular exchange or country of listing. This can often lead to confusion or ambiguity when a company has cross-listings. For instance, the Japanese car manufacturer Toyota is listed as 7203 on the Tokyo Stock Exchange, TYT on the London Stock Exchange and TM on the New York Stock Exchange.

Some identical securities have different names. Depending on the source, searching for data on the S&P 500 (a major American stock market index) requires the label SPX (Bloomberg), ^GSPC (Yahoo Finance), INX (Google Finance), GSPC.INDX (EODHD), and so on.

Firms change names, merge and disappear. Their tickers change too, and exchanges sometimes recycle them.

Always check the vendor’s description and confirm that the ticker matches the series you want.

2.2.2 Permanent asset identifiers — ISIN and PERMNO

Vendors solve this problem with permanent asset identifiers. These labels stay with the asset even when the name, exchange or ticker changes. There are multiple data vendors, each with its own nomenclature and usually inconsistent labelling.

The ISIN is the International Securities Identification Number, an internationally recognised 12-character code unique to each security. Unlike tickers, the ISIN for a given instrument is the same regardless of the market in which it trades, but a depositary receipt, such as Toyota’s NYSE-listed TM, is a separate instrument with its own ISIN.

The PERMNO is the permanent issue identifier of the CRSP dataset.

When downloading data, it is best not to refer to stocks by their ticker symbol but rather by one of the permanent asset identifiers.

2.2.3 Adjusted and unadjusted prices

If you download stock prices, your data will often be unusable because the number of shares outstanding has changed, usually through stock splits, while the recorded prices have not been adjusted.

For example, Amazon announced a 20-for-1 stock split in 2022. This meant that every Amazon stock became 20 stocks, and similarly, the price dropped by a factor of 20, from about $2,400 to $120.

Treating the unadjusted series as returns would record a one-day loss of about 95%, even though the split leaves the investor’s wealth unchanged. It manufactures a return, and a volatility observation, out of nothing.

Use adjusted prices. They remove stock splits and spin-offs and make the series fit for analysis. Some vendors’ adjusted close also reflects dividend reinvestment, a different convention from the split-and-spin-off adjustment used here, so check the vendor’s documentation before comparing series across sources.

Unadjusted data can also contain artificial price jumps from missed corporate actions. Signs include a sudden large price drop that is in fact an unrecorded stock split, a price series that no longer aligns with market indices over the same period, or an unrealistic volatility spike on a specific date.

2.2.4 Asynchronous prices

Problems arise when data comes from different markets and countries due to:

  1. Holidays;
  2. Time zones.

Public holidays, days when the markets are closed, are often different across countries. Examples include national holidays, such as 4 July in the United States, and religious holidays. The exchanges are usually open Monday through Friday, but the Saudi Stock Exchange is open from Sunday through Thursday. Some exchanges close for a lunch break. Some countries have summertime, and others do not, and summertime often happens on different dates in the US and Europe.

Name Time Zone Trading Hours Midday pause
New York Stock Exchange ET 9:30 a.m. to 4:00 p.m. No
Shanghai Stock Exchange China Standard Time 9:30 a.m. to 11:30 a.m., 1:00 p.m. to 3:00 p.m. 11:30 a.m. to 1:00 p.m.
Tokyo Stock Exchange JST 9:00 a.m. to 3:30 p.m. 11:30 a.m. to 12:30 p.m.
London Stock Exchange GMT/BST 8:00 a.m. to 4:30 p.m. 12:00 p.m. to 12:02 p.m.
Frankfurt Stock Exchange CET/CEST 9:00 a.m. to 5:30 p.m. No

The New York market overlaps with London, and since Tokyo is 9 hours ahead of London (8 hours during British Summer Time), there is no overlap in trading hours with Tokyo.

This means that any research comparing prices across countries at a daily frequency needs to consider these issues. They can be mitigated by using weekly or monthly data.

2.3 Data quality and validation

2.3.1 Missing values and gaps

Financial markets sometimes have trading halts, holidays or technical issues that create gaps in price series. Missing data points can distort volatility calculations and risk metrics. Always check for:

  • Consecutive missing dates
  • Unusual gaps in trading volumes
  • Zero or negative prices (often indicating data errors)

2.3.2 Outliers and data entry errors

Erroneous data points can dramatically affect risk measurements. Common issues include:

  • Prices quoted in wrong currency units (pence vs pounds)
  • Decimal point errors (prices off by factors of 10 or 100)
  • Typographical errors in manual data entry

2.3.3 Basic validation procedures

Before using data for risk calculations:

  1. Plot the price series and look for anomalies.
  2. Check for unusually large returns that may indicate data errors.
  3. Compare data across multiple sources when possible.
  4. Check that adjusted prices align with known stock splits and spin-offs.
  5. Check that zero prices do not coincide with normal trading volumes.

2.4 Data frequency considerations

Tick data captures every trade or quote update, potentially generating millions of observations per day. It is useful for high-frequency trading and market microstructure analysis. Tick data is expensive to store — a single year can run to terabytes.

Minute and hourly data are aggregated from tick data and serve as a practical middle ground, useful for intraday volatility estimation and real-time risk monitoring, with a much smaller data volume than tick data.

Daily data, consisting of end-of-day closing prices, is the most widely used frequency for portfolio risk assessment and regulatory reporting. It is cheap to store and process.

Weekly and monthly data provide further aggregation that is particularly useful for long-term analysis and helps avoid asynchronous trading issues that arise when comparing markets across different time zones and holiday schedules.

Higher frequency data creates three practical problems — storage, cost and processing speed.

Data frequency changes the risks you see. Higher frequency data captures intraday volatility that daily data misses.

2.5 Other data types

Derivatives data adds strikes, expiry dates and contract specifications that equity data does not need. Options data also carries the Greeks — delta, gamma and the other sensitivities — and implied volatility.

Most data vendors provide FX coverage including spot rates, forward rates and cross-currency pairs. FX data is needed to convert foreign positions into the base currency when a portfolio holds international assets.

Alternative data sources have become more common in risk assessment, supplementing traditional financial metrics with information from news sentiment, social media, satellite imagery and ESG ratings. Take satellite images of port congestion or news flow around a troubled bank. Both can signal stress before it shows up in returns. These data types require careful validation and standardisation. Many vendors now offer alternative data products alongside their core financial data services.

Which vendor supplies the data matters less than knowing what the vendor did to it before you saw it. Two series carrying the same ticker over the same dates can differ in their split adjustment, their dividend treatment and their handling of a day the exchange was shut, and each of those differences moves a volatility estimate. Chapter 30 lists the sources used in these notes and what each one covers.

Chapter 11 describes the specific data files used in these notes.