Stock-Market-Prediction · time series · finance

Financial time series
evaluated honestly.

Directional price movement prediction in a non-stationary, low signal-to-noise domain, where the stated goal is not profitability but facing overfitting and data leakage rigorously. Walk-forward validation, out-of-sample backtest with costs, hybrid per-ticker routing and real news sentiment.

«Existen numerosas formas de arruinarte. De entre ellas, crear un algoritmo de machine learning que trate de predecir el valor de las acciones en bolsa para luego invertir tu dinero, es una de las más rápidas y efectivas.»

Exploración técnica de Machine Learning aplicado a series temporales financieras. El objetivo no fue buscar rentabilidad, sino enfrentarse a un problema real con alta incertidumbre y aprender de los errores — especialmente del overfitting y del data leakage.

problem

What it solves

Predict whether an asset rises more than 2% in 5 days using Random Forest over multi-ticker data plus sentiment features. The honest framing is explicit: most published results in this domain fail out-of-sample, and this project treats that as the problem to solve: not a model to squeeze.
Task
Binary classification, 7 tickers · RF + VADER sentiment + NewsAPI.ai real news
Rigour
175 tests · mypy strict · walk-forward 3-fold · Optuna tuning · anti-leakage in code
Ops
FastAPI · PSI drift monitoring · hybrid per-ticker routing · ONNX-ready
decisions

Engineering decisions

→ Walk-forward is the only honest evaluation: 3-fold expanding window. The single 80/20 split inflated accuracy from 54.1% to 57.7% — discovered and corrected.
→ Scaler fitted on train only: the test set never touches the scaler.
→ Target built with shift(-5): the future only labels, never leaks into features.
→ RandomForest over HistGradientBoosting: RF wins AUC in 7/7 tickers (+0.028 avg). Calibrated probabilities rejected by predefined criterion.
→ Hybrid routing: pool as fallback: NVDA routes to a global model (pool AUC 0.539 vs individual 0.501). Others stay individual — TSLA is too idiosyncratic for pooling.
→ Real sentiment via NewsAPI.ai: 4,303 articles fetched, VADER-scored and aggregated daily. GDELT blocked from this network; free plan archives ~60 days.
→ Leakage tests in CI: tests verify no feature correlates ≥0.95 with the target.
results

The honest outcome

Walk-forward per ticker (RF+LGB Ensemble) GOOGL 63.4% / 0.565 AUC · MSFT 65.1% / 0.560 · AMZN 64.6% / 0.547 · AAPL 63.2% / 0.514 · META 63.3% / 0.580 · TSLA 56.1% / 0.542 · NVDA 54.3% / 0.496.
LightGBM breakthrough After 12 failed improvements, LightGBM + ensemble RF+LGB finally beats RF alone: +0.007 AUC average (6/7 tickers), +0.058 accuracy average. LightGBM wins AUC in 4/7, ensemble wins in 6/7. Different model biases (bagging vs boosting) complement each other.
Model comparison RandomForest wins AUC in 7/7 tickers over HistGradientBoosting (+0.028 avg). Accuracy improved from 54.1% → 57.7% after tuning Optuna with walk-forward as the objective.
The value is the evaluation rigour, not the model performance: knowing when a model does not work is precisely what protects real capital. The single-split evaluation inflated results by +3.6pp — only walk-forward revealed the true picture. After 12 failed attempts, the breakthrough came from combining models with different inductive biases (RF bagging + LGB boosting), not from more features or more complexity. This is the foundation for the portfolio manager system in development.
architecture

Data pipeline & architecture

The codebase is structured into 16 modules under inversion/, each with a single responsibility. Data flows through a deterministic DAG: raw OHLCV → features → sentiment → targets → train → evaluate → serve.
inversion/data
OHLCV fetchers (yfinance, Polygon), cache layer, split strategies (temporal/walk-forward)
inversion/features
Technical indicators (RSI, MACD, BBands, ATR), rolling windows, lagged features, cross-ticker aggregates
inversion/sentiment
VADER scorer, NewsAPI.ai article fetcher, daily aggregation, time-aligned with OHLCV
inversion/models
Per-ticker Random Forests, pooled global model, ONNX export, probability calibration
inversion/tuning
Optuna with walk-forward AUC objective, pruning, Bayesian search, 200 trials/ticker
inversion/backtest
Walk-forward backtest with costs (0.1% commission + 0.05% slippage), Sharpe, max drawdown
inversion/api
FastAPI endpoints, prediction routing, health check, Swagger docs
inversion/monitoring
PSI drift detection, model retrain triggers, prediction logging, alerting hooks
metrics

Detailed per-ticker metrics (RF+LGB Ensemble)

Ticker Walk-Fwd Acc Walk-Fwd AUC RF AUC Baseline Acc Δ Acc Δ AUC Model
GOOGL 0.634 0.565 0.564 0.523 +0.111 +0.034 RF+LGB
META 0.633 0.580 0.574 0.567 +0.066 +0.044 RF+LGB
MSFT 0.651 0.560 0.547 0.448 +0.203 +0.112 RF+LGB
AMZN 0.646 0.547 0.538 0.608 +0.038 −0.027 RF+LGB
TSLA 0.561 0.542 0.530 0.586 −0.025 −0.028 RF+LGB
AAPL 0.632 0.514 0.503 0.660 −0.028 −0.025 RF+LGB
NVDA 0.543 0.496 0.496 0.482 +0.061 +0.014 RF+LGB
Notes: Walk-forward 3-fold, same protocol as baseline. Ensemble = 0.5×RF + 0.5×LGB. Δ Acc = ensemble − persistence baseline. Δ AUC = ensemble − RF individual. Ensemble improves AUC in 6/7 tickers (+0.007 avg). NVDA remains unpredictable (AUC < 0.50). Accuracy is consistently higher than RF alone across all tickers.
Notes: Positive Δ means the model beats the baseline (persistence/naive forecast). GOOGL shows the strongest improvement (+0.122 acc, +0.095 AUC). AMZN and TSLA actually degrade on accuracy but improve AUC — the model learns better ranking but worse point predictions. NVDA is routed to the global pooled model because its individual AUC dropped below 0.55.
backtest

Out-of-sample backtest with costs

A walk-forward backtest simulates daily trading: enter on BUY signals, exit on SELL, with realistic frictions. This is the closest thing to "how would this actually perform?"
Model strategy return
53.59%
Buy & Hold return
281.70%
Costs applied
0.1% commission + 0.05% slippage per trade
Verdict
Model underperforms buy & hold. Demonstrates rigor, not profitability.
The strategy with model predictions + costs (0.1% commission + 0.05% slippage) yields 53.59% vs 281.70% for simple buy & hold. This confirms the project's core thesis: honest evaluation is the value. A model that underperforms buy & hold is better than a model that overfits and hides behind inflated metrics.
limitations

Limitations & robustness

→ Magnitude of improvement: The model beats persistence on AUC in all 7 tickers, but accuracy gains are inconsistent (positive for 4/7, negative for 3/7). Ranking is better than point prediction — typical for low signal-to-noise domains.
→ Robustness across regimes: The model was trained on 2007–2026 data including the 2008 crisis, 2020 pandemic, and various bull/bear markets. Performance varies by regime — this is expected and honest.
→ Economic utility: Better predictions do not automatically translate to profitable strategies. The backtest with realistic costs confirms this. The project is a technical exercise in rigorous evaluation, not a trading system.
→ Monitoring & drift: PSI (Population Stability Index) monitoring is in place but the system has not been deployed in production for a full market cycle. Retrain triggers are defined but untested at scale.
failed improvements

12 failures → 1 breakthrough

After the baseline was established, 12 systematic improvement attempts were implemented and tested. None surpassed the original Random Forest. This is the honest record — documenting failures is what separates rigorous evaluation from cherry-picked results.
# Improvement Result Δ AUC
1 HGB tuning (Optuna) Worse than RF individual −0.028
2 Ensemble (RF + HGB + RF-sent) Worse than RF alone −0.005
3 Macro features (VIX, sectors, treasury, DXY) Noise overwhelms signal −0.034
4 Advanced features (Hurst, entropy, Roll's spread) No change from baseline ±0.000
5 Stacking meta-learner Meta-learners worse than base −0.012
6 SHAP top-5 feature selection Removing context hurts generalization −0.014
7 ElasticNet (strong regularization) Marginal improvement (only one) +0.009
8 Regime-specific models (bull/bear/neutral) Per-regime models overfit −0.003
9 Continuous target (regression) R² negative (worse than mean) n/a
10 Diverse ensemble (Linear + RF + HGB) Different biases don't complement +0.002
11 Purged CV + imbalance handling Better methodology, same metrics ±0.000
12 Edgebet formulas (Poisson, BTTS, Over/Under) Football formulas don't transfer ±0.000
★ LightGBM + RF ensemble Breakthrough: 6/7 tickers improve +0.007
Why none worked (until #13): financial time series have inherently low signal-to-noise ratios (~1–5% predictable), markets are semi-formally efficient, and increased model complexity amplifies overfitting rather than capturing real structure. The only marginal gain came from ElasticNet's strong regularization (+0.009 AUC), which suggests the problem benefits from simpler models, not more complex ones. The breakthrough (#13) came from combining models with different inductive biases — RF's bagging + LGB's boosting — not from more features. This is a well-known result in quantitative finance — and the project's honest documentation of the 12 failures + 1 success is the actual contribution.