Knowledge-grounded agentic quantitative strategy research & anti-overfitting validation framework.
agentic-quant-lab (agentic_quant) combines a local RAG-grounded LLM research loop (Generate -> Backtest -> Diagnose -> Iterate -> Track) with a statistical anti-overfitting validation suite (CSCV/PBO, Deflated Sharpe Ratio, Purged Walk-Forward Analysis, 4-Quadrant Market Regime decomposition, and Out-of-Sample Embargo tracking).
Market data ingestion and live broker execution are intentionally decoupled: every pipeline, backtest engine, and validation gate operates purely on caller-supplied Bar lists, NumPy OHLCV arrays, or an explicit fetch_func callback.
+-------------------------------------------------------------------------+
| 1. Knowledge-Grounded RAG |
| docs/knowledge_base/*.md |
| -> MarkdownChunker -> TfidfEmbedder / SentenceTransformersEmbedder |
| -> InMemoryVectorStore -> HybridRetriever / BookRetriever |
+------------------------------------+------------------------------------+
| retrieved theory snippets
v
+-------------------------------------------------------------------------+
| 2. Agentic Research Loop |
| ResearchOrchestrator |
| -> StrategyGenerationPipeline (natural language -> AST-checked code) |
| -> BacktestEngine (event-driven, close / next_open execution) |
| -> DiagnosisPipeline (AFML-inspired flaw & left-tail risk diagnosis) |
| -> AutoIterationPipeline <-> ExperimentTracker (JSONL lineage log) |
+------------------------------------+------------------------------------+
| candidate strategies & param grids
v
+-------------------------------------------------------------------------+
| 3. Anti-Overfitting Validation Gate |
| +-----------------------+ +--------------------+ +-----------------+ |
| | Overfit Detection | | Purged Walk-Forward| | Embargo & | |
| | - CSCV / PBO | | - Purge gap | | Regime Analysis | |
| | - Deflated Sharpe | | - Majority rule | | - 4-quadrant | |
| | - Parameter stability | | - Catastrophic veto| | trend x vol | |
| | & cliff detection | | - Regime breakdown | | - OOS tracking | |
| +-----------------------+ +--------------------+ +-----------------+ |
+-------------------------------------------------------------------------+
- Zero-bloat LLM Client (
LLMClient): Standard-libraryurllibHTTP client supporting OpenAI-compatible endpoints (DEFAULT_LLM_PROVIDER,<PROVIDER>_API_KEY,<PROVIDER>_BASE_URL), with automatic retries, token accounting, and SHA-256 disk response caching (24hTTL). - Local Hybrid RAG (
agentic_quant.ai.rag):MarkdownChunker+TfidfEmbedder(zero external ML dependencies) +InMemoryVectorStore+HybridRetriever(semantic cosine + keyword matching, optionalSentenceTransformersEmbedder).BookRetriever: lightweight BM25-style retriever over local markdown knowledge bases (docs/knowledge_base/).
- End-to-End Orchestration (
ResearchOrchestrator):StrategyGenerationPipeline: retrieves domain context, constructs prompts, extracts Python code, and validates syntax viaast.parseplusStrategysubclass checks.DiagnosisPipeline: summarizes backtest metrics, worst-performing trades, and drawdown windows into structured JSON diagnoses (DiagnosisReport).AutoIterationPipeline&ExperimentTracker: runs iterative parameter refinement rounds and persists experiment lineage (records.jsonl) for parent-child comparison.
- CSCV & Probability of Backtest Overfitting (
overfit.py):cscv_pbo: Combinatorially Symmetric Cross-Validation over chronological time blocks (preserving autocorrelation within each block), outputtingPBOResultwith PBO, logit scores, IS-to-OOS performance degradation, Deflated Sharpe Ratio (_compute_deflated_sharpe), and letter grades (A/B/C/D).ParameterStabilityAnalyzer: 1D and grid marginal parameter plateau width and cliff detection (>50% adjacent drop).run_pbo_single&run_pbo_batch: parameter-grid PBO evaluation over OHLCV arrays or caller-suppliedfetch_func.
- Purged Walk-Forward Analysis (
walk_forward.py):WalkForwardOptimizer&WalkForwardAnalyzer: rolling IS-OOS splits with explicit purge gap to prevent label leakage, majority rule (>=70% positive OOS windows), catastrophic veto on severe window drawdowns, parameter stability scoring, and Markdown/JSON report generation.
- 4-Quadrant Market Regime Classifier (
regime.py):MarketRegime: classifies bars intobull_high_vol,bull_low_vol,bear_high_vol, andbear_low_volusing MA(200) + rolling slope and rolling 20-day volatility vs. historical median, enabling regime-segmented WFA evaluation.
- Out-of-Sample Embargo Tracking (
embargo.py):EmbargoManager: logs post-backtest OOS signals, verifies realized next-day OHLCV outcomes, tracks cumulative PnL, triggers stop-loss / daily / cumulative circuit breakers, and produces daily reports.
- Pure-NumPy Signal Engine & Risk Metrics (
signal_engine.py,labeling.py,vpa.py,weekly.py,risk/metrics.py):- Vectorized signals (
sma,kama,donchian,donchian_regime,bollinger), Volume-Price Analysis confirmation, Triple-Barrier labeling (triple_barrier_labels), and non-normal Sharpe diagnostics (probabilistic_sharpe,deflated_sharpe,calmar_ratio,sortino_ratio,omega_ratio).
- Vectorized signals (
3. Event-Driven Backtest Engine & Sample Strategies (agentic_quant.backtest, agentic_quant.strategy)
BacktestEngine: bar-by-bar simulation supportingexecution_timing="close"or"next_open"(to avoid look-ahead bias), commissions, slippage, andBacktestResultmetrics via Polars.- Generic textbook sample strategies in
agentic_quant.strategy.examples:SmaCrossStrategy,DonchianBreakout,BollingerMeanReversion,PairsTrading,VolumeProfileStrategy,MetaLabelingVpaStrategy.
Requires Python >=3.11. Core runtime dependencies are only numpy and polars.
# With uv (recommended)
uv sync --extra dev
# Or with pip
pip install -e ".[dev]"Optional extras:
.[rag]— addssentence-transformersfor dense embeddings inSentenceTransformersEmbedder.[optimize]— addsoptunaforBayesianOptimizer
import numpy as np
from agentic_quant.validation.overfit import cscv_pbo, ParameterStabilityAnalyzer
# Shape: (n_strategy_variants, n_periods)
rng = np.random.default_rng(42)
returns_matrix = rng.normal(0.0002, 0.01, size=(12, 240))
pbo_res = cscv_pbo(returns_matrix, n_combinations=100, n_blocks=16)
print(f"PBO={pbo_res.pbo:.2%}, DSR={pbo_res.dsr:.4f}, Grade={pbo_res.grade}")
stability = ParameterStabilityAnalyzer.analyze(
param_values=[5, 10, 15, 20, 25],
performance_scores=[0.6, 1.1, 1.15, 1.05, 0.7],
param_name="fast_window",
)
print(f"Plateau={stability.plateau_width:.1%}, HasCliff={stability.has_cliff}")from datetime import datetime, timedelta
import numpy as np
from agentic_quant.backtest.data.structures import Bar
from agentic_quant.optimization.walk_forward import WalkForwardAnalyzer
from agentic_quant.strategy.examples import SmaCrossStrategy
rng = np.random.default_rng(7)
prices = 100.0 + np.cumsum(rng.normal(0.05, 1.0, size=300))
start = datetime(2024, 1, 1)
bars = [
Bar(
timestamp=start + timedelta(days=i),
open=float(p),
high=float(p + 0.5),
low=float(p - 0.5),
close=float(p),
volume=10_000.0,
)
for i, p in enumerate(prices)
]
wfa = WalkForwardAnalyzer(
total_start=bars[0].timestamp,
total_end=bars[-1].timestamp,
train_rolls=5,
train_pct=0.6,
purge_days=5,
)
report = wfa.analyze(
strategy_cls=SmaCrossStrategy,
bars=bars,
param_grid=[{"fast": f, "slow": s} for f in (5, 10) for s in (20, 30)],
)
print(f"OOS Positive Rate={report.oos_positive_rate:.1%}, Grade={report.grade}")Place domain markdown notes under docs/knowledge_base/, configure your OpenAI-compatible LLM environment variables (e.g., DEFAULT_LLM_PROVIDER=deepseek, DEEPSEEK_API_KEY, DEEPSEEK_BASE_URL), and run:
from agentic_quant.ai.pipeline.orchestrator import ResearchOrchestrator
orch = ResearchOrchestrator(docs_dir="docs/knowledge_base")
session = orch.run(
description="Trend-following moving average crossover with volatility-scaled stop loss",
bars=bars,
max_iterations=2,
)
print(session.diagnosis)src/agentic_quant/
├── ai/
│ ├── experiment/tracker.py # JSONL experiment lineage tracker
│ ├── pipeline/
│ │ ├── generate.py # RAG + LLM code generation & AST check
│ │ ├── diagnose.py # Structured failure & tail-risk diagnosis
│ │ ├── iterate.py # Automated parameter mutation & re-backtest
│ │ └── orchestrator.py # End-to-end research session coordinator
│ ├── rag/ # Markdown chunker, TF-IDF/BM25/hybrid retrievers, prompts
│ ├── llm_client.py # Stdlib urllib LLM client with disk cache
│ └── mvp_validation.py # Snapshot & risk-review prompt builder
├── backtest/ # Event-driven engine, portfolio, result, Bar/Tick, features
├── indicators/ # Wilder ATR
├── optimization/ # GridSearch, Bayesian (optional optuna), Purged WFA
├── risk/ # Calmar, Sortino, Omega, PSR, DSR
├── strategy/ # Strategy registry + generic sample strategies
└── validation/ # CSCV/PBO, Embargo, Regime, Signal Engine, VPA, Labeling, Weekly
tests/ # Offline unit & contract test suite
uv run --with pytest --with numpy --with polars pytestAll tests are offline and self-contained using synthetic market series and mocked LLM responses.
- Not a live trading, brokerage, or market-data platform — Market data feeds and order routing are decoupled (
fetch_func=NoneraisesValueError). - Not an autonomous sandboxed strategy runner —
ResearchOrchestrator.run()intentionally does notexec()LLM-generated strategies (it runs a placeholderSmaCrossStrategyuntil a human reviews and integrates generated.pyfiles). - Not an intraday tick/order-book or margin simulator —
BacktestEnginesimulates single-asset bars with flat commission rates and additive slippage, without borrow/funding fees or partial fills. - Not a persistent vector database —
InMemoryVectorStoreholds NumPy arrays in RAM and rebuilds its TF-IDF index per process. - Not financial advice — Statistical validation gates reduce overfitting risk on historical bars but do not guarantee future performance.
- Default
closetiming vs strict T+1 execution —BacktestEnginedefaults toexecution_timing="close"for legacy compatibility across WFA/grid/iteration loops; strict next-open T+1 execution (which excludes overnight entry gaps) is enforced invalidation/signal_engine.py(run_strategy()). - CSCV sampling & direction-agnostic DSR —
cscv_pbo()uses Monte Carlo block sampling rather than exhaustiveC(S, S/2)combinations, and_compute_deflated_sharpe()usesabs(mean_logit)so a consistently overfitting strategy (logit > 0) also scores highdsr; always evaluatedsrtogether withpbo. - Helper look-ahead & short-window boundaries —
apply_weekly_filter()broadcasts each week's final close across Mon–Fri unless explicitly lagged by 1 week;MarketRegime.classify()computes full-sample median volatility for post-hoc WFA attribution (useclassify_regime_percentile()for causal features); defaultpurge_days=5drops short intraday windows whose test span is shorter than the purge gap.
- Zero dynamic execution in core package —
src/agentic_quant/contains zerosubprocess,eval,exec, orpicklecalls; generated strategies are only AST-parsed for structure (ast.parse) and must be audited manually before import/execution. - Scoped network & disk cache — Only
LLMClient.chat()touches the network (urllib.request.urlopento<PROVIDER>_BASE_URL/chat/completionswith a Bearer token); treatbase_urlas trusted configuration (no URL scheme/host allowlist is enforced). Plaintext LLM prompts and responses are cached for 24 hours underdata/cache/llm_cache/keyed by a 16-hex-char SHA-256 prefix. - Trusted local inputs — Markdown files in
docs/knowledge_base/are injected verbatim into RAG prompts without prompt-injection filtering, andStrategyGenerationPipeline.save()builds output filenames from lightly sanitizeddescriptionprefixes.
- This library provides research and statistical validation tooling; it makes no claims of future trading returns or live execution readiness without your own out-of-sample, transaction-cost, and slippage verification.
- Passing
fetch_func=Nonetorun_pbo_batchintentionally raisesValueErrorbecause live market data providers are decoupled from the core package.
MIT © rockcli