Reproducible Benchmark · NAB + ETT Datasets
33,000–48,000 Predictions/Second. Lower Error Than Deep Learning.
DriftMind was benchmarked against Adaptive ARIMA, Prophet, and OneNet — the leading deep learning architecture for online forecasting. It ran on a standard CPU with no GPU, no pre-training, and no retraining.
Results Summary
| Metric | Value |
|---|---|
| Predictions per second (CPU) | 33–48K |
| MAE on NAB Machine Temp. | 0.82 |
| Speed compared to OneNet | 140× faster (25 seconds vs 58 minutes) |
| Retraining cycles | 0 (vs up to 17,486 for Prophet) |
Benchmark 1 — NAB Multi-Dataset
Three-Way Direct Comparison
All three models were evaluated on six datasets from the Numenta Anomaly Benchmark (NAB) — spanning machine sensors, cloud infrastructure, urban demand, ad-exchange pricing, and social-media volume — at three forecast horizons (h = 1, 6, 24). Results are reported per horizon:
| Model | MAE (lower is better) | Throughput (pred/s) | Total Time | Retraining Cycles |
|---|---|---|---|---|
| DriftMind | 0.8526 | 10,274 | 2.2 s | Continuous |
| Adaptive ARIMA | 0.8529 | 47 | 8m 3s | 901 |
| Triggered Prophet | 2.9901 | 13 | 29m 0s | 4,105 |
Key Finding
On seasonal workloads (NYC Taxi, CPU Utilization, Twitter Volume), DriftMind wins accuracy at every horizon — by 1.4–5× at h = 6 and h = 24 — and is orders of magnitude faster. On non-seasonal, low-amplitude signals (Machine / Ambient Temperature, Exchange-2), ARIMA is competitive or better at short and long horizons.
Benchmark 2 — ETTh2 & ETTm1 vs OneNet
DriftMind vs. State-of-the-Art Deep Learning
DriftMind was benchmarked against OneNet, the leading deep learning architecture for online time series forecasting under concept drift.
| Model | MAE | MSE | Total Runtime | Hardware | Warm-up Required |
|---|---|---|---|---|---|
| DriftMind | 0.232 | 0.145 | 00:00:25 | CPU only | None (cold-start) |
| OneNet (NeurIPS 2023) | 0.348 | 0.380 | 00:58:32 | RTX 3080 Ti GPU | 25% dataset |
What this shows
On ETTh2, DriftMind is both faster and more accurate than OneNet. On ETTm1, OneNet achieves a marginal accuracy edge at longer horizons, while DriftMind remains 27–28× faster and requires no warm-up window.
The Latency of Learning
Retraining Lag
ARIMA and Prophet rebuild internal models as new data arrives, leading to structural lag in production systems.
Continuous Adaptation
DriftMind does not pause, does not retrain, and does not reprocess history. Each new observation updates the internal representation immediately, sustaining thousands of predictions per second.
CPU-Only Economics
Running on commodity hardware eliminates GPU cluster costs, making DriftMind economically advantageous in environments with many time series.
Benchmark Methodology
Setup & Reproducibility
Both benchmarks were designed to expose all models to the same data in the same streaming order without cherry-picking or hyperparameter tuning.
1. NAB Dataset
Machine Temperature System Failure series with 22,695 data points.
2. ETTh2 & ETTm1 Datasets
Publicly available datasets for fair comparison.
3. Hardware
DriftMind: Intel Core i7-12700K @ 3.60GHz, 32 GB DDR4. OneNet: same machine, NVIDIA RTX 3080 Ti.
4. Cold-Start Condition
DriftMind begins predictions from the first observation. OneNet requires 25% of the dataset as pre-training.
5. OneNet Replication
The official OneNet GitHub implementation was used, confirming benchmark integrity.
6. DriftMind Settings
Input length: 20–60, Max clusters: 200, Sliding window gap rate: 2.0.
Reproduce the NAB benchmark yourself — run the full comparison locally with a single Docker command.
Run DriftMind on Your Own Data
Results in seconds.