What is an AI forecast audit?

An independent test of whether a forecasting system actually beats a baseline. The model is re-run on your own history using walk-forward validation, scored out-of-sample against naive and seasonal naive baselines, and reported as forecast value-add by category and horizon. The full method and code are handed over with the result.

Why not just trust the vendor's accuracy numbers?

Vendor accuracy figures are usually produced in-sample, on data the model has already seen, and are rarely compared against a seasonal naive baseline. A model can post a strong in-sample score and still fail to beat a simple baseline out-of-sample. Testing out-of-sample is the only way to know which one you have.

What happens if the audit finds the AI does not beat the baseline?

That is the reported result. Vinayak has already published a negative result on his own system: AlphaOS moved from +0.27 Sharpe in-sample to -0.10 out-of-sample, replicated on crypto at +0.09 to -0.21. Publishing what fails is the point of an independent test.

Distributors, retailers and manufacturers in the GCC who are selecting a demand planning system, have recently implemented one, or suspect their forecast accuracy numbers are not measuring what they think.

What has Vinayak measurably delivered as an operator?

At ANDS Dubai, Vinayak's documented outcomes include 20% fewer freight errors and a 25% improvement in on-time delivery. Planning about 100,000 units per month describes the scope of that role, not a performance outcome.

Accepting engagements · Dubai & GCC

Does it actually beat the baseline?

I'm Vinayak Bhadani, a demand planner in Dubai who builds the models he plans with. I run independent, out-of-sample tests on AI and statistical forecasting — the tests vendors don't run and consultancies don't publish — and I tell you whether the system beats a seasonal naive baseline. Including when it doesn't.

Request a forecast audit See the method → LinkedIn

Independent forecast validation, S&OP advisory and planning-system builds · no software to sell you

Exhibit A

Evidence I publish what fails

I tested my own trading platform properly. It didn't work.

AlphaOS is an agentic multi-market trading system I built and deployed — four research agents, an AI signal pipeline, a full backtesting engine. Then I ran walk-forward validation on it and published the result on my own site: the strategies do not beat buy-and-hold out-of-sample. The in-sample numbers looked good. The out-of-sample numbers did not. I said so publicly, in writing, with the code.

Equities — Sharpe

+0.27→−0.10

Crypto — Sharpe

+0.09→−0.21

What did hold

−50% drawdown

In-sample to out-of-sample, both asset classes. The drawdown reduction of roughly half was the one result that survived every variant — so that's what I reported as real. This is the standard I hold your forecasting vendor to, and it's the standard I hold my own work to first.

Read the full validation write-up

Engagements

Three ways to work together. All of them start with the same call: what number are you trusting, and who produced it?

Core engagement

AI Forecast Audit

An independent verdict on whether your current or proposed forecasting system earns its licence fee. Most useful when you're mid-way through a vendor selection and the numbers in the deck came from the people selling you the software.

Walk-forward validation on your own demand history
Scored against naive and seasonal naive baselines
Forecast value-add by category and horizon
Written verdict, full method and code handed over

2–3 weeks · scoped to your SKU count and history depth

Ongoing

Validation retainer

I sit beside you through vendor selection and the first quarters of go-live, and hold the vendor to the accuracy number they promised in the pitch. The only person in the room with no incentive to tell you it's going well.

Independent scoring each planning cycle
Vendor claim vs. measured reality, in writing
S&OP cadence run as a decision forum
Escalation memos your board can read

Monthly · minimum three months

Build

Planning system build

When the audit says the answer is to build rather than buy. I've shipped the forecasting models, the ETL, and the dashboards — every tool on this site is mine, with public commit history.

Python forecasting with validated baselines
ERP and Shopify ETL pipelines
Power BI and web planning dashboards
Deployed on live data, not a prototype

Project-scoped · after an audit, usually

Why an independent

Everyone in this market has a reason to tell you the forecast is working. Here's who can't give you a straight answer, and why.

Who you'd normally ask	Why the answer is compromised
The software vendor	Structurally cannot publish a test showing their own product loses to a seasonal naive baseline.
The implementation partner	Earns on the rollout. A negative verdict kills their own pipeline before it starts.
Your data science team	Usually benchmarks against a moving average or against nothing, because a planning baseline is a planning concept, not an ML one.
Your planners	Know exactly what the right baseline is. Rarely have the harness to test against it.
An independent operator	No licence to sell, no rollout to win. Paid for the answer, not for the answer being yes.

The method

Nothing proprietary and nothing hidden. You get the code, so you can re-run it after I leave.

Reconstruct the history

Clean demand history, not shipment history. Promotions, stockouts and Ramadan shifts separated out before anything gets scored.

Set the honest baseline

Naive and seasonal naive, computed on your data. If a model can't beat these, nothing else about it matters.

Walk it forward

Roll the model through time, forecasting only from what it could have known at each point. No hindsight, no leakage, no in-sample flattery.

Report the value-add

Accuracy, bias and forecast value-add by category and horizon — including the segments where the model loses. Especially those.

Operator record

I'm not an auditor who read about planning. I've carried the number.

+25%

Documented on-time delivery improvement at ANDS Dubai

−20%

Documented reduction in freight errors at ANDS Dubai

100K+

Prior monthly planning scope, not a performance outcome

UAE stores in current demand-planning scope

Distribution and consumer goods at ANDS Dubai; automotive spare parts at Global Automobiles India. Systems: SAP, Odoo, Dubai Trade Platform, Power BI. Tri-city MBA exposure across Dubai, Singapore and Sydney.

Don't take my word for it

Walk-forward validation, in fullThe method, applied to my own system, with the result that didn't flatter me. AI in demand planning: hype vs realityWhat the forecasting vendors are actually selling, and what survives contact with data. The seasonal index trapHow a standard ABC and seasonality setup quietly manufactures a wrong number. Planning calculatorsSafety stock, EOQ, reorder point, MAPE. Free, no signup. Run them yourself. All field notesWritten from inside GCC operations, not from a template. GitHub ↗The code behind the models, with public commit history.

Credentials

MGB, SP Jain School of Global Management
BEng, Sri Venkateshwara College of Engineering

CISCP & CISCM, IPSCMI (USA)
McKinsey Forward Program, 2025 · APICS CPIM in progress

Start the conversation

Tell me what system you're running or considering, how much history you have, and what made you doubt the numbers. If an audit isn't the right answer, I'll say that too.

Emailvinayakbhadani@zohomail.in WhatsApp+971 55 627 0561 LinkedIn/in/vinayakbhadani BackgroundFull CV (PDF)