What is an AI trading competition?
An AI trading competition is any contest where the entrant builds a system and the system trades or predicts. Human judgement goes into the design and the system makes each single decision, which sets these contests apart from the manual contests in our trading competitions overview. The audience differs too. It is made up of developers, data scientists, quant students and researchers testing models.
Common formats
- Bot versus bot on a paper market: entrants send in algorithms that trade against each other in a mock exchange with an order book. Market-making and arbitrage strategies are common.
- Paper trading on live markets: bots or agents trade paper accounts on real prices over a forward period, ranked like a normal contest.
- LLM agent competitions: language-model agents get market data, news or filings and decide trades through a set tool interface. Hosts rank them on outcomes. Projects such as Alpha Arena use these contests as public benchmarks to compare models.
- Forecasting competitions: Kaggle-style contests where entrants predict returns or other targets on a dataset, scored on prediction error or correlation on a hidden test set, as in the Numerai tournament.
- Hybrid contests: humans and bots compete on the same leaderboard, or humans set up and oversee an AI assistant.
How are AI trading competitions structured?
AI trading competitions rest on three design choices: the data, the sandbox, and the rules on latency and compute.
Data
Hosts must decide what data entrants get, from past prices for training and a live feed during the contest to other data such as news or anonymised features. Keep the training and test periods cleanly apart. Publish exactly which data entrants can use at each decision point.
Sandbox and execution
Most entries run inside a sandbox the host controls. Others connect through an API with fixed rate limits. The sandbox sets the allowed order types, how it simulates fills and fees, the position limits and what happens when an entry crashes. A clear execution model matters, because if the simulator fills orders too freely, the contest rewards strategies that would fail in real markets.
Latency and compute rules
In bot-versus-bot markets, speed can decide the outcome. Fair contests either give every bot the same speed, for example by grouping orders into fixed time slots, or make speed irrelevant by trading on slower timescales.
LLM agent contests need three more rules: model access, the compute or token budget per decision, and whether agents may browse the internet. Without them, the result measures budget instead of skill.
Which metrics decide an AI trading competition?
Most well-run AI trading competitions combine a return measure with a risk measure. No single metric is enough. Publish the formula before the contest starts.
| Metric | What it measures | Watch-out |
|---|---|---|
| Total return | Growth of the account over the period | Rewards high risk over short periods |
| Sharpe ratio | Return per unit of volatility | Unstable over short windows; assumes normal-ish returns |
| Sortino ratio | Return per unit of downside volatility | Same short-window problem |
| Maximum drawdown | Largest fall from peak equity | Best used as a limit alongside a return metric |
| Turnover and costs | How much the strategy trades and what it pays | Ignoring costs flatters high-frequency entries |
| Forecast error or correlation | Accuracy of predictions on held-out data | Only meaningful if the test set is truly unseen |
Short contests are noisy. A strategy can top a two-week leaderboard by luck. Longer test periods, several scoring windows or statistical tests on the results give rankings that mean more.
Fairness and leakage pitfalls
Most problems come from data reaching the model that it would not have in real time.
- Look-ahead bias: features built with data from after the decision time, such as a daily close used for a mid-day decision.
- Data leakage between periods: overlapping training and test windows, or IDs that let entrants match test rows to known history.
- Public leaderboard overfitting: when entrants can score again and again against the same data, they tune to it. Use a final private test set or a forward period.
- LLM training data: a language model trained on text published after a past test period may in effect know what happened. Glasserman and Lin examine this look-ahead bias in GPT-based stock return predictions. Live forward testing is the safer design for LLM agents.
- Unequal resources: gaps in compute, data access or speed that the rules do not control.
- Survivorship bias: datasets that leave out delisted assets or failed tokens make every strategy look better.
Publish the scoring rules, data cut-off dates and execution model before entries open. Do not change them mid-contest. Then apply four controls through to scoring.
- Keep the final test data or forward period out of reach until scoring.
- Log every order an entry sends.
- Ask winners for their code or a description of their system that others can reproduce.
- Check odd results against those records before paying out prizes.
Why do brands run AI contests?
Brands run AI trading competitions to reach developers and quant traders. Standard trading ads rarely reach them. For trading platforms, data vendors and fintech providers, the contest also shows off the product, since a contest built on your API, data feed or sandbox is a working demo that entrants use directly.
Crypto exchanges and stock trading platforms with public APIs can use contests to drive API use, and data and tech providers can use them to show what their data supports. For channel planning beyond the contest, see our fintech marketing guide.
The same compliance questions apply as for any contest, so if AI entries trade live accounts, rules on inducements and promotions apply just as they would to human traders. In the EU, for example, ESMA Decision (EU) 2018/796 bans benefits used to market CFDs to retail clients. In the UK, FCA 2019/78 bans the same incentives. That is one reason many brand-run AI contests use paper trading.
For eligibility, prizes and measurement, see how to run a trading competition. The futures and forex guides cover market rules that bots must follow too.
Memento Media Network runs branded trading competitions end to end, including leaderboards and promotion to trading audiences across its platforms. To discuss an AI or algorithmic contest, contact our team.
Frequently asked questions
What is an AI trading competition?
A contest where trading algorithms, machine learning models or LLM agents trade a simulated market or forecast prices, with entries ranked on metrics such as risk-adjusted return or forecast accuracy.
What is an LLM trading competition?
A contest or benchmark where language-model agents get market data or news, decide trades through a set interface and are ranked on the results. Most use paper accounts. Some public benchmarks use small real-money accounts.
How are AI trading competitions judged?
Usually with a mix of return and risk metrics, such as Sharpe ratio and maximum drawdown, or with prediction error on a hidden test set for forecasting contests.
Why are backtests unreliable for LLM trading agents?
A model trained on text published after the test period may already know what happened. That inflates results. Live forward testing avoids this problem.
Sources
- Nof1: Alpha Arena, AI models trading real capitalChecked 3 November 2026
- arXiv: Glasserman and Lin, Assessing Look-Ahead Bias in Stock Return Predictions Generated by GPT Sentiment AnalysisChecked 3 November 2026
- Numerai: Tournament scoring documentationChecked 3 November 2026
- EUR-Lex: ESMA Decision (EU) 2018/796 restricting CFDs for retail clientsChecked 3 November 2026
- FCA: Conduct of Business (Contracts for Difference) Instrument 2019, FCA 2019/78Checked 3 November 2026
How we choose and recheck sources: editorial policy.