Evaluation methodology
How AI market notes are evaluated.
MarketPulse Hub is building evaluation before publishing performance-style summaries. This page explains how generated educational notes can be reviewed against later real-market quote history, and why early results stay internal until there is enough reviewed data.
What this page explains
This is not a public performance page. It describes the internal review method used for paper tests, targeted tests, and future subscriber market notes. Public statistics will only be considered after enough evaluated notes exist, the sample is reviewed, and the limitations can be shown clearly.
From note to quote snapshot
A market note records the instrument, timeframe, broad direction language, text, and creation time.
The note stores the real quote snapshot available at generation time, including source-count and quality metadata.
Fresh market quotes are stored separately in quote history so later review has a time-based reference.
When a note becomes evaluable
A note is reviewed only after its evaluation due time. The evaluator compares the stored entry snapshot with later quote history after that due time. It is designed to avoid choosing the most favorable future price or searching for the best possible movement.
Quality filters
Missing quote snapshots, manual messages without market data, or old rows without entry context are excluded.
If later quote history is missing or too weak, the note is not counted as a reviewed result.
Stale, single-source, or strongly disagreeing quote data can be skipped to avoid overstating quality.
What directional alignment means
Directional alignment is a narrow internal QA metric. It checks whether later market movement aligned with the broad scenario stored with an educational note under the evaluator rules. It is not a trading result, not a profit measurement, not a broker execution result, and not a recommendation to buy or sell.
Why public statistics are not shown yet
Early paper notes are useful for testing the pipeline, but a few rows are not enough for public claims. Before any public summary is considered, the sample size, date range, scopes, excluded rows, quote-quality limits, and methodology must be visible beside any number.
How future stats should be read
Every percentage needs a visible count of reviewed notes, not only a headline number.
Not-evaluable, stale, or insufficient-data rows must remain visible so weak data does not disappear.
Internal paper notes, targeted tests, and subscriber broadcasts should not be mixed without labels.
Limits of evaluation
Evaluation uses indicative quote history, not broker fills, spreads, fees, slippage, taxes, order execution, position sizing, or personal risk limits. Markets can move quickly and data can be delayed, interrupted, or unavailable. Educational notes remain informational only.
FAQ
Are these results a prediction accuracy score?
No. Directional alignment only checks whether a later market move matched the broad scenario stored with a note under the evaluator rules. It is not a trading win rate, profit measurement, or prediction score.
Why are some notes not evaluated?
A note may be skipped if the entry snapshot is missing, later quote data is stale, source agreement is weak, or the message was manual and did not contain a quote snapshot.
Do evaluations use the best price after a note?
No. The evaluator uses later quote history after the due time. It is designed to avoid choosing the most favorable future price.
When will public statistics be available?
Only after enough evaluated notes exist and the results can be reviewed with clear limitations. Until then, evaluation data remains internal quality assurance.
Can I trade based on these notes?
No. The notes are educational market context, not investment advice, trading instructions, or a recommendation to buy or sell.