News, signals and discovery — course checkpoint

You started this course able to fetch a headline. You finish able to say what a news row is, what a sentiment score measures, which parameters make an indicator mean something, and how to find an instrument you cannot yet name — plus, in every case, what the number is not.

Unit 1 — news as structured data

A /news item is seven fields: date, title, content, link, symbols, tags, sentiment. You query by s (a ticker) or t (a topic) — one is required, never both — with from, to, limit (1–1000, default 50) and offset.

Duplication arrives two ways. One article, many symbols: a single consumer-finance piece came back with 22 identifiers covering 5 companies, twelve of them Apple listings across eleven venues. One story, many publishers: wire syndication gives every copy its own link. No dedup rule is exact; you are choosing which error you prefer.

The field that decides whether a study is honest is date. Three "January 6" articles were stamped 21:31, 22:00 and 23:15 UTC — 16:31, 17:00 and 18:15 New York time, all after the 21:00 UTC close. A date-level join pairs each with a price that printed first. The rule that survives is item.date <= T, mapped to the next tradable bar.

Unit 2 — sentiment, and what a score is not

Per article, neg + neu + pos = 1.000 exactly, and neutral dominates at 78% to 98%. polarity is a separate compound score: 0.205 − 0.013 = 0.192 was reported as polarity 1.000. Two measurements, one label.

Daily, /sentiments returns {date, count, normalized} per ticker. Apple's five-day mean of 0.780 rested on 422 articles; Bitcoin's 0.125 rested on 20, and Bitcoin had no row at all for 2026-01-09 — absent is neither zero nor neutral. Word weights are equally candid: 300 of 1,986 matching articles were processed, the top ten weights summed to 0.08835, and eight of the ten were generic financial English.

A sentiment score measures text. It is not a measurement of the world, not validated by you, not on a stationary scale, not independent of volume, and not a forecast — excellent as an index into the text, dangerous as a substitute for it.

Unit 3 — indicators without the arithmetic

/technical/{ticker} computes twenty functions server-side, one per request, and the price indicators run on adjusted_close — so they are total-return figures that move when a dividend goes ex. The exceptions are named ones: splitadjusted returns split-adjusted OHLCV, avgvol a share count, and splitadjusted_only=1 restricts selected functions to splits. For AAPL.US on 2026-07-27, fetched that day: SMA(50) 306.9922, SMA(200) 276.0692, RSI(14) 67.2912, MACD(12,26,9) 8.9310 (fields macd, signal, divergence), ATR(14) 8.1406, ADX(14) 27.6999. SMA(2) returned 334.965, exactly the mean of the two closes that day — but close and adjusted_close were both 336.91 that week, so that run identified the formula only. The rerun after the next ex-dividend returned 334.6763, the mean of the adjusted pair, and that is what named the column.

The 50- and 200-day averages differ by 11.20% of the smaller, so "above its moving average" is not a fact until you name the period. ATR of 8.1406 is 2.42% of price — a price-unit number is not comparable across instruments. Store every parameter beside every value.

And the trap: a five-day mean of raw closes across Apple's 4-for-1 split on 2020-08-31 gives 278.778 for a stock that closed at 131.40. Split-adjusted, the same window gives 128.8875; on adjusted_close, 125.0175. One is broken and two are correct answers to different questions.

Pivot levels are arithmetic, not observation: P = (H+L+C)/3 = 336.8333, R1 = 2P − L = 339.6467 — computed from the row's own bar, so join it unshifted and you have re-created the Unit 1 look-ahead.

Unit 4 — finding things

/screener takes filters as [field, operation, value] triples, plus signals, sort, limit (max 500) and offset (max 999) — so at most row 1,499 is reachable. It is latest-day only, which builds survivorship into the tool. GOOG and GOOGL are two rows for one company. And without an exchange constraint, market_capitalization came back at 5,403,301,117,952 beside a ₫ symbol — a global size filter selects on currency.

/search/{query} returns ranked candidates: six rows for "Apple" included a tokenised crypto product, a Buenos Aires DRC priced in pesos, 0R2V on two UK venues, and a different company called Apple Hospitality — with one ISIN, US0378331005, spanning four of them. /symbol-change-history adds the time dimension: FB → META on 2022-06-09, and RELI → EZRA with RELIW → EZRAW on the same day. A ticker is a lease, not a name.

Under /mp/ sit separately licensed vendor datasets, where the token that fetched everything else in this course returned 403 — not 401, not 404 — sometimes as HTML, sometimes as a bare string.

The three sentences worth keeping

1. Every field in this course is a claim by somebody, and the timestamp is the only one that can invalidate all the others. Get date handling right first; everything else is recoverable.
2. A score is a model's output, not a measurement of the world — which makes it a superb index into the text and a poor replacement for reading it.
3. A number without its parameters is not data. Period, method, adjustment, currency, bar alignment: each one silently changes the answer, and none of them shows up in the value.

Before you sit it

Each of these is a minute at your desk. Any one that is not names the lesson to reopen first.

• Say which timestamp field decides whether a news study is honest — Which field decides whether your study is honest?
• Say what a sentiment score of 0.73 needs beside it before it means anything — What does a daily sentiment score of 0.73 actually mean?
• Say why a moving average computed across a split is simply wrong — Why is a moving average across a stock split simply wrong?
• Write a screener filter for one condition, from memory — How do you ask which instruments satisfy a set of conditions?

Try it now

1. Here is one small pipeline end to end for Apple, on the closed week of 21 to 25 September 2026: /news for the last day of that window, /sentiments for the whole window, and a /technical series ending on the same day. Reconcile the dates by hand once: which news items were available before the 25 September close, which sentiment row they fed, and which bar the indicator's last row belongs to. Everything in this course is in that reconciliation.

2. Take the SMA(2) audit and run it against every indicator your project uses. Any mismatch is a discovery.
3. Write down, for each dataset you have decided to use, what it measures and what it does not. If the second list is empty, you have not finished reading it.

Nothing in this course recommends an instrument, a dataset, a vendor or a strategy. It describes how published market data is produced, what it can support, and where it will mislead you if you take it at face value — which is a skill, not a signal.