Run 2026-07-28. This is the gate phase: everything downstream (storage design, the harvester, every domain backtest) depends on the answer here.
FactIQ stores and serves only the current, latest-revised value for every observation. It does not retain the originally-published print, does not expose any way to request "value as known on date D," and has no column, dimension, or parameter anywhere in its schema that distinguishes a vintage from the current best estimate. This was true for every single period tested, with no exceptions, across two unrelated data providers (BLS, BEA).
Practical consequence: any backtest built directly on FactIQ historical data is backtesting against information that did not exist at the time — it is testing hindsight, not a strategy that could have been run live. This is exactly the failure mode this project exists to catch.
Confirmed via information_schema.columns against the shared series,
data_points, and dimensions tables in two unrelated schemas (bls and
china — deliberately picked from different providers to rule out a
one-off gap):
data_points: series_id, time, value, updated_at — no vintage,
revision, publication_date, release_date, or ingestion_timestamp column.series: id, dataset_code, series_id, series_title,
human_friendly_title, human_friendly_description, measurement_units,
frequency, adjusted_for_seasonality, state, county_or_metro_area,
begin_time, end_time, footnote_codes, updated_at, data_type,
tabular_columns — same story, updated_at is the only date-like field
beyond the observation time itself.dimensions (checked china's distinct dimension_type values):
frequency, hierarchy_level_0..3, unit — no base-year or vintage
dimension type.data_points.updated_at looked like it might be a usable proxy — it
turned out not to be. See Test 3.
Method: pulled current FactIQ values via run_sql for nonfarm payrolls
(bls.CES0000000001, FRED counterpart PAYEMS) and real GDP % change
(bea.nipa|A191RL|T10301, FRED counterpart A191RL1Q225SBEA), then queried
every historical vintage ALFRED has ever recorded for those same
observation dates. Comparison and diffing are done entirely in
src/validation/vintage_probe.py (re-run with python -m
src.validation.vintage_probe) — no hand-typed numbers, no LLM arithmetic.
Nonfarm Payrolls (thousands of persons):
| Period | FactIQ value | Original print (ALFRED, date) | Current revised (ALFRED) | # vintages | Verdict |
|---|---|---|---|---|---|
| 2025-11 | 158,449 | 159,552 (2025-12-16) | 158,449 | 3 | matches CURRENT |
| 2025-12 | 158,432 | 159,526 (2026-01-09) | 158,432 | 3 | matches CURRENT |
| 2026-01 | 158,592 | 158,627 (2026-02-11) | 158,592 | 3 | matches CURRENT |
| 2026-02 | 158,436 | 158,466 (2026-03-06) | 158,436 | 3 | matches CURRENT |
| 2026-03 | 158,650 | 158,637 (2026-04-03) | 158,650 | 3 | matches CURRENT |
The November 2025 row is the clearest single data point in this whole survey: the originally published payrolls figure was 159,552,000. As of today FactIQ reports 158,449,000 for that same month — a 1,103,000 person gap, and FactIQ shows only the revised number, with zero indication that a different figure was ever published.
Real GDP, % change annualized:
| Period | FactIQ value | Original print (ALFRED, date) | Current revised (ALFRED) | # vintages | Verdict |
|---|---|---|---|---|---|
| 2025 Q1 | -0.6 | -0.3 (2025-04-30) | -0.6 | 4 | matches CURRENT |
| 2025 Q2 | 3.8 | 3.0 (2025-07-30) | 3.8 | 3 | matches CURRENT |
| 2025 Q3 | 4.4 | 4.3 (2025-12-23) | 4.4 | 2 | matches CURRENT |
| 2025 Q4 | 0.5 | 1.4 (2026-02-20) | 0.5 | 3 | matches CURRENT |
| 2026 Q1 | 2.1 | 2.0 (2026-04-30) | 2.1 | 3 | matches CURRENT |
2025 Q4 is the standout: the advance estimate was +1.4%, the currently-revised figure is +0.5% — a full percentage point of swing, enough on its own to flip a directional call on growth momentum. FactIQ shows only 0.5%.
Every single row in both tables matches the current revised value and none match the original print. Ten out of ten. This is not a partial or inconsistent result — it's a clean, unambiguous pattern.
Raw output: findings/vintage-probe-results.json.
Tried every plausible mechanism:
data_points.updated_at as a proxy for "value as it stood before date
D": queried WHERE updated_at <= '2025-12-01' AND time = '2025-11-01'
for nonfarm payrolls — zero rows returned. The single row for that
observation carries updated_at = 2026-07-02; there is no earlier row
holding the original 159,552 value. This confirms the update is
destructive (overwrite-in-place), not append — the prior value is gone,
not superseded by a newer row that could be filtered out.run_sql, get_series, or describe_dataset
accepts an as-of date.Conclusion: there is no way, today, to ask FactIQ for a historical value as it was known on a past date. The information required to answer that question has already been discarded.
Time-boxed given how unambiguous Tests 1–3 already are. Two observations, neither a full worked example of silent restatement (none was found or ruled out conclusively — flagged as open, not resolved):
dimensions table has no base-year or vintage dimension type at
all (Test 1), which is structurally consistent with not preserving an
original basis when a series is rebased — but this wasn't proven with a
concrete before/after example.china_customs carries two dataset codes: china_customs (HS-level
detail) and china_customs_prelim (aggregate preliminary totals, e.g.
china_customs_prelim_total_export_cny). This looked initially like it
might be a paired preliminary/final vintage capture, but no non-prelim
counterpart series exists for the same aggregate concept — it appears to
be China's own separately-published preliminary release, not a
FactIQ-side vintage mechanism. Not chased further; worth a direct
question if raised.china_customs_prelim_total_export_cny has three consecutive
months (2025-06, 2025-07, 2025-09) carrying the identical value
23445.46350339 — looks like a stale carried-forward figure. Not
independently verified against the Chinese-language source; flagged as a
data-quality caveat, not a vintage finding.Baseline snapshot taken today across three series spanning three unrelated
providers (BLS payrolls, BEA GDP, China Customs preliminary exports) —
findings/drift-baseline.json. Diff logic (src/validation/drift_check.py,
diff_snapshots) is written and deterministic. findings/drift-log.md is
initialized and will accumulate entries.
This is not yet running on a schedule. FactIQ has no plain REST endpoint in this project — it's reachable only via MCP tool calls inside a Claude Code session. The Phase 3 harvester needs its own answer for how a standalone daily cron process authenticates to FactIQ before Test 5 can run unattended; until then, snapshots have to be re-pulled manually. This is a real architectural open question, not a detail — flagging it now so it doesn't get silently assumed away when Phase 3 starts.
See findings/questions-for-factiq-vintage.md (already sent). Given what
Test 2 shows, the highest-priority follow-ups from that list are A2/A3
(is the update in-place, and is any pre-revision value retained anywhere
internally) and C7/C8 (does FactIQ pull from the advance release or a
batch/bulk file that already reflects revisions) — everything else in that
list is now secondary to confirming those.
FactIQ replied to findings/questions-for-factiq-vintage.md directly, in
writing. This section records what they said, verbatim where it matters,
against the specific tests above — this is now a source-confirmed finding,
not just an empirically inferred one.
We currently do not save pre-revision data and don't have point-in-time vintages of the data. It's not on our near-term (1 month) roadmap, but I will certainly look into it.
Mapped to the open questions this project raised:
updated_at resets on every refresh
regardless of whether the value actually changed, which rules it out even
as a weak "did this change recently" signal.harvest_date remains the only point-in-time mechanism this project has
or can plan around, and re-running the vintage probe periodically remains
the correct way to notice if that ever changes, per
findings/status-summary.md.FactIQ also flagged newly-released satellite-derived data — night lights,
port activity, and fires. All three were already found present and load-
bearing in Phase 0 (findings/00-capability-survey.md's satellite table),
run one day before this survey's query date — night lights was in fact
already identified there as the substitute proxy for Domain A after the
Yunnan reservoir gap. No new capability is added by this reply; it's
corroboration that FactIQ considers these intentional, current products
worth highlighting, not an incidental or unsupported dataset.
harvest_date insurance-policy column
described in the original spec is not a nice-to-have, it is the only
mechanism by which this project will ever have point-in-time data for
anything sourced from FactIQ. Every FactIQ-derived observation is only
as "as-of" as the day it happened to be harvested.Independent personal project by Ian Bergin. Not affiliated with, endorsed by or sponsored by FactIQ, which is used as one of several data sources.
Password required.