{
  "schema_version": 1,
  "project": "World Economic Atlas",
  "site": "https://worldeconatlas.com",
  "updated_date": "2026-11-14",
  "status_basis": "Snapshot checked against completed GitHub Actions results through November 14, 2026 at approximately 4:35 p.m. EST. This page is a dated reference, not live telemetry; check linked runs for subsequent changes.",
  "headline": "Finish and integrate real economic datasets, not more audit infrastructure.",
  "mission": "Build a comprehensive, reproducible, cross-country economic data foundation for future research on growth, jobs, taxation, inequality, economic policy and the transition to an AI economy. Collect source-native numerical evidence first; publish or analyze only after required checks.",
  "direction": [
    "Completion first: determine the exact intended scope of each dataset, compare it with verified saved source IDs/series, and fetch the missing data. Count actual new verified observations, files and geographic coverage—not workflow runs, catalog entries or metadata.",
    "Use and repair existing GitHub collectors, the existing 125-source catalog, source registry, warehouse, watchdog and private backup. Do not create parallel generic downloaders, registries, warehouses, audit systems or dashboards.",
    "Preserve official raw sources, URLs, vintage, units, flags, geographies, source SHA-256, retrieval and storage receipts. Validation happens inside each download; it is a gate to usable data, not a separate research program.",
    "Keep source-native data, historical canonical records, derived panels and published website editions distinct. Do not add overlapping counts or infer a provider's complete universe from a selected series collection.",
    "Do not run new causal economic studies or generate policy findings until data collection, definitions, versions and treatment validation are complete. The present work phase is collection and integration.",
    "Keep source data and private warehouse archives out of the public website unless the specific release, licensing and data-quality gates are passed. A GitHub success is not proof that Cloudflare published the changes."
  ],
  "numbers": [
    {
      "label": "Catalogued source descriptions",
      "value": "125",
      "detail": "71 high-priority P0; catalog status is not proof of collection"
    },
    {
      "label": "Historical canonical warehouse",
      "value": "2,780,117",
      "detail": "Source-separated observations in 24 families; historical pinned baseline"
    },
    {
      "label": "Additional indexed source-native layer",
      "value": "2,104,609",
      "detail": "Five collections; possibly overlapping, not additive"
    },
    {
      "label": "Published WDI edition",
      "value": "300,240",
      "detail": "32 series, 261 geographies (44 aggregates), pinned November 9"
    },
    {
      "label": "Expanded WDI source",
      "value": "572,179",
      "detail": "59 configured series, 265 geography codes including aggregates; not yet the public edition"
    }
  ],
  "workstreams": [
    {
      "name": "ILOSTAT official bulk",
      "stage": "Latest 50-file batch completed; broader archive incomplete",
      "evidence": "November 14 improved 50-file run completed: 969/1,964 official source tables verified, 995 remaining. This run added 49 new files, reported one temporary World Bank-independent ILOSTAT HTTP 502 failure for EMP_PIFL_SEX_AGE_ECO_RT_A, and verified zero lost old files with SHA-checked private checkpoint backup. The earlier checkpoint had 920.",
      "next": "Continue the existing 50-file resumable schedule to retrieve the 995 outstanding official files. Retry the single HTTP 502 source on later runs; do not claim a green workflow means all attempted files succeeded.",
      "links": [
        {
          "label": "Last completed 920-file checkpoint",
          "url": "https://github.com/diegoduque16/Global-economic-policy-tool/actions/runs/43629083582"
        },
        {
          "label": "Latest successful batch (49 added, 1 failed source)",
          "url": "https://github.com/diegoduque16/Global-economic-policy-tool/actions/runs/43639113626"
        },
        {
          "label": "Batch-size PR #21",
          "url": "https://github.com/diegoduque16/Global-economic-policy-tool/pull/21"
        }
      ]
    },
    {
      "name": "World Inequality Database (WID)",
      "stage": "20 previous new countries complete; next 20 country candidates in live acquisition",
      "evidence": "Historical original WID panel: 23 countries, 21,430 selected numeric observations. The earlier monthly '20-country expansion' only refreshed baseline countries, not expansion. The November 14 five-batch WID collection successfully added 20 disjoint new country codes (FI, BE, AT, CH, PT, PL, CZ, IE, AR, PE, UY, CR, CN, ID, PH, TH, EG, MA, SN, UG) with 14,472 source-native selected numeric income/wealth distribution observations; all 20 country downloads backed up to private OneDrive with SHA verification, and all five batches reported zero failures and zero original-baseline overlap. The total is a separate source-native count, not an increment to canonical observations.",
      "next": "Existing WID workflow is now collecting another 20 candidate countries absent from all 43 previously verified country codes, run 43643338607. Count each candidate only on source-native numeric evidence and SHA-verified private OneDrive backup; track failures separately. Preserve baseline and first-wave source vintages; do not claim complete country or full-variable WID coverage.",
      "links": [
        {
          "label": "New-country PR #22",
          "url": "https://github.com/diegoduque16/Global-economic-policy-tool/pull/22"
        },
        {
          "label": "WID 20-country run",
          "url": "https://github.com/diegoduque16/Global-economic-policy-tool/actions/runs/43640485476"
        },
        {
          "label": "Second wave PR #27",
          "url": "https://github.com/diegoduque16/Global-economic-policy-tool/pull/27"
        },
        {
          "label": "Second wave active run",
          "url": "https://github.com/diegoduque16/Global-economic-policy-tool/actions/runs/43643338607"
        }
      ]
    },
    {
      "name": "Eurostat macro and labor costs",
      "stage": "Six-series source release verified and privately backed up",
      "evidence": "November 14 new Eurostat collector run completed successfully: six official source series, 3,436 numeric observations and 31 geography codes. Includes GDP and unemployment plus employment 20–64, hourly wages/salaries and total hourly employer labor cost. Original source checks and private OneDrive archive SHA-256 round-trip passed. Not yet published as Atlas's canonical or public dataset.",
      "next": "Reconcile exact series-vintage/geo/year keys with historical Eurostat observations, integrate nonoverlapping data into a versioned warehouse edition, then selectively expand employment, wages and industry coverage.",
      "links": [
        {
          "label": "Expansion PR #23",
          "url": "https://github.com/diegoduque16/Global-economic-policy-tool/pull/23"
        },
        {
          "label": "Successful Eurostat run",
          "url": "https://github.com/diegoduque16/Global-economic-policy-tool/actions/runs/43640875170"
        }
      ]
    },
    {
      "name": "World Bank WDI",
      "stage": "59 selected indicators source-verified AND permanently privately backed up; public release separate",
      "evidence": "Official full WDI bulk ZIP run 43637384781 verified 59 of 59 selected source indicators, 572,179 numeric source observations and 265 geographic codes including aggregates, with zero key/value conflicts. Follow-up watchdog run 43643193303 completed successfully and independently validated the source ZIP-to-panel values, created a permanent private OneDrive archive of the complete original source bundle, and reread it with SHA-256 6f6253a1626bb0bbcf5c9e3e31c49e5f03736e7c15f1e8f73064f8b1666f8f9a. The official original ZIP SHA was dba33648d6f453a635b076a62b91db5939f5e640c8dcfeabd094219210533100. Public Atlas still uses the earlier approved 32-series WDI edition; canonical integration and new public release are not claimed.",
      "next": "The 59-series original-source private backup is complete. Next distinguish 265 geography codes between country/territory and aggregates using official classifications, review provider source vintage and indicator-specific redistribution terms, then integrate the verified source-native panel and prepare a separately versioned 59-series public release. Do not change current 32-series release without source licensing/QA.",
      "links": [
        {
          "label": "Successful 59-series WDI run",
          "url": "https://github.com/diegoduque16/Global-economic-policy-tool/actions/runs/43637384781"
        },
        {
          "label": "Bulk recovery PR #18",
          "url": "https://github.com/diegoduque16/Global-economic-policy-tool/pull/18"
        },
        {
          "label": "Archive format fix PR #19",
          "url": "https://github.com/diegoduque16/Global-economic-policy-tool/pull/19"
        },
        {
          "label": "Verified outcome PR #20",
          "url": "https://github.com/diegoduque16/Global-economic-policy-tool/pull/20"
        },
        {
          "label": "Existing private backup watchdog",
          "url": "https://github.com/diegoduque16/Global-economic-policy-tool/blob/main/.github/workflows/atlas-data-pipeline-watchdog.yml"
        },
        {
          "label": "Original standalone importer failure",
          "url": "https://github.com/diegoduque16/Global-economic-policy-tool/actions/runs/43642665591"
        },
        {
          "label": "Permanent private WDI source backup verified",
          "url": "https://github.com/diegoduque16/Global-economic-policy-tool/actions/runs/43643193303"
        },
        {
          "label": "Direct-script validator repair PR #26",
          "url": "https://github.com/diegoduque16/Global-economic-policy-tool/pull/26"
        }
      ]
    },
    {
      "name": "FRED, BLS, U.S. Census, fiscal and remaining P0 sources",
      "stage": "Partially covered or scope not yet reconciled",
      "evidence": "Catalog lists 13 selected FRED series and partial source-family evidence for several BLS, Census, IMF, OECD and tax sources. Of 71 P0 catalog entries, the original registry links 23 to historical partial families and one to separate WID numeric data; 47 have no explicit source-specific mapping. Unmapped does NOT prove absent or uncollected.",
      "next": "Identify exact missing official indicator/table IDs from existing stored data, then extend current collectors by measured gaps. Prioritize employment, wages, firms, productivity, taxes and income distribution.",
      "links": [
        {
          "label": "125-source master catalog",
          "url": "https://github.com/diegoduque16/Global-economic-policy-tool/blob/main/open_data/expanded_source_catalog.csv"
        },
        {
          "label": "Source evidence registry",
          "url": "https://github.com/diegoduque16/Global-economic-policy-tool/blob/main/data/registry/registry.json"
        }
      ]
    }
  ],
  "decisions": [
    {
      "date": "2026-11-12",
      "title": "Consolidate existing infrastructure",
      "description": "Do not spawn additional registries, warehouses, generic ingesters or dashboards. Keep the historical baseline and source-native materializations distinct.",
      "evidence_url": "https://github.com/diegoduque16/Global-economic-policy-tool/blob/main/docs/ATLAS_CONSOLIDATION_DECISION_2026-11-12.md"
    },
    {
      "date": "2026-11-14",
      "title": "Clarify the highest-level goal",
      "description": "The user explicitly chose completion and acquisition of genuinely missing data over producing audit reports. Source-level integrity remains a built-in quality gate.",
      "evidence_url": "https://github.com/diegoduque16/Global-economic-policy-tool/blob/main/open_data/expanded_source_catalog.csv"
    },
    {
      "date": "2026-11-14",
      "title": "Recover full World Bank collection",
      "description": "Replaced outage-prone full WDI API crawl with the official source ZIP; 59 selected indicators and 572,179 source values passed independent original-source numeric verification.",
      "evidence_url": "https://github.com/diegoduque16/Global-economic-policy-tool/actions/runs/43637384781"
    },
    {
      "date": "2026-11-14",
      "title": "Make ILOSTAT collection faster, without losing safeguards",
      "description": "Changed the existing resumable scheduled loader from 20 to 50 missing tables per run; retained private SHA-verified OneDrive checkpoints and existing schedules.",
      "evidence_url": "https://github.com/diegoduque16/Global-economic-policy-tool/pull/21"
    },
    {
      "date": "2026-11-14",
      "title": "Expand WID outside the 23-country baseline",
      "description": "Replaced the old duplicative 20-country refresh matrix with 20 candidate countries outside the original numerical panel, preserving frozen historical refresh evidence. Eight new countries had succeeded by this snapshot.",
      "evidence_url": "https://github.com/diegoduque16/Global-economic-policy-tool/pull/22"
    },
    {
      "date": "2026-11-14",
      "title": "Collect more Eurostat labor and wage data",
      "description": "Expanded the existing Eurostat economic downloader from 3 to 6 source-defined series; all 3,436 numerical observations and private backup succeeded.",
      "evidence_url": "https://github.com/diegoduque16/Global-economic-policy-tool/actions/runs/43640875170"
    },
    {
      "date": "2026-11-14",
      "title": "Establish one permanent Atlas operating reference",
      "description": "Preserve completion-first direction, status boundaries, verified workflow evidence and a cumulative dated decision log in one version-controlled JSON file. Publish it as a dedicated website Project Brief with a machine-readable link and require future repository work to read it first.",
      "evidence_url": "https://github.com/diegoduque16/Global-economic-policy-tool/blob/main/docs/atlas_project_brief.json"
    },
    {
      "date": "2026-11-14",
      "title": "Complete first genuine WID geographic expansion",
      "description": "All 20 official WID country CSV downloads outside the original 23-country panel succeeded across five independent batches, yielding 14,472 selected source-native income/wealth numeric observations backed up in private OneDrive. No original-baseline country overlap; no canonical/public integration yet.",
      "evidence_url": "https://github.com/diegoduque16/Global-economic-policy-tool/actions/runs/43640485476"
    },
    {
      "date": "2026-11-14",
      "title": "Prioritize durable original-source protection for expanded WDI",
      "description": "The existing automatic coverage watchdog is being configured to run immediately after its main workflow change and to fail rather than greenwash an unverified backup of the temporary 59-series WDI artifact. This is a planned verification gate, not a backed-up-source success claim.",
      "evidence_url": "https://github.com/diegoduque16/Global-economic-policy-tool/blob/main/.github/workflows/atlas-data-pipeline-watchdog.yml"
    },
    {
      "date": "2026-11-14",
      "title": "Fix private-backup watchdog direct-script import instead of hiding failure",
      "description": "Initial WDI permanent-backup watchdog correctly failed because direct execution from scripts/ could not import the sibling open_data.wdi_bulk validator. The fix resolves the package path from the watchdog file location and adds a -I isolated subprocess regression; do not count private backup until rerun passes.",
      "evidence_url": "https://github.com/diegoduque16/Global-economic-policy-tool/actions/runs/43642665591"
    },
    {
      "date": "2026-11-14",
      "title": "Launch second completion-first WID expansion wave",
      "description": "Reused the existing monthly WID downloader to attempt 20 more official country CSVs, excluding both the baseline 23 countries and the 20 newly verified in the preceding wave. First-wave verified source counts are frozen to prevent repeat acquisition. The new run is underway, not yet a claim of 20 additional successful sources.",
      "evidence_url": "https://github.com/diegoduque16/Global-economic-policy-tool/pull/27"
    },
    {
      "date": "2026-11-14",
      "title": "Permanently back up complete 59-series World Bank WDI source",
      "description": "Following the fail-closed import repair, successful watchdog run 43643193303 re-parsed and checked the official WDI bulk ZIP against its processed 572,179 observed values, wrote the full original source archive to private OneDrive, and verified its restored SHA-256 6f6253a1626bb0bbcf5c9e3e31c49e5f03736e7c15f1e8f73064f8b1666f8f9a. Canonical integration and public 59-series publication remain separate.",
      "evidence_url": "https://github.com/diegoduque16/Global-economic-policy-tool/actions/runs/43643193303"
    }
  ],
  "next_actions": [
    "Check any future ILOSTAT scheduled source-ID runs after 969/1,964 and retain exact numeric/file evidence; verify GitHub statuses again before quoting these November 14 results.",
    "Continue acquiring the 995 currently missing official ILOSTAT bulk tables; retry temporary source HTTP 502 failures with existing resumable logic, preserve immutable checkpoints.",
    "WDI original 59-series source archive is permanently privately backed up (run 43643193303, verified archive SHA 6f6253a1626bb0bbcf5c9e3e31c49e5f03736e7c15f1e8f73064f8b1666f8f9a). Do source-series geography classification, vintage/license checks and explicitly versioned canonical/public integration; do not recollect only to redo the backup.",
    "Promote the six-source Eurostat release to a versioned source-native/canonical warehouse stage after exact vintage and overlap checks. Do not silently replace the current public website WDI edition.",
    "Check second WID expansion run 43643338607: count actual successful source-verified country files and values out of 20 candidate countries; preserve original 23 and first 20 separate, no overlap or canonical/public inclusion.",
    "Complete data acquisition in other high-priority areas based on concrete missing native series—not an assumption that 47 unmapped catalog entries are empty.",
    "Keep this dated briefing and its evidence links current when any material Atlas collection scope, successful release, or project direction changes."
  ],
  "assistant_start_here": [
    "Read the current source of truth at docs/atlas_project_brief.json (also published as /project/brief.json) before continuing Atlas. This file captures the user's chosen goal and dated decisions; it is not live telemetry.",
    "Read the live GitHub Actions run outcomes for workflows referenced here. Do not repeat historical 'running' or 'missing' counts as current without checking completion logs and artifacts.",
    "Fetch source files and the existing workflow code before proposing a duplicate new collector; prefer actual source download and integration over audits or project-planning output.",
    "Never claim new source-native counts are additive to historical canonical records, or that a successful artifact is published to Cloudflare or the production website.",
    "After implementing material work, update the project brief with links and verified outcomes as part of the same PR, and verify the site build. If the live website cannot be checked, explicitly say deployment is unverified."
  ]
}