Economic History Benchmark — TMK Research Lab

Project Overview

Frontier AI browsing systems can answer many questions by locating obvious keywords or matching information from highly ranked webpages. This project explores how to construct a research question that cannot be solved through a single search. The answer must remain short, stable and objectively verifiable, while the clues require the researcher to connect different people, dates, places, organisations, publications, events, records and quantities.

Research Goal

Design one difficult but fair browsing challenge.

Required Answer

A short, stable and objectively verifiable fact.

Evidence Standard

Every clue must be independently supported by reliable evidence.

Final Deliverable

A question, answer, clue set, source trail, validation log and JSON record.

AIBRC-009

The Research Challenge

Challenge ID: AIBRC-009
Research domain: Economic History and Central Banking Records
Difficulty level: Advanced
Date created: 4 September 2026
Last verified: 4 September 2026
Expected answer type: Short text
Research question: A United States central-banking statute was signed into law on 23 December 1913 and divided the country into twelve districts, each with its own regional bank. All twelve of those banks opened on the same day, roughly seven months after the committee that drew the districts announced which cities had been selected. On what date did they open?
Correct answer: 16 November 1914
Answer-normalisation rules: Case-insensitive; punctuation ignored. Accepted variations: 16 November 1914; November 16, 1914; 16/11/1914; 1914-11-16; Monday 16 November 1914.
Confidence level: High — stated by the system's own history service, by a member bank, and by a contemporaneous Treasury announcement
Challenge status: Complete and verified

Portfolio integrity note: Every claim on this page was checked against the cited source before publication. No source, quotation, locator or search result is invented; where a detail could not be verified directly it is described rather than quoted.

Research Method

Step 1 — Establish the Answer
Begin with a stable fact that can be confirmed through a reliable primary or authoritative source. Record the exact wording, document title, issuing organisation and publication date.
Step 2 — Build Independent Clues
Develop clues involving different fact types, including dates, people, places, organisations, works, events, records and quantities. Each clue must be independently checkable.
Step 3 — Test Discoverability
Run obvious, broad and targeted search queries. Record the results to identify accidental shortcuts, direct answer leakage and misleading pages.
Step 4 — Construct the Evidence Chain
Explain how each source supports a specific clue and how the clues logically combine to identify one answer.
Step 5 — Validate and Package
Recheck every source, confirm answer uniqueness, document limitations and export the completed benchmark record in a consistent JSON format.

Clue Matrix

Each clue constrains the answer from a different direction. No single row is sufficient on its own, and every row can be checked against a source another researcher can reach.

Verified clue set

Statuses are editable so the matrix can be worked through row by row. Scroll sideways on small screens.

Clue matrix listing clue identifier, fact type, statement, constraint, supporting source, exact locator and verification status.
Clue ID	Fact type	Clue statement	Constraint	Supporting source	Exact locator	Verification status
C01	Legislation	The statute that created the system was signed by President Woodrow Wilson on 23 December 1913.	Fixes the institution and supplies the decoy date, which is not the date being asked for.	Federal Reserve History — Federal Reserve Act Signed into Law	Headline date line and closing paragraph	VerifiedVerification status for clue C01
C02	Quantity	Twelve regional banks were established, one for each district of the system.	Confirms the decentralised structure without naming an opening date.	Federal Reserve History — Reserve Banks Open for Business	Summary paragraph beneath the title	VerifiedVerification status for clue C02
C03	Interval	All twelve opened on the same day, roughly seven months after the organisation committee announced which cities and districts had been selected.	Allows the answer to be bracketed arithmetically from the district announcement in April 1914.	Federal Reserve History — Reserve Banks Open for Business	Paragraph beginning “Some seven months after the Reserve Bank Organization Committee…”	VerifiedVerification status for clue C03
C04	Government record	The Treasury Secretary issued a press announcement the day before, stating that he would formally announce the establishment of the banks the following morning.	Points at a dated primary document that sits one day before the answer.	FRASER (Federal Reserve Bank of St. Louis) — Announcement of Opening, Box 659	Dateline “Washington, D. C., November 15, 1914.”	VerifiedVerification status for clue C04
C05	Institutional record	The New York bank of the system began operating under Benjamin Strong with seven officers and eighty-five clerks on that same opening day.	Gives an independent institutional confirmation of the same date from a different bank's own history.	Federal Reserve Bank of New York — History of the New York Fed	Section “1914: New York Fed Opens for Business”	VerifiedVerification status for clue C05

Evidence Records

Each card records a source that was opened and read before it was cited. Excerpts are quoted only where the wording was read directly; otherwise the record describes what the source shows.

E01

Verified

Claim supported: The answer, and clues C02 and C03.
Source title: Reserve Banks Open for Business
Author / issuing institution: Federal Reserve History (Federal Reserve System)
Source type: Central bank history essay
Publication date: Continuously maintained page
Direct URL: https://www.federalreservehistory.org/essays/reserve-banks-open
Page number: n/a (web page)
Table, figure or section: Summary paragraph and the paragraph on the Reserve Bank Organization Committee
Relevant excerpt: “Some seven months after the Reserve Bank Organization Committee announced which cities and districts were selected, all twelve Reserve Banks opened on the same day, Monday, November 16, 1914.”
Researcher's interpretation: States the date, the simultaneity of the twelve openings and the seven-month interval in one place, which is what allows the answer to be derived as well as read.
Access date: 4 September 2026
Archive URL: [Not archived yet]
Classification: Primary
Reliability assessment: High — published by the system being described

E02

Verified

Claim supported: Clue C01 — the signing date used as the decoy.
Source title: Federal Reserve Act Signed into Law
Author / issuing institution: Federal Reserve History (Federal Reserve System)
Source type: Central bank history essay
Publication date: Continuously maintained page
Direct URL: https://www.federalreservehistory.org/essays/federal-reserve-act-signed
Page number: n/a (web page)
Table, figure or section: Headline date line and final paragraph
Relevant excerpt: “The Federal Reserve Act was signed by President Wilson on December 23, 1913.”
Researcher's interpretation: Establishes the competing date, so the two milestones can be separated rather than conflated.
Access date: 4 September 2026
Archive URL: [Not archived yet]
Classification: Primary
Reliability assessment: High

E03

Verified

Claim supported: Clue C04 — contemporaneous documentary confirmation one day before the answer.
Source title: Announcement of Opening, Box 659
Author / issuing institution: FRASER digital library, Federal Reserve Bank of St. Louis
Source type: Digitised contemporaneous press announcement
Publication date: 15 November 1914
Direct URL: https://fraser.stlouisfed.org/files/docs/meltzer/bogsub111514.pdf
Page number: First page of the scanned announcement
Table, figure or section: Opening paragraph
Relevant excerpt: “Secretary McAdoo will formally announce tomorrow, Monday morning, November 16th, the establishment of the Federal Reserve Banks in the twelve Federal Reserve Districts of the country.”
Researcher's interpretation: A document written at the time rather than a later account, and the strongest item in the chain because it is dated 15 November 1914 and points forward to the next morning.
Access date: 4 September 2026
Archive URL: [Not archived yet]
Classification: Primary
Reliability assessment: High — archival document held by a Reserve Bank library

E04

Verified

Claim supported: Clue C05 — independent confirmation from a second institution.
Source title: History of the New York Fed
Author / issuing institution: Federal Reserve Bank of New York
Source type: Institutional history page
Publication date: Continuously maintained page
Direct URL: https://www.newyorkfed.org/aboutthefed/history_article
Page number: n/a (web page)
Table, figure or section: 1914: New York Fed Opens for Business
Relevant excerpt: “The New York Fed opened for business under the leadership of Benjamin Strong, previously president of the Bankers Trust Company, on November 16, 1914.”
Researcher's interpretation: Cross-checks the system-wide claim against a single bank's own record, so the answer does not rest on one page.
Access date: 4 September 2026
Archive URL: [Not archived yet]
Classification: Primary
Reliability assessment: High

Search Validation Record

A strong benchmark documents the searches a capable researcher or browsing agent would naturally attempt, including the ones that exposed the answer. Leakage is recorded rather than hidden, and the question is rewritten around it.

Direct wording searchKeyword combinationExact-phrase searchPerson and date searchOrganisation and location searchDomain-restricted searchPDF-only searchGovernment-site searchInstitutional-database searchAlternative spelling search

5 searches run and recorded

Search validation record.
Search number	Exact search query	Search engine or database	Date searched	Top relevant results	Answer revealed?	Problem discovered	Action taken
S01	when was the Federal Reserve created	Google	4 September 2026	Returns 23 December 1913 immediately, from the system's own centennial page — the decoy, not the answer.	No	Confirms the obvious query resolves to the wrong milestone.	Kept; the question deliberately asks for the operational date.
S02	date the twelve Federal Reserve Banks opened for business	Web search	4 September 2026	Exposes 16 November 1914 in the first result summary, and also surfaced the 1914 Treasury announcement held on FRASER.	Yes	A direct wording search answers the task in one step. This is a disclosed shortcut, not a hidden one.	The published question avoids the words “Federal Reserve”, “Reserve Banks” and “opened”, describing the 1913 statute, the twelve districts and the seven-month interval instead.
S03	central bank statute December 1913 twelve districts banks began operating date	Google	4 September 2026	Mostly returns the 1913 signing and general system overviews; the operational date appears only after opening an institutional history page.	No	None — this is the intended solving path.	Retained as the reference path.
S04	Reserve Bank Organization Committee districts announced April 1914	Google	4 September 2026	Reaches the district-selection material, which supports the seven-month interval used in clue C03.	No	None; it supports the arithmetic cross-check without giving the date.	Retained.
S05	McAdoo press announcement November 15 1914 establishment of the banks	Web search	4 September 2026	Reaches the FRASER scan used as E03.	Yes	Only exposes the answer to a solver who already knows the Secretary's name and the date.	Documented; neither detail appears in the question.

Source Quality Assessment

Every candidate source is scored from 1 to 5 against five criteria before it is allowed into the evidence chain.

1Authority
3 / 5
Who created or issued the information?
2Proximity
3 / 5
Is the source directly connected to the event or record?
3Specificity
3 / 5
Does it support the precise claim being made?
4Stability
3 / 5
Is the information unlikely to change?
5Traceability
3 / 5
Can another researcher locate the same evidence?

Evidence-Chain Audit

Export a report that maps every clue to the source it rests on, records the verification status and access date of each source, and lists the searches that exposed the answer.

Claims audited: 5
Sources in the chain: 4
Disclosed shortcuts: 2

Structured Benchmark Record

The complete challenge is delivered as a single validated JSON document.

aibrc-009-benchmark-record.json

{
  "challenge_id": "AIBRC-009",
  "title": "Economic History Benchmark",
  "project_type": "independent_demonstration",
  "research_domain": "Economic History and Central Banking Records",
  "difficulty": "advanced",
  "question": "A United States central-banking statute was signed into law on 23 December 1913 and divided the country into twelve districts, each with its own regional bank. All twelve of those banks opened on the same day, roughly seven months after the committee that drew the districts announced which cities had been selected. On what date did they open?",
  "accepted_answers": [
    "16 November 1914"
  ],
  "answer_type": "short_text",
  "answer_normalisation": {
    "case_sensitive": false,
    "ignore_punctuation": true,
    "accepted_variations": [
      "16 November 1914",
      "November 16, 1914",
      "16/11/1914",
      "1914-11-16",
      "Monday 16 November 1914"
    ]
  },
  "clues": [
    {
      "clue_id": "C01",
      "fact_type": "legislation",
      "statement": "The statute that created the system was signed by President Woodrow Wilson on 23 December 1913.",
      "constraint": "Fixes the institution and supplies the decoy date, which is not the date being asked for.",
      "source": "Federal Reserve History — Federal Reserve Act Signed into Law",
      "locator": "Headline date line and closing paragraph",
      "status": "verified"
    },
    {
      "clue_id": "C02",
      "fact_type": "quantity",
      "statement": "Twelve regional banks were established, one for each district of the system.",
      "constraint": "Confirms the decentralised structure without naming an opening date.",
      "source": "Federal Reserve History — Reserve Banks Open for Business",
      "locator": "Summary paragraph beneath the title",
      "status": "verified"
    },
    {
      "clue_id": "C03",
      "fact_type": "interval",
      "statement": "All twelve opened on the same day, roughly seven months after the organisation committee announced which cities and districts had been selected.",
      "constraint": "Allows the answer to be bracketed arithmetically from the district announcement in April 1914.",
      "source": "Federal Reserve History — Reserve Banks Open for Business",
      "locator": "Paragraph beginning “Some seven months after the Reserve Bank Organization Committee…”",
      "status": "verified"
    },
    {
      "clue_id": "C04",
      "fact_type": "government record",
      "statement": "The Treasury Secretary issued a press announcement the day before, stating that he would formally announce the establishment of the banks the following morning.",
      "constraint": "Points at a dated primary document that sits one day before the answer.",
      "source": "FRASER (Federal Reserve Bank of St. Louis) — Announcement of Opening, Box 659",
      "locator": "Dateline “Washington, D. C., November 15, 1914.”",
      "status": "verified"
    },
    {
      "clue_id": "C05",
      "fact_type": "institutional record",
      "statement": "The New York bank of the system began operating under Benjamin Strong with seven officers and eighty-five clerks on that same opening day.",
      "constraint": "Gives an independent institutional confirmation of the same date from a different bank's own history.",
      "source": "Federal Reserve Bank of New York — History of the New York Fed",
      "locator": "Section “1914: New York Fed Opens for Business”",
      "status": "verified"
    }
  ],
  "evidence": [
    {
      "evidence_id": "E01",
      "claim_supported": "The answer, and clues C02 and C03.",
      "source_title": "Reserve Banks Open for Business",
      "issuing_organisation": "Federal Reserve History (Federal Reserve System)",
      "source_type": "Central bank history essay",
      "url": "https://www.federalreservehistory.org/essays/reserve-banks-open",
      "publication_date": "Continuously maintained page",
      "page_number": "n/a (web page)",
      "section": "Summary paragraph and the paragraph on the Reserve Bank Organization Committee",
      "access_date": "4 September 2026",
      "reliability": "High — published by the system being described",
      "verification_status": "verified"
    },
    {
      "evidence_id": "E02",
      "claim_supported": "Clue C01 — the signing date used as the decoy.",
      "source_title": "Federal Reserve Act Signed into Law",
      "issuing_organisation": "Federal Reserve History (Federal Reserve System)",
      "source_type": "Central bank history essay",
      "url": "https://www.federalreservehistory.org/essays/federal-reserve-act-signed",
      "publication_date": "Continuously maintained page",
      "page_number": "n/a (web page)",
      "section": "Headline date line and final paragraph",
      "access_date": "4 September 2026",
      "reliability": "High",
      "verification_status": "verified"
    },
    {
      "evidence_id": "E03",
      "claim_supported": "Clue C04 — contemporaneous documentary confirmation one day before the answer.",
      "source_title": "Announcement of Opening, Box 659",
      "issuing_organisation": "FRASER digital library, Federal Reserve Bank of St. Louis",
      "source_type": "Digitised contemporaneous press announcement",
      "url": "https://fraser.stlouisfed.org/files/docs/meltzer/bogsub111514.pdf",
      "publication_date": "15 November 1914",
      "page_number": "First page of the scanned announcement",
      "section": "Opening paragraph",
      "access_date": "4 September 2026",
      "reliability": "High — archival document held by a Reserve Bank library",
      "verification_status": "verified"
    },
    {
      "evidence_id": "E04",
      "claim_supported": "Clue C05 — independent confirmation from a second institution.",
      "source_title": "History of the New York Fed",
      "issuing_organisation": "Federal Reserve Bank of New York",
      "source_type": "Institutional history page",
      "url": "https://www.newyorkfed.org/aboutthefed/history_article",
      "publication_date": "Continuously maintained page",
      "page_number": "n/a (web page)",
      "section": "1914: New York Fed Opens for Business",
      "access_date": "4 September 2026",
      "reliability": "High",
      "verification_status": "verified"
    }
  ],
  "search_validation": [
    {
      "search_id": "S01",
      "query": "when was the Federal Reserve created",
      "platform": "Google",
      "search_date": "4 September 2026",
      "relevant_results": "Returns 23 December 1913 immediately, from the system's own centennial page — the decoy, not the answer.",
      "answer_exposed": false,
      "notes": "Confirms the obvious query resolves to the wrong milestone. Kept; the question deliberately asks for the operational date."
    },
    {
      "search_id": "S02",
      "query": "date the twelve Federal Reserve Banks opened for business",
      "platform": "Web search",
      "search_date": "4 September 2026",
      "relevant_results": "Exposes 16 November 1914 in the first result summary, and also surfaced the 1914 Treasury announcement held on FRASER.",
      "answer_exposed": true,
      "notes": "A direct wording search answers the task in one step. This is a disclosed shortcut, not a hidden one. The published question avoids the words “Federal Reserve”, “Reserve Banks” and “opened”, describing the 1913 statute, the twelve districts and the seven-month interval instead."
    },
    {
      "search_id": "S03",
      "query": "central bank statute December 1913 twelve districts banks began operating date",
      "platform": "Google",
      "search_date": "4 September 2026",
      "relevant_results": "Mostly returns the 1913 signing and general system overviews; the operational date appears only after opening an institutional history page.",
      "answer_exposed": false,
      "notes": "None — this is the intended solving path. Retained as the reference path."
    },
    {
      "search_id": "S04",
      "query": "Reserve Bank Organization Committee districts announced April 1914",
      "platform": "Google",
      "search_date": "4 September 2026",
      "relevant_results": "Reaches the district-selection material, which supports the seven-month interval used in clue C03.",
      "answer_exposed": false,
      "notes": "None; it supports the arithmetic cross-check without giving the date. Retained."
    },
    {
      "search_id": "S05",
      "query": "McAdoo press announcement November 15 1914 establishment of the banks",
      "platform": "Web search",
      "search_date": "4 September 2026",
      "relevant_results": "Reaches the FRASER scan used as E03.",
      "answer_exposed": true,
      "notes": "Only exposes the answer to a solver who already knows the Secretary's name and the date. Documented; neither detail appears in the question."
    }
  ],
  "quality_checks": {
    "The answer is short and objectively verifiable.": true,
    "The answer is supported by an authoritative source.": true,
    "The question has only one defensible answer.": true,
    "No single clue directly exposes the answer.": true,
    "Each clue is independently checkable.": true,
    "The clue set spans multiple fact types.": true,
    "The task requires multi-step reasoning.": true,
    "Obvious searches have been documented.": true,
    "Search-result snippets do not reveal the answer.": false,
    "Sources are publicly accessible.": true,
    "Exact pages, tables or sections are recorded.": true,
    "The evidence chain can be reproduced.": true,
    "The question avoids subjective interpretation.": true,
    "The answer is stable over time.": true,
    "No fabricated evidence is included.": true
  },
  "researcher": {
    "name": "TMK Research Lab",
    "portfolio": "https://theophilusmorkporkpor.lovable.app",
    "email": "tkudjordjie@gmail.com"
  }
}

Skills Demonstrated

Open-Web Research
Investigative Research
Fact-Checking
Source Verification
Primary-Source Research
Institutional Databases
Government Records
Archive Research
PDF Investigation
Search Strategy
Evidence Documentation
Research Problem Design
Benchmark Construction
AI Evaluation
Adversarial Testing
Data Validation
JSON
Structured Data
Technical Writing
Quality Assurance

Research Tools

Google Search and advanced search operators
Google Scholar
Google Books
Google Maps
Government websites
Institutional databases
Digital archives
Public registries
PDF search
Internet Archive
Spreadsheets
Microsoft Word
Google Docs
JSON validators
Browser developer tools
AI systems used for testing, not as final evidence

AI-generated statements are never treated as evidence. Every factual claim must be verified against an accessible external source.

Project Outcome

This benchmark is complete: the answer is fixed, 4 evidence records are attached to specific clues, and 5 validation searches are documented, including those that exposed the answer. It demonstrates the ability to approach an unfamiliar subject systematically, separate evidence from assumption and document research so that another person can audit it.

Reflection and Limitations

What made the question difficult

The legal founding date is one of the most heavily indexed facts in economic history, and it is written into the question as a clue, so a careless solver simply repeats it.

Which searches failed

Any query phrased around the creation of the system returned December 1913 and never the opening day.

Which source proved decisive

The 1914 Treasury press announcement held in the St. Louis Reserve Bank's digital library, because it was written the day before the event rather than decades after it.

How ambiguity was removed

The question fixes the twelve districts and the simultaneous opening, which rules out later branch openings and the first meetings of the board.

What an AI browsing agent might get wrong

Answering 23 December 1913, giving the district-announcement date in April 1914, or dating the system from the first board meeting.

Remaining limitations

A direct query using the words the question avoids exposes the answer in one step; search S02 records that shortcut openly.

Recommended future improvements

Ask for the number of officers and clerks on the New York bank's opening-day staff, which forces the solver into a single institution's own history rather than a system-level summary.

Need a careful researcher for complex web investigations?

I am available for web research, fact-checking, AI evaluation, benchmark construction, source verification and structured research documentation.

Start a research enquiry View My Resume Email Me

TMK Research Labtkudjordjie@gmail.com https://theophilusmorkporkpor.lovable.app https://www.linkedin.com/in/theophilus-kudzordzi

Project Overview

Research Goal

Required Answer

Evidence Standard

Final Deliverable

The Research Challenge

Research Method

Step 1 — Establish the Answer

Step 2 — Build Independent Clues

Step 3 — Test Discoverability

Step 4 — Construct the Evidence Chain

Step 5 — Validate and Package

Clue Matrix

Evidence Records

Search Validation Record

Source Quality Assessment

1Authority

2Proximity

3Specificity

4Stability

5Traceability

Evidence-Chain Audit

Benchmark Quality Checks

Structured Benchmark Record

Skills Demonstrated

Research Tools

Project Outcome

Reflection and Limitations

What made the question difficult

Which searches failed

Which source proved decisive

How ambiguity was removed

What an AI browsing agent might get wrong

Remaining limitations

Recommended future improvements

Need a careful researcher for complex web investigations?