Consolidated Appeal Benchmark

Project Overview

Frontier AI browsing systems can answer many questions by locating obvious keywords or matching information from highly ranked webpages. This project explores how to construct a research question that cannot be solved through a single search. The answer must remain short, stable and objectively verifiable, while the clues require the researcher to connect different people, dates, places, organisations, publications, events, records and quantities.

Research Goal

Design one difficult but fair browsing challenge.

Required Answer

A short, stable and objectively verifiable fact.

Evidence Standard

Every clue must be independently supported by reliable evidence.

Final Deliverable

A question, answer, clue set, source trail, validation log and JSON record.

AIBRC-004

The Research Challenge

Challenge ID: AIBRC-004
Research domain: United States Supreme Court Records
Difficulty level: Advanced
Date created: 4 September 2026
Last verified: 4 September 2026
Expected answer type: Short text
Research question: Five school cases from Kansas, South Carolina, Virginia, Delaware and the District of Columbia were consolidated into a single appeal, argued once, then reargued a year later before a unanimous Supreme Court ruled in 1954. On what date was the case first argued?
Correct answer: 9 December 1952
Answer-normalisation rules: Case-insensitive; punctuation ignored. Accepted variations: 9 December 1952; December 9, 1952; 09/12/1952; 1952-12-09.
Confidence level: High — stated in the official United States Reports header
Challenge status: Complete and verified

Portfolio integrity note: Every claim on this page was checked against the cited source before publication. No source, quotation, locator or search result is invented; where a detail could not be verified directly it is described rather than quoted.

Research Method

Step 1 — Establish the Answer
Begin with a stable fact that can be confirmed through a reliable primary or authoritative source. Record the exact wording, document title, issuing organisation and publication date.
Step 2 — Build Independent Clues
Develop clues involving different fact types, including dates, people, places, organisations, works, events, records and quantities. Each clue must be independently checkable.
Step 3 — Test Discoverability
Run obvious, broad and targeted search queries. Record the results to identify accidental shortcuts, direct answer leakage and misleading pages.
Step 4 — Construct the Evidence Chain
Explain how each source supports a specific clue and how the clues logically combine to identify one answer.
Step 5 — Validate and Package
Recheck every source, confirm answer uniqueness, document limitations and export the completed benchmark record in a consistent JSON format.

Clue Matrix

Each clue constrains the answer from a different direction. No single row is sufficient on its own, and every row can be checked against a source another researcher can reach.

Verified clue set

Statuses are editable so the matrix can be worked through row by row. Scroll sideways on small screens.

Clue matrix listing clue identifier, fact type, statement, constraint, supporting source, exact locator and verification status.
Clue ID	Fact type	Clue statement	Constraint	Supporting source	Exact locator	Verification status
C01	Event	Five separate school cases from four states and the District of Columbia were brought before one court and considered together in a consolidated opinion.	Cases from Kansas, South Carolina, Virginia and Delaware were consolidated under one docket number.	United States Reports, volume 347, syllabus	Syllabus, page 483	VerifiedVerification status for clue C01
C02	Organisation	The lead appeal came up from a federal district court in Kansas and carried docket number 1.	"No. 1. Appeal from the United States District Court for the District of Kansas."	United States Reports, volume 347	Case caption above the syllabus	VerifiedVerification status for clue C02
C03	Person	The Chief Justice delivered the opinion of the Court, and the decision was unanimous.	Chief Justice Earl Warren; no dissent recorded.	Opinion of the Court	First line of the opinion	VerifiedVerification status for clue C03
C04	Publication or work	The decision is printed in volume 347 of the official reports, beginning at page 483.	Citation 347 U.S. 483 (1954).	Library of Congress scan of United States Reports volume 347	Page 483	VerifiedVerification status for clue C04
C05	Record or quantity	The case was heard twice: an initial argument, then a reargument in the following term after the Court asked for further briefing.	Two hearing dates precede the decision date in the report header.	United States Reports, volume 347, header line	Line beginning "Argued"	VerifiedVerification status for clue C05
C06	Place	The original record of the opinion is held in a federal archive in Washington, DC, in the records of the court itself.	Record Group 267, Records of the Supreme Court of the United States.	National Archives milestone documents entry	Citation line	VerifiedVerification status for clue C06
C07	Date	The judgment itself was handed down on 17 May 1954 — but the question asks for the earlier of the two argument dates, not this one.	Distinguishes the widely quoted decision date from the first oral argument.	United States Reports, volume 347, header line	Line beginning "Argued"	VerifiedVerification status for clue C07

Evidence Records

Each card records a source that was opened and read before it was cited. Excerpts are quoted only where the wording was read directly; otherwise the record describes what the source shows.

E01

Verified

Claim supported: Supports the answer, plus C01, C02, C04, C05 and C07.
Source title: U.S. Reports: Brown v. Board of Education, 347 U.S. 483 (1954)
Author / issuing institution: Library of Congress (official United States Reports scan)
Source type: Government records
Publication date: United States Reports volume 347, 1954 term
Direct URL: https://tile.loc.gov/storage-services/service/ll/usrep/usrep347/usrep347483/usrep347483.pdf
Page number: Page 483
Table, figure or section: Caption and syllabus header
Relevant excerpt: "NO. 1. APPEAL FROM THE UNITED STATES DISTRICT COURT FOR THE DISTRICT OF KANSAS. Argued December 9, 1952.-Reargued December 8, 1953.-Decided May 17, 1954."
Researcher's interpretation: The official report gives all three dates in one line, so the first argument date can be separated from the decision date without inference.
Access date: 4 September 2026
Archive URL: [Not archived yet]
Classification: Primary
Reliability assessment: High — official reporter of the Court's decisions

E02

Verified

Claim supported: Supports C06 — the archival location of the original opinion.
Source title: Brown v. Board of Education (1954) — Milestone Documents
Author / issuing institution: U.S. National Archives and Records Administration
Source type: Government records
Publication date: Milestone documents entry
Direct URL: https://www.archives.gov/milestone-documents/brown-v-board-of-education
Page number: n/a (web page)
Table, figure or section: Citation line
Relevant excerpt: "Brown v. Board of Education of Topeka, Opinion; May 17, 1954; Records of the Supreme Court of the United States; Record Group 267; National Archives."
Researcher's interpretation: Ties the printed report back to the physical record series, so the chain does not stop at a published volume.
Access date: 4 September 2026
Archive URL: [Not archived yet]
Classification: Primary
Reliability assessment: High — holding institution

E03

Verified

Claim supported: Supports C06 and the catalogue-level identification of the opinion.
Source title: Opinion, 1954-05-17 — National Archives Identifier 1656510
Author / issuing institution: Civil Rights Digital Library (National Archives record)
Source type: Reference databases
Publication date: Catalogue record for the 1954 opinion
Direct URL: https://crdl.usg.edu/record/nara_supreme-court_1656510
Page number: n/a (catalogue entry)
Table, figure or section: Metadata
Relevant excerpt: "Collection: Records of the Supreme Court of the United States 1772 – 2007. Title: Opinion. Date of Original: 1954-05-17. Local Identifier: NAID: 1656510"
Researcher's interpretation: Gives a stable archival identifier another researcher can use to request the same document.
Access date: 4 September 2026
Archive URL: [Not archived yet]
Classification: Secondary
Reliability assessment: Medium-high — catalogue description of a primary record

E04

Verified

Claim supported: Supports C01, C03 and C05 from the opinion text itself.
Source title: Brown v. Board of Education of Topeka — Opinion of the Court
Author / issuing institution: Wikisource transcription of the opinion
Source type: Archived webpages
Publication date: Transcription of the 1954 opinion
Direct URL: https://en.wikisource.org/wiki/Brown_v._Board_of_Education_of_Topeka_(347_U.S._483)/Opinion_of_the_Court
Page number: Report pages 486-495
Table, figure or section: Opening paragraphs
Relevant excerpt: "MR. CHIEF JUSTICE WARREN delivered the opinion of the Court. These cases come to us from the States of Kansas, South Carolina, Virginia, and Delaware... a common legal question justifies their consideration together in this consolidated opinion."
Researcher's interpretation: Searchable text of the opinion; used for wording only, with the official scan as the authority.
Access date: 4 September 2026
Archive URL: [Not archived yet]
Classification: Secondary
Reliability assessment: Medium — transcription, checked against the official scan

E05

Verified

Claim supported: Records a conflicting account of the argument dates (see reflection).
Source title: Brown v. Board of Education of Topeka, 347 U.S. 483 (1954) — case page
Author / issuing institution: Justia US Supreme Court Center
Source type: Reference databases
Publication date: Case summary page
Direct URL: https://supreme.justia.com/cases/federal/us/347/483/
Page number: n/a (web page)
Table, figure or section: Argument dates
Relevant excerpt: "Argued: December 9, 1952. Argued: December 10, 1952. Argued: December 11, 1952. Reargued: December 7, 1953... Decided: May 17, 1954."
Researcher's interpretation: Lists argument across three days, where the official report prints only the first day. Both agree the argument began on 9 December 1952, which is what the question asks for.
Access date: 4 September 2026
Archive URL: [Not archived yet]
Classification: Secondary
Reliability assessment: Medium — useful cross-check, not authoritative over the official reporter

Search Validation Record

A strong benchmark documents the searches a capable researcher or browsing agent would naturally attempt, including the ones that exposed the answer. Leakage is recorded rather than hidden, and the question is rewritten around it.

Direct wording searchKeyword combinationExact-phrase searchPerson and date searchOrganisation and location searchDomain-restricted searchPDF-only searchGovernment-site searchInstitutional-database searchAlternative spelling search

3 searches run and recorded

Search validation record.
Search number	Exact search query	Search engine or database	Date searched	Top relevant results	Answer revealed?	Problem discovered	Action taken
S01	Brown v. Board of Education decided May 17 1954 347 U.S. 483 National Archives judgment primary document	Web search	4 September 2026	National Archives milestone documents entry; Library of Congress scan of 347 U.S. 483; Justia case page; National Archives opinion highlight; PBS copy of the decision	No	Every result leads with the decision date, which is not the answer; the argument date appears only inside the report header.	Kept as the route to evidence E01 and E02.
S02	school segregation case first argued December 1952 reargued 1953 Supreme Court	Web search	4 September 2026	Case summaries and encyclopaedia entries giving argument and reargument dates	Yes	Naming the argument years narrows the field to this case immediately and some summaries print the date.	The published question never mentions December, the years, or the word reargument; it describes the consolidation and asks when the first hearing took place.
S03	"Records of the Supreme Court of the United States" Record Group 267 opinion 1954 NAID	Web search	4 September 2026	Civil Rights Digital Library catalogue record NAID 1656510; National Archives catalogue pages	No	Reaches the archival record without exposing any of the hearing dates.	Kept as the route to evidence E03.

Source Quality Assessment

Every candidate source is scored from 1 to 5 against five criteria before it is allowed into the evidence chain.

1Authority
3 / 5
Who created or issued the information?
2Proximity
3 / 5
Is the source directly connected to the event or record?
3Specificity
3 / 5
Does it support the precise claim being made?
4Stability
3 / 5
Is the information unlikely to change?
5Traceability
3 / 5
Can another researcher locate the same evidence?

Evidence-Chain Audit

Export a report that maps every clue to the source it rests on, records the verification status and access date of each source, and lists the searches that exposed the answer.

Claims audited: 7
Sources in the chain: 5
Disclosed shortcuts: 1

Structured Benchmark Record

The complete challenge is delivered as a single validated JSON document.

aibrc-004-benchmark-record.json

{
  "challenge_id": "AIBRC-004",
  "title": "Consolidated Appeal Benchmark",
  "project_type": "independent_demonstration",
  "research_domain": "United States Supreme Court Records",
  "difficulty": "advanced",
  "question": "Five school cases from Kansas, South Carolina, Virginia, Delaware and the District of Columbia were consolidated into a single appeal, argued once, then reargued a year later before a unanimous Supreme Court ruled in 1954. On what date was the case first argued?",
  "accepted_answers": [
    "9 December 1952"
  ],
  "answer_type": "short_text",
  "answer_normalisation": {
    "case_sensitive": false,
    "ignore_punctuation": true,
    "accepted_variations": [
      "9 December 1952",
      "December 9, 1952",
      "09/12/1952",
      "1952-12-09"
    ]
  },
  "clues": [
    {
      "clue_id": "C01",
      "fact_type": "event",
      "statement": "Five separate school cases from four states and the District of Columbia were brought before one court and considered together in a consolidated opinion.",
      "constraint": "Cases from Kansas, South Carolina, Virginia and Delaware were consolidated under one docket number.",
      "source": "United States Reports, volume 347, syllabus",
      "locator": "Syllabus, page 483",
      "status": "verified"
    },
    {
      "clue_id": "C02",
      "fact_type": "organisation",
      "statement": "The lead appeal came up from a federal district court in Kansas and carried docket number 1.",
      "constraint": "\"No. 1. Appeal from the United States District Court for the District of Kansas.\"",
      "source": "United States Reports, volume 347",
      "locator": "Case caption above the syllabus",
      "status": "verified"
    },
    {
      "clue_id": "C03",
      "fact_type": "person",
      "statement": "The Chief Justice delivered the opinion of the Court, and the decision was unanimous.",
      "constraint": "Chief Justice Earl Warren; no dissent recorded.",
      "source": "Opinion of the Court",
      "locator": "First line of the opinion",
      "status": "verified"
    },
    {
      "clue_id": "C04",
      "fact_type": "publication or work",
      "statement": "The decision is printed in volume 347 of the official reports, beginning at page 483.",
      "constraint": "Citation 347 U.S. 483 (1954).",
      "source": "Library of Congress scan of United States Reports volume 347",
      "locator": "Page 483",
      "status": "verified"
    },
    {
      "clue_id": "C05",
      "fact_type": "record or quantity",
      "statement": "The case was heard twice: an initial argument, then a reargument in the following term after the Court asked for further briefing.",
      "constraint": "Two hearing dates precede the decision date in the report header.",
      "source": "United States Reports, volume 347, header line",
      "locator": "Line beginning \"Argued\"",
      "status": "verified"
    },
    {
      "clue_id": "C06",
      "fact_type": "place",
      "statement": "The original record of the opinion is held in a federal archive in Washington, DC, in the records of the court itself.",
      "constraint": "Record Group 267, Records of the Supreme Court of the United States.",
      "source": "National Archives milestone documents entry",
      "locator": "Citation line",
      "status": "verified"
    },
    {
      "clue_id": "C07",
      "fact_type": "date",
      "statement": "The judgment itself was handed down on 17 May 1954 — but the question asks for the earlier of the two argument dates, not this one.",
      "constraint": "Distinguishes the widely quoted decision date from the first oral argument.",
      "source": "United States Reports, volume 347, header line",
      "locator": "Line beginning \"Argued\"",
      "status": "verified"
    }
  ],
  "evidence": [
    {
      "evidence_id": "E01",
      "claim_supported": "Supports the answer, plus C01, C02, C04, C05 and C07.",
      "source_title": "U.S. Reports: Brown v. Board of Education, 347 U.S. 483 (1954)",
      "issuing_organisation": "Library of Congress (official United States Reports scan)",
      "source_type": "Government records",
      "url": "https://tile.loc.gov/storage-services/service/ll/usrep/usrep347/usrep347483/usrep347483.pdf",
      "publication_date": "United States Reports volume 347, 1954 term",
      "page_number": "Page 483",
      "section": "Caption and syllabus header",
      "access_date": "4 September 2026",
      "reliability": "High — official reporter of the Court's decisions",
      "verification_status": "verified"
    },
    {
      "evidence_id": "E02",
      "claim_supported": "Supports C06 — the archival location of the original opinion.",
      "source_title": "Brown v. Board of Education (1954) — Milestone Documents",
      "issuing_organisation": "U.S. National Archives and Records Administration",
      "source_type": "Government records",
      "url": "https://www.archives.gov/milestone-documents/brown-v-board-of-education",
      "publication_date": "Milestone documents entry",
      "page_number": "n/a (web page)",
      "section": "Citation line",
      "access_date": "4 September 2026",
      "reliability": "High — holding institution",
      "verification_status": "verified"
    },
    {
      "evidence_id": "E03",
      "claim_supported": "Supports C06 and the catalogue-level identification of the opinion.",
      "source_title": "Opinion, 1954-05-17 — National Archives Identifier 1656510",
      "issuing_organisation": "Civil Rights Digital Library (National Archives record)",
      "source_type": "Reference databases",
      "url": "https://crdl.usg.edu/record/nara_supreme-court_1656510",
      "publication_date": "Catalogue record for the 1954 opinion",
      "page_number": "n/a (catalogue entry)",
      "section": "Metadata",
      "access_date": "4 September 2026",
      "reliability": "Medium-high — catalogue description of a primary record",
      "verification_status": "verified"
    },
    {
      "evidence_id": "E04",
      "claim_supported": "Supports C01, C03 and C05 from the opinion text itself.",
      "source_title": "Brown v. Board of Education of Topeka — Opinion of the Court",
      "issuing_organisation": "Wikisource transcription of the opinion",
      "source_type": "Archived webpages",
      "url": "https://en.wikisource.org/wiki/Brown_v._Board_of_Education_of_Topeka_(347_U.S._483)/Opinion_of_the_Court",
      "publication_date": "Transcription of the 1954 opinion",
      "page_number": "Report pages 486-495",
      "section": "Opening paragraphs",
      "access_date": "4 September 2026",
      "reliability": "Medium — transcription, checked against the official scan",
      "verification_status": "verified"
    },
    {
      "evidence_id": "E05",
      "claim_supported": "Records a conflicting account of the argument dates (see reflection).",
      "source_title": "Brown v. Board of Education of Topeka, 347 U.S. 483 (1954) — case page",
      "issuing_organisation": "Justia US Supreme Court Center",
      "source_type": "Reference databases",
      "url": "https://supreme.justia.com/cases/federal/us/347/483/",
      "publication_date": "Case summary page",
      "page_number": "n/a (web page)",
      "section": "Argument dates",
      "access_date": "4 September 2026",
      "reliability": "Medium — useful cross-check, not authoritative over the official reporter",
      "verification_status": "verified"
    }
  ],
  "search_validation": [
    {
      "search_id": "S01",
      "query": "Brown v. Board of Education decided May 17 1954 347 U.S. 483 National Archives judgment primary document",
      "platform": "Web search",
      "search_date": "4 September 2026",
      "relevant_results": "National Archives milestone documents entry; Library of Congress scan of 347 U.S. 483; Justia case page; National Archives opinion highlight; PBS copy of the decision",
      "answer_exposed": false,
      "notes": "Every result leads with the decision date, which is not the answer; the argument date appears only inside the report header. Kept as the route to evidence E01 and E02."
    },
    {
      "search_id": "S02",
      "query": "school segregation case first argued December 1952 reargued 1953 Supreme Court",
      "platform": "Web search",
      "search_date": "4 September 2026",
      "relevant_results": "Case summaries and encyclopaedia entries giving argument and reargument dates",
      "answer_exposed": true,
      "notes": "Naming the argument years narrows the field to this case immediately and some summaries print the date. The published question never mentions December, the years, or the word reargument; it describes the consolidation and asks when the first hearing took place."
    },
    {
      "search_id": "S03",
      "query": "\"Records of the Supreme Court of the United States\" Record Group 267 opinion 1954 NAID",
      "platform": "Web search",
      "search_date": "4 September 2026",
      "relevant_results": "Civil Rights Digital Library catalogue record NAID 1656510; National Archives catalogue pages",
      "answer_exposed": false,
      "notes": "Reaches the archival record without exposing any of the hearing dates. Kept as the route to evidence E03."
    }
  ],
  "quality_checks": {
    "The answer is short and objectively verifiable.": true,
    "The answer is supported by an authoritative source.": true,
    "The question has only one defensible answer.": true,
    "No single clue directly exposes the answer.": true,
    "Each clue is independently checkable.": true,
    "The clue set spans multiple fact types.": true,
    "The task requires multi-step reasoning.": true,
    "Obvious searches have been documented.": true,
    "Search-result snippets do not reveal the answer.": false,
    "Sources are publicly accessible.": true,
    "Exact pages, tables or sections are recorded.": true,
    "The evidence chain can be reproduced.": true,
    "The question avoids subjective interpretation.": true,
    "The answer is stable over time.": true,
    "No fabricated evidence is included.": true
  },
  "researcher": {
    "name": "TMK Research Lab",
    "portfolio": "https://theophilusmorkporkpor.lovable.app",
    "email": "tkudjordjie@gmail.com"
  }
}

Skills Demonstrated

Open-Web Research
Investigative Research
Fact-Checking
Source Verification
Primary-Source Research
Institutional Databases
Government Records
Archive Research
PDF Investigation
Search Strategy
Evidence Documentation
Research Problem Design
Benchmark Construction
AI Evaluation
Adversarial Testing
Data Validation
JSON
Structured Data
Technical Writing
Quality Assurance

Research Tools

Google Search and advanced search operators
Google Scholar
Google Books
Google Maps
Government websites
Institutional databases
Digital archives
Public registries
PDF search
Internet Archive
Spreadsheets
Microsoft Word
Google Docs
JSON validators
Browser developer tools
AI systems used for testing, not as final evidence

AI-generated statements are never treated as evidence. Every factual claim must be verified against an accessible external source.

Project Outcome

This benchmark is complete: the answer is fixed, 5 evidence records are attached to specific clues, and 3 validation searches are documented, including those that exposed the answer. It demonstrates the ability to approach an unfamiliar subject systematically, separate evidence from assumption and document research so that another person can audit it.

Reflection and Limitations

What made the question difficult

The case has four candidate dates — argument, reargument, decision, and the later implementation ruling — and the famous one is not the one asked for.

Which searches failed

Searching the case by name returns the decision date in almost every snippet; none of the top results surfaced the argument date without opening a document.

Which source proved decisive

The Library of Congress scan of United States Reports volume 347, whose header prints argued, reargued and decided dates on one line.

How ambiguity was removed

Clue C07 names the decision date explicitly and rules it out, so the only remaining reading is the first oral argument.

What an AI browsing agent might get wrong

Answering 17 May 1954, giving the 1953 reargument, or picking the last of the three 1952 argument days that some databases list.

Remaining limitations

The official report prints a single argument date while Justia lists 9, 10 and 11 December 1952. They agree the argument began on 9 December 1952, and the question asks when it was first argued, but a solver quoting a closing day would be marked wrong.

Recommended future improvements

Add the National Archives catalogue images of the docket sheet so the hearing days can be read from the Court's own record.

Need a careful researcher for complex web investigations?

I am available for web research, fact-checking, AI evaluation, benchmark construction, source verification and structured research documentation.

Start a research enquiry View My Resume Email Me

TMK Research Labtkudjordjie@gmail.com https://theophilusmorkporkpor.lovable.app https://www.linkedin.com/in/theophilus-kudzordzi

Project Overview

Research Goal

Required Answer

Evidence Standard

Final Deliverable

The Research Challenge

Research Method

Step 1 — Establish the Answer

Step 2 — Build Independent Clues

Step 3 — Test Discoverability

Step 4 — Construct the Evidence Chain

Step 5 — Validate and Package

Clue Matrix

Evidence Records

Search Validation Record

Source Quality Assessment

1Authority

2Proximity

3Specificity

4Stability

5Traceability

Evidence-Chain Audit

Benchmark Quality Checks

Structured Benchmark Record

Skills Demonstrated

Research Tools

Project Outcome

Reflection and Limitations

What made the question difficult

Which searches failed

Which source proved decisive

How ambiguity was removed

What an AI browsing agent might get wrong

Remaining limitations

Recommended future improvements

Need a careful researcher for complex web investigations?