# Trump Admin Actions

An LLM that predicts what the Trump administration will do next

**+29%** better calibration than baseline on political outcomes

**682** held-out test questions evaluated on live political events

## What we did

- Collected timestamped administration statements, executive orders, and news coverage.
- Generated forecasting questions about policy actions, court rulings, and legislative outcomes.
- Labeled each with the actual subsequent action or decision.
- Fine-tuned a specialized model and evaluated on 682 held-out political questions vs. GPT-5.

---

## Example datapoint

A sample training example — question, source, and outcome-derived label.

### DATASET Policy actions

#### Question
Will the Jan 27, 2025 executive order threatening 25% tariffs on Canadian imports face a successful federal court injunction or vacatur within 90 days of signing?

#### Question source
New York Times Jan 27, 2025

Trump Threatens Canada With 25% Tariffs Over Border Security

#### Label
Yes.

#### Type
binary

#### Confidence
0.92

#### Label source
Reuters Apr 22, 2025

Federal judge enjoins enforcement of 25% Canadian import tariffs pending merits briefing

### DATASET Policy actions

#### Question
What is the probability that EPA’s March 2025 light-duty vehicle emissions standard is withdrawn or replaced via notice-and-comment rulemaking before the Nov 3, 2026 midterm elections?

#### Question source
The Hill Mar 10, 2025

Administration officials defend rule as final after interagency review

#### Label
0.34

#### Type
continuous

#### Confidence
0.77

#### Label source
Federal Register Sep 2, 2025

Agency publishes notice of proposed rulemaking to replace prior standard

### DATASET Policy actions

#### Question
Will the Senate pass the FY2026 National Defense Authorization Act conference report with at least 60 votes before the Aug 2025 recess, given current Armed Services Committee markup timing?

#### Question source
Defense News Jun 9, 2025

SASC chair targets floor vote by late July after UAV procurement fight

#### Label
Yes.

#### Type
binary

#### Confidence
0.88

#### Label source
U.S. Senate Jul 22, 2025

NDAA FY2026 passes 68–29 after Ukraine aid title compromise

---

## Results

Benchmark comparisons against frontier models.

### Calibration on 682 Political Test Questions

Trump-Forecaster achieves lower Expected Calibration Error than GPT-5 in both context-aware (ECE 0.079 vs. 0.091) and context-free (ECE 0.164 vs. 0.191) settings — 13–14% better calibration, with the largest gains when no additional context is available.

---