NICAR 2026

From Vibes to Scores

Jonathan Soma, Columbia University

Build your own AI benchmark.

js4571@columbia.edu  •  @dangerscarf  •  Lede Program  •  jonathansoma.com
Download materials
01

AI Agents with Pydantic AI

Pydantic!

Code-along Download .ipynb
Ref: Pydantic AI
Open in Colab Read
02

Tracing with Braintrust

.

Code-along Download .ipynb
Ref: Braintrust
Open in Colab Read
03

Datasets, Scorers and Evals

Code-along Download .ipynb
Open in Colab Read
04

MCP Examples

Code-along Download .ipynb
Open in Colab Read