Avik Roy

Former Advisor to Marco Rubio

Kevin McCarthy

Former Speaker of the House

Salena Zito

Author and journalist

Hon. Ivan Duque Marquez

Former President of Colombia

Jackie Reses

Chair and CEO Lead Bank

Sir Niall Ferguson

Historian, Author, Senior Fellow Hoover Institute, Stanford

Fareed Zakaria

Author and host, CNN

Dr. Jordan Shlain

Physician, Private Medical

Scott Jennings

Republican Political Strategist

Kristen Soltis Anderson

Republican Pollster

Elizabeth Economy

Senior Fellow Hoover Institute, Stanford (China)

Vuk Jeremic

Former President UN General Assembly

Emmanuel Acho

Sports Analyst, former NFL player, author

Next-level expertise for next-level AI

Our systems scale the judgment of world-leading experts to help AI teams evaluate and improve their models in areas that demand careful oversight.

Book a demo

Apply to the network

Featured institutional partners

FOCUS

We focus on high-stakes AI use cases, where trusted expert guidance is essential to getting it right

News
& Current Events

Mental
Health Advice

Culture
& Society

Ethics
& Safety

Education
& Guidance

Finance
& Economics

Offerings

We scale the judgment of world-leading experts to evaluate and improve AI

Our expert-in-the-loop AI systems scale expert judgment efficiently—surgically deploying their limited time for model evaluation and beyond.

Evaluation

Data Annotation

Data Production

Custom Evaluation & RL

Leading domain experts build custom rubrics, evaluators, and prompt sets, helping clients achieve frontier-level performance in challenging domains.

Standardized Benchmark Evaluations

Third-party, expert-backed certifications that help teams credibly demonstrate their system’s conversational capabilities.

Training Data Annotation

Forum AI labels training data to help teams curate their datasets.

Retrieval Source Annotation

Forum AI integrates into your search & retrieval stack to label sources with nuanced detail, improving improve LLM prioritization and interpretation of real-time sources.

Licensed Retrieval Packs

Licensed retrieval sources to ensure you have reliable, comprehensive coverage of news and evolving topics.

SFT Data Packs

Expert-designed packs of prompt-response pairs, targeted at addressing specific gaps or issues.

Evaluating Models

We partner with teams to evaluate their models, offering bespoke support from start to finish

Expert Evaluation Reports

Comprehensive evaluation of model performance, with detailed reports and expert-backed recommendations for the team.

Benchmarks

Custom evaluation frameworks designed by our experts for your use cases.

Expert-trained LLM Judges

Access Forum AI’s judges via API, fine-tuned for your use cases, built for auto evals and reward modeling.

Evaluating Data

Prompts sets and evaluation rubrics to support internal evaluation efforts, custom built with experts for your use cases.

Training Data Annotation

Forum AI labels training data to help teams curate their datasets.

Retrieval Source Annotation

Forum AI integrates into the search & retrieval stack to label sources with nuanced details to improve LLM prioritization and interpretation of real-time sources.

Producing Data

Access Forum AI’s judges via API, fine-tuned for your use cases, built for auto evals and reward modeling.

Licensed Retrieval Packs

Licensed retrieval sources to ensure you have reliable, comprehensive coverage of news and evolving topics.

SFT Data Packs

Expert-designed packs of prompt-response pairs, targeted at addressing specific gaps or issues.

“History shows that new information technologies can lose trust and credibility even as they are adopted. AI is exceptionally vulnerable to this as the content is not generated by humans. Forum AI's approach is to bring intellectually and politically diverse human intelligence to evaluate independently the AI models and their output. I think it’s a vital initiative.”

Sir Niall FergusonHistorian, Author, Senior Fellow, Hoover Institute

Blog