Bench Documentation Home

Bench is an open source python library for evaluating generative text models for production use cases.

Get started by visiting our quickstart page, reading about key concepts, or viewing our Github.

Join the bench community on Discord.

Contents

• Bench Documentation Home
• Setup
  • Package installation
  • Choosing Local vs SaaS:
  • Local
  • SaaS (Coming Soon!)
• Quickstart
  • Environment setup
  • Creating your first test suite
  • Running your first test suite
  • View results in local UI
  • Next steps
• Scoring
  • Prompt-Based Scorers
  • Embedding-Based Scorers
  • Lexicon-Based Scorers
  • Code Evaluators
• Guides
  • Concepts
  • Creating test suites
  • Compare LLM Providers
  • Compare Prompts
  • Compare Generation Settings
  • Add Scorer Configurations
  • Custom Scoring
  • Code Evaluation
• Python API Reference
  • arthur_bench.client
  • arthur_bench.client.auth
  • arthur_bench.client.http
  • arthur_bench.client.local
  • arthur_bench.client.rest
  • arthur_bench.client.rest.admin
  • arthur_bench.client.rest.bench
  • arthur_bench.exceptions
  • arthur_bench.models
  • arthur_bench.run
  • arthur_bench.scoring
  • arthur_bench.server
  • arthur_bench.telemetry
  • arthur_bench.utils
• Contributing
  • Creating a custom scorer
• Usage Data Collection
  • Data being collected
  • Opting-out

Indices and tables

• Index

• Module Index

• Search Page