arthur_bench.scoring.scorer.Scorer#
- class arthur_bench.scoring.scorer.Scorer#
Base class for all scorers. Compute a float score for a given model generation.
- __init__()#
Methods
__init__()arun(candidate_outputs[, reference_outputs, ...])Async version of run method.
arun_batch(candidate_batch[, ...])Async version of run_batch method.
All possible values returned by the scorer if output type is categorical.
from_dict(config)Load a scorer from a json configuration file.
Whether the scorer is continuous or categorical.
name()Get the name of this Scorer :return: the Scorer name
True if scorer requires reference output to compute score, False otherwise
run(candidate_outputs[, reference_outputs, ...])Score a set of test cases.
run_batch(candidate_batch[, ...])Score a batch of candidate generations.
to_dict([warn])Provides a json serializable representation of the scorer.
type()Supplies whether a scorer is built-in or custom.