arthur_bench.scoring.exact_match.ExactMatch#
- class arthur_bench.scoring.exact_match.ExactMatch(case_sensitive=True)#
Returns 1 if candidate matches reference, 0 if candidate does not match reference.
- __init__(case_sensitive=True)#
Methods
__init__([case_sensitive])arun(candidate_outputs[, reference_outputs, ...])Async version of run method.
arun_batch(candidate_batch[, ...])Async version of run_batch method.
All possible values returned by the scorer if output type is categorical.
from_dict(config)Load a scorer from a json configuration file.
Whether the scorer is continuous or categorical.
name()Get the name of this Scorer :return: the Scorer name
requires_reference()True if scorer requires reference output to compute score, False otherwise
run(candidate_outputs[, reference_outputs, ...])Score a set of test cases.
run_batch(candidate_batch[, ...])Score a batch of candidate generations.
to_dict([warn])Provides a json serializable representation of the scorer.
to_metadata()type()Supplies whether a scorer is built-in or custom.