arthur_bench.scoring.word_count_match.WordCountMatch

class arthur_bench.scoring.word_count_match.WordCountMatch

Calculates how similar the number of words in the candidate output is to the the number of words in the reference output. Scores span from 0 to 1. A score of 1.0 indicates that there are the same number of words in the candidate output as in the reference output. Scores less than 1.0 are calculated as ((len_reference-delta)/len_reference) where delta is the absolute difference in word lengths between the candidate and reference outputs. All negative computed values are truncated to 0. Utilizes lexicon count, removing punctuations: https://pypi.org/project/textstat/

__init__()

Methods

__init__()
arun(candidate_outputs[, reference_outputs, ...])	Async version of run method.
arun_batch(candidate_batch[, ...])	Async version of run_batch method.
categories()	All possible values returned by the scorer if output type is categorical.
from_dict(config)	Load a scorer from a json configuration file.
is_categorical()	Whether the scorer is continuous or categorical.
name()	Get the name of this Scorer :return: the Scorer name
requires_reference()	True if scorer requires reference output to compute score, False otherwise
run(candidate_outputs[, reference_outputs, ...])	Score a set of test cases.
run_batch(candidate_batch[, ...])	Score a batch of candidate generations.
to_dict([warn])	Provides a json serializable representation of the scorer.
to_metadata()
type()	Supplies whether a scorer is built-in or custom.