Quality Score

The quality endpoint computes a writing-quality score for a given text.

The base score (1–5) is based on how "surprising" the text is to a language model trained on well-formed text. If a text is highly surprising, that often indicates it contains errors or is a low-quality text, so high surprisal corresponds to a lower score; text that is not surprising to the model gets a higher score. This score mostly measures syntax and grammaticality and is not intended to evaluate factors such as the originality of a text. Because it is model-based, fine-tuning can often greatly improve performance for a specific domain.

Two optional measurements make the score actionable:

sentence_scores — the same 1–5 score for each sentence, with character offsets, so you can find the weakest sentences in a document. Computed from the same model pass as the base score.
rubric — an LLM-judged rubric: an overall score, scores on four dimensions (clarity, coherence, correctness, concision), a short summary, and a list of concrete issues, each quoting the problematic excerpt with its offsets, a note, and a suggested rewrite. The rubric is billed at the Tone rate (see Pricing); the base score and sentence_scores are not.

Try it out

Sample Code

cURL
JavaScript
Python
Python (Sapling client)

curl -X POST https://api.sapling.ai/api/v1/quality \
     -H "Content-Type: application/json" \
     -d '{"key":"<api-key>", "text":"Thanks for reaching out about your order. In order to process the refund we will need for you to provide us with the order number. Once we recieve it, the refund will be issued within 3-5 business days.", "sentence_scores": true, "rubric": true}'

import axios from 'axios';

async function run(text) {
    try {
        const response = await axios.post(
            'https://api.sapling.ai/api/v1/quality',
            {
                key: '<api-key>',
                text,
                sentence_scores: true,
                rubric: true,
            },
        );
        const {status, data} = response;
        console.log({status});
        console.log(JSON.stringify(data, null, 4));
    } catch (err) {
        const msg = err.response?.data?.msg || err.message;
        console.log({err: msg});
    }
}

run('Thanks for reaching out about your order. In order to process the refund we will need for you to provide us with the order number. Once we recieve it, the refund will be issued within 3-5 business days.');

import requests
from pprint import pprint

response = requests.post(
    "https://api.sapling.ai/api/v1/quality",
    json={
        "key": "<api-key>",
        "text": "Thanks for reaching out about your order. In order to process the refund we will need for you to provide us with the order number. Once we recieve it, the refund will be issued within 3-5 business days.",
        "sentence_scores": True,
        "rubric": True,
    }
)

if 200 <= response.status_code < 300:
    pprint(response.json())
else:
    print('Error: ', response.status_code, response.text)

from sapling import SaplingClient

api_key = '<api-key>'
client = SaplingClient(api_key=api_key)
result = client.quality(
    'Thanks for reaching out about your order. In order to process the refund we will need for you to provide us with the order number. Once we recieve it, the refund will be issued within 3-5 business days.',
    sentence_scores=True,
    rubric=True,
)
print(result['score'], result['rubric']['overall'], result['rubric']['dimensions'])
weakest = min(result['sentences'], key=lambda s: s['score'])
print('Weakest sentence:', weakest['text'], weakest['score'])
for issue in result['rubric']['issues']:
    print(issue['dimension'], '|', issue['quote'], '->', issue['suggestion'])

Sample Response

{
    "score": 3.41,
    "sentences": [
        {"start": 0, "end": 41, "text": "Thanks for reaching out about your order.", "score": 4.62},
        {"start": 42, "end": 130, "text": "In order to process the refund we will need for you to provide us with the order number.", "score": 3.05},
        {"start": 131, "end": 202, "text": "Once we recieve it, the refund will be issued within 3-5 business days.", "score": 2.71}
    ],
    "rubric": {
        "overall": 3,
        "dimensions": {
            "clarity": 4,
            "coherence": 4,
            "correctness": 3,
            "concision": 2
        },
        "summary": "A polite, well-organized reply, but the middle sentence is padded and there is a spelling error.",
        "issues": [
            {
                "quote": "In order to process the refund we will need for you to provide us with the order number.",
                "dimension": "concision",
                "note": "Wordy phrasing (\"in order to\", \"we will need for you to provide us with\").",
                "suggestion": "To process the refund, please send us your order number.",
                "start": 42,
                "end": 130
            },
            {
                "quote": "recieve",
                "dimension": "correctness",
                "note": "Misspelling.",
                "suggestion": "receive",
                "start": 139,
                "end": 146
            }
        ]
    }
}

Without sentence_scores and rubric, the response contains only score.

Request Parameters

POST to https://api.sapling.ai/api/v1/quality

key: String
32-character API key. Can also be supplied via the Authorization header as a bearer token; if both are provided, the key parameter takes precedence.

text: String
Text to score. The base score and sentence_scores are computed on the leading ~4,000 characters of the text (cut at a word boundary); the rubric evaluates the whole text and accepts up to 20,000 characters (longer text returns a 400).

sentence_scores: Boolean
Whether to also return a score for each sentence. Defaults to false. Sentences are split on line breaks and sentence boundaries; very short fragments are merged with a neighboring sentence on the same line.

rubric: Boolean
Whether to also return the LLM-judged rubric (overall and per-dimension scores, summary and issues). Defaults to false. Adds a few seconds of latency and is billed at the Tone rate.

Response Parameters

score: Float
A score from 1 to 5, with 1 being the lowest (low quality) and 5 the highest (high quality).

Note that partial sentences such as I will go to will have a high score as the model assumes that the sequence has not yet been completed. On the other hand, I will go to. (including the period) will have a low score.

sentences: List
Only present when sentence_scores is true. One item per sentence, in document order, each with:

start, end: character offsets of the sentence into the submitted text (text[start:end] is the sentence). Offsets are code-point indices; null in the rare case the sentence can't be located in the submitted text.
text: the sentence text.
score: the sentence's own 1–5 score, computed the same way as the top-level score over that sentence's tokens. Short sentences score a little more leniently than long ones, so compare sentences of similar length, or look at the lowest-scoring sentences first.

rubric: Object
Only present when rubric is true:

overall: integer 1–5, the evaluator's holistic judgment (not an average of the dimensions, and independent of the model-based score).
dimensions: integer 1–5 for each of:

Dimension	What it measures
`clarity`	Easy to understand; precise, unambiguous wording
`coherence`	Logical flow and organization; sentences and paragraphs connect
`correctness`	Grammar, spelling, punctuation and mechanics
`concision`	No filler, redundancy or padding

summary: one or two sentences on the writing's main strengths and weaknesses.
issues: up to 10 concrete issues, most important first (an empty list means the evaluator found nothing to flag). Each has quote (a verbatim excerpt from the text), dimension (one of the four above), note (what's wrong), suggestion (an improved rewrite of the excerpt, or ""), and start/end offsets of the excerpt into the submitted text (null if the excerpt couldn't be located verbatim).

The rubric works for any language the underlying model handles well; the base score is calibrated for English.

Sample Code​

Sample Response​

Request Parameters​

Response Parameters​

Sample Code

Sample Response

Request Parameters

Response Parameters