Quality Score

The quality endpoint computes a writing-quality score for a given text.

The base score (1–5) is based on how "surprising" the text is to a language model trained on well-formed text. If a text is highly surprising, that often indicates it contains errors or is a low-quality text, so high surprisal corresponds to a lower score; text that is not surprising to the model gets a higher score. This score mostly measures syntax and grammaticality and is not intended to evaluate factors such as the originality of a text. Because it is model-based, fine-tuning can often greatly improve performance for a specific domain.

Two optional measurements make the score actionable:

sentence_scores — the same 1–5 score for each sentence, with character offsets, so you can find the weakest sentences in a document. Computed from the same model pass as the base score.
rubric — an LLM-judged rubric: an overall score, scores on four dimensions (clarity, coherence, correctness, concision), a short summary, and a list of concrete issues, each quoting the problematic excerpt with its offsets, a note, and a suggested rewrite. The rubric is billed at the Tone rate (see Pricing); the base score and sentence_scores are not.

Try it out

Sample Code

cURL
JavaScript
Python
Python (Sapling client)

curl -X POST https://api.sapling.ai/api/v1/quality \
     -H "Content-Type: application/json" \
     -d '{"key":"<api-key>", "text":"Thanks for reaching out about your order. In order to process the refund we will need for you to provide us with the order number. Once we recieve it, the refund will be issued within 3-5 business days.", "sentence_scores": true, "rubric": true}'

import axios from 'axios';

async function run(text) {
    try {
        const response = await axios.post(
            'https://api.sapling.ai/api/v1/quality',
            {
                key: '<api-key>',
                text,
                sentence_scores: true,
                rubric: true,
            },
        );
        const {status, data} = response;
        console.log({status});
        console.log(JSON.stringify(data, null, 4));
    } catch (err) {
        const msg = err.response?.data?.msg || err.message;
        console.log({err: msg});
    }
}

run('Thanks for reaching out about your order. In order to process the refund we will need for you to provide us with the order number. Once we recieve it, the refund will be issued within 3-5 business days.');

import requests
from pprint import pprint

response = requests.post(
    "https://api.sapling.ai/api/v1/quality",
    json={
        "key": "<api-key>",
        "text": "Thanks for reaching out about your order. In order to process the refund we will need for you to provide us with the order number. Once we recieve it, the refund will be issued within 3-5 business days.",
        "sentence_scores": True,
        "rubric": True,
    }
)

if 200 <= response.status_code < 300:
    pprint(response.json())
else:
    print('Error: ', response.status_code, response.text)

from sapling import SaplingClient

api_key = '<api-key>'
client = SaplingClient(api_key=api_key)
result = client.quality(
    'Thanks for reaching out about your order. In order to process the refund we will need for you to provide us with the order number. Once we recieve it, the refund will be issued within 3-5 business days.',
    sentence_scores=True,
    rubric=True,
)
print(result['score'], result['rubric']['overall'], result['rubric']['dimensions'])
weakest = min(result['sentences'], key=lambda s: s['score'])
print('Weakest sentence:', weakest['text'], weakest['score'])
for issue in result['rubric']['issues']:
    print(issue['dimension'], '|', issue['quote'], '->', issue['suggestion'])

Sample Response

{
    "score": 3.41,
    "sentences": [
        {"start": 0, "end": 41, "text": "Thanks for reaching out about your order.", "score": 4.62},
        {"start": 42, "end": 130, "text": "In order to process the refund we will need for you to provide us with the order number.", "score": 3.05},
        {"start": 131, "end": 202, "text": "Once we recieve it, the refund will be issued within 3-5 business days.", "score": 2.71}
    ],
    "rubric": {
        "overall": 3,
        "dimensions": {
            "clarity": 4,
            "coherence": 4,
            "correctness": 3,
            "concision": 2
        },
        "summary": "A polite, well-organized reply, but the middle sentence is padded and there is a spelling error.",
        "issues": [
            {
                "quote": "In order to process the refund we will need for you to provide us with the order number.",
                "dimension": "concision",
                "note": "Wordy phrasing (\"in order to\", \"we will need for you to provide us with\").",
                "suggestion": "To process the refund, please send us your order number.",
                "start": 42,
                "end": 130
            },
            {
                "quote": "recieve",
                "dimension": "correctness",
                "note": "Misspelling.",
                "suggestion": "receive",
                "start": 139,
                "end": 146
            }
        ]
    }
}

Without sentence_scores and rubric, the response contains only score.

Batch Requests

To score many texts — a page of product descriptions, a queue of drafted replies — in one request, send texts (a list of 1–10 strings) instead of text. The same sentence_scores and rubric options apply to every item. Exactly one of text and texts must be provided.

curl -X POST https://api.sapling.ai/api/v1/quality \
     -H "Content-Type: application/json" \
     -d '{"key":"<api-key>", "texts": ["Thanks for reaching out about your order.", "Once we recieve it, the refund will be issued within 3-5 business days."]}'

The batch response is {"results": [...]} with one entry per input, in input order; each entry has exactly the single-response shape above (offsets in sentences and rubric issues are relative to that item's own text):

{
  "results": [
    {"score": 4.62},
    {"score": 2.71}
  ]
}

Each item is scored — and, with rubric, evaluated — independently, and billed individually (a batch of N costs the same as N single requests). With rubric, the combined length of all items may be up to 20,000 characters (the same cap as a single text); without it, each item is independently truncated to the leading ~4,000 characters like a single request. If any item fails, the whole request returns a 502 and nothing is billed; rubrics already evaluated are cached, so a retry only re-evaluates the rest.

Request Parameters

POST to https://api.sapling.ai/api/v1/quality

key: String
32-character API key. Can also be supplied via the Authorization header as a bearer token; if both are provided, the key parameter takes precedence.

text: String
Text to score. The base score and sentence_scores are computed on the leading ~4,000 characters of the text (cut at a word boundary); the rubric evaluates the whole text and accepts up to 20,000 characters (longer text returns a 400). Provide exactly one of text and texts.

texts: List[String]
Batch form (see Batch Requests): 1–10 texts to score in one request, each scored — and, with rubric, evaluated — independently under the same options.

sentence_scores: Boolean
Whether to also return a score for each sentence. Defaults to false. Sentences are split on line breaks and sentence boundaries; very short fragments are merged with a neighboring sentence on the same line.

rubric: Boolean
Whether to also return the LLM-judged rubric (overall and per-dimension scores, summary and issues). Defaults to false. Adds a few seconds of latency and is billed at the Tone rate.

Response Parameters

score: Float
A score from 1 to 5, with 1 being the lowest (low quality) and 5 the highest (high quality).

Note that partial sentences such as I will go to will have a high score as the model assumes that the sequence has not yet been completed. On the other hand, I will go to. (including the period) will have a low score.

sentences: List
Only present when sentence_scores is true. One item per sentence, in document order, each with:

start, end: character offsets of the sentence into the submitted text (text[start:end] is the sentence). Offsets are code-point indices; null in the rare case the sentence can't be located in the submitted text.
text: the sentence text.
score: the sentence's own 1–5 score, computed the same way as the top-level score over that sentence's tokens. Short sentences score a little more leniently than long ones, so compare sentences of similar length, or look at the lowest-scoring sentences first.

rubric: Object
Only present when rubric is true:

overall: integer 1–5, the evaluator's holistic judgment (not an average of the dimensions, and independent of the model-based score).
dimensions: integer 1–5 for each of:

Dimension	What it measures
`clarity`	Easy to understand; precise, unambiguous wording
`coherence`	Logical flow and organization; sentences and paragraphs connect
`correctness`	Grammar, spelling, punctuation and mechanics
`concision`	No filler, redundancy or padding

summary: one or two sentences on the writing's main strengths and weaknesses.
issues: up to 10 concrete issues, most important first (an empty list means the evaluator found nothing to flag). Each has quote (a verbatim excerpt from the text), dimension (one of the four above), note (what's wrong), suggestion (an improved rewrite of the excerpt, or ""), and start/end offsets of the excerpt into the submitted text (null if the excerpt couldn't be located verbatim).

The rubric works for any language the underlying model handles well; the base score is calibrated for English.

Sample Code​

Sample Response​

Batch Requests​

Request Parameters​

Response Parameters​

Sample Code

Sample Response

Batch Requests

Request Parameters

Response Parameters