# Vendor evaluation scorecard

NoArt Systems · instrument A3 · https://noart.io/instruments/vendor-evaluation-scorecard/ · CC BY 4.0

## Rules

1. Fix the weights before any proposal is read, and write down why.
2. Score evidence, not promises. A confident paragraph is a one, not a four.
3. Write the rationale before the number.
4. Score independently, then reconcile. Differences of two points or more are discussed with the evidence open; the card records the reconciled score and any dissent.

## Evidence scale

| Score | Name | Meaning |
|---|---|---|
| 0 | Absent | Not addressed, or addressed in a way that contradicts the requirement. |
| 1 | Asserted | Claimed in the proposal; no artifact, demonstration or reference behind it. |
| 2 | Described | A document, diagram or plan exists and was read; plausible but untested. |
| 3 | Demonstrated | Shown working, or confirmed by a reference the reviewer spoke to, for a situation like ours. |
| 4 | Proven at our scale | Demonstrated, and with evidence it holds at our volume, constraints and data. |

## Criteria

| Ref | Criterion | The question | What counts as evidence | Default weight |
|---|---|---|---|---|
| C1 | Fit to the stated requirements | Does the proposal answer the requirements brief as written, requirement by requirement, including the ones it does not meet? | A traceability table from our numbered requirements to their response; explicit gaps named by the vendor. | 20 |
| C2 | Architecture and its limits | Is the architecture described well enough to review, with its limits stated: volumes, latency, failure modes, dependency outages? | An architecture document with the limits in it; a walkthrough with an engineer who built it. | 15 |
| C3 | Data handling and ownership | Where does our data live, who can read it, how do we get all of it back, what does the vendor keep when we leave? | Contract language, a data-flow diagram, an export we have actually run on sample data. | 15 |
| C4 | Delivery plan and team | Who exactly will do the work, in what sequence, with which milestones we can verify? | Named people with availability; milestones defined as artifacts or tests, not dates alone. | 15 |
| C5 | Operability after handover | Can our people run, change and extend it without the vendor? What does a change cost after go-live? | Runbooks, infrastructure as code, a change demonstrated by someone outside the vendor's core team. | 10 |
| C6 | Security and compliance posture | What controls exist, who has verified them, how do incidents reach us? | A current independent assessment, the incident process in writing, evidence it has been exercised. | 10 |
| C7 | Total cost and its drivers | What will this cost over the life we expect, and which usage or decision of ours drives it up? | A cost model we can change the inputs of; the pricing clauses that bite; the price of leaving. | 10 |
| C8 | Independence of claims | How much rests on the vendor's own word, and how much have we checked? | The share of cells scored three or four; references chosen by us. | 5 |

Weighted score = sum(score × weight) / 4, so the maximum is 100. The decision memo quotes the rationale column, not the total.

## The card (one per vendor, per reviewer; then one reconciled card per vendor)

Vendor: · Proposal version and date: · Reviewer: · Date scored: · Requirements brief version:

| Ref | Criterion | Weight | Evidence seen (one sentence) | Score 0–4 | Weighted |
|---|---|---|---|---|---|
| C1 | Fit to the stated requirements | 20 | | | |
| C2 | Architecture and its limits | 15 | | | |
| C3 | Data handling and ownership | 15 | | | |
| C4 | Delivery plan and team | 15 | | | |
| C5 | Operability after handover | 10 | | | |
| C6 | Security and compliance posture | 10 | | | |
| C7 | Total cost and its drivers | 10 | | | |
| C8 | Independence of claims | 5 | | | |
| | Total | 100 | Dissent, if any: | | / 100 |