Vendor evaluation scorecard
NoArt Systems' own scorecard for when proposals are on the table. Eight criteria, each scored on a five-point evidence scale; weights agreed by the buying group before the first proposal is opened; and a rationale column that has to be written, because a number without a reason cannot be audited or defended. It is published as the instrument itself, not a description of one.
Use it as it is, or change the weights and keep the rest. Attribution required; nothing else. What you write in it is yours.
↓ Download as MarkdownHow to use it
Four rules that do most of the work.
Fix the weights first. The buying group agrees the weight of each criterion before any proposal is read, and writes down why. Weights changed after reading a proposal are a common way a selection is quietly decided by one person.
Score evidence, not promises. The scale below rewards what a vendor can show: a running system, a reference you can call, a document you can read. A confident paragraph in a proposal is a one, not a four.
Write the rationale before the number. Each cell gets a sentence naming the evidence, then a score. Reviewers who score first and justify later converge on their first impression.
Score independently, then reconcile. Each reviewer completes the card alone. Differences of two points or more on any cell are discussed with the evidence open; the card records the reconciled score and the dissent, if any.
The evidence scale
Zero to four. The same scale for every criterion, so scores can be compared across columns.
The eight criteria
With the question each one asks, and what counts as evidence. Default weights sum to one hundred; change them before you open a proposal.
| Ref | Criterion | The question | What counts as evidence | Default weight |
|---|---|---|---|---|
| C1 | Fit to the stated requirements | Does the proposal answer the requirements brief as written, requirement by requirement, including the ones it does not meet? | A traceability table from our numbered requirements to their response; explicit gaps named by the vendor, not found by us. | 20 |
| C2 | Architecture and its limits | Is the architecture described well enough to review, and are its limits stated: data volumes, latency, failure modes, what happens when a dependency is down? | An architecture document with the limits in it; a walkthrough with an engineer who built it, not a salesperson. | 15 |
| C3 | Data handling and ownership | Where does our data live, who can read it, how do we get all of it back, and what does the vendor keep when we leave? | Contract language, a data-flow diagram, an export we have actually run on sample data. | 15 |
| C4 | Delivery plan and team | Who exactly will do the work, in what sequence, with which milestones that we can verify rather than accept? | Named people with their availability; milestones defined as artifacts or tests, not as dates alone. | 15 |
| C5 | Operability after handover | Can our people run, change and extend it without the vendor? What does a change cost, in time and money, after go-live? | Runbooks, infrastructure as code, a demonstrated change made by someone outside the vendor's core team. | 10 |
| C6 | Security and compliance posture | What controls exist, who has verified them, and how do incidents reach us? | A current independent assessment, the incident process in writing, evidence it has been exercised. | 10 |
| C7 | Total cost and its drivers | What will this cost over the life we expect, and which usage or decision of ours drives the cost up? | A cost model we can change the inputs of; the pricing clauses that bite; the price of leaving. | 10 |
| C8 | Independence of claims | How much of the proposal rests on the vendor's own word, and how much have we been able to check? | The share of cells above scored at three or four; references chosen by us, not supplied as a list. | 5 |
The weighted score is the sum of each criterion's score multiplied by its weight, divided by four, so that the maximum is one hundred. The number is a ranking aid. The decision memo quotes the rationale column, not the total, because that is where the reasons live and where the selection can be defended a year later.
The card
One per vendor, per reviewer; then one reconciled card per vendor.
| Ref | Criterion | Weight | Evidence seen (sentence) | Score 0–4 | Weighted |
|---|---|---|---|---|---|
| C1 | Fit to the stated requirements | 20 | |||
| C2 | Architecture and its limits | 15 | |||
| C3 | Data handling and ownership | 15 | |||
| C4 | Delivery plan and team | 15 | |||
| C5 | Operability after handover | 10 | |||
| C6 | Security and compliance posture | 10 | |||
| C7 | Total cost and its drivers | 10 | |||
| C8 | Independence of claims | 5 | |||
| Total | 100 | Dissent, if any: | / 100 |
Header fields for each card: vendor, proposal version and date, reviewer, date scored, and the version of the requirements brief scored against. A card without the brief's version is not comparable to another card.
Proposals on the table and no one independent to score them?
That is the pre-selection decision point. Say how many proposals, what is being bought, and when the decision is due.