A quality score for a product traded on one number: method, platform and paper

The product is bought and sold mainly on one headline measurement, checked as pass or fail against a contract specification. Buyers know the cost of that: two lots with nearly the same headline number can perform very differently in use, and nothing in the trade shows the difference before the purchase.

An industry group wanted buyers and sellers to see what the headline measurement misses: one score that compares items on overall quality and shows what drives the difference. It had a clear hypothesis that such a score could work, and provided access to a large proprietary dataset of historical test results. What it needed was the specialized capability to turn that hypothesis into a tested method, a working platform, and evidence that the score means what it claims.

Turning that hypothesis into a method starts with one question: how do measurements in different units add up to one number? Think of scoring a decathlon: ten events, timed in seconds or measured in meters, and you cannot add seconds to meters. So every event is first converted onto one common points scale, and only the points are added. A quality score has the same problem and the same answer: each measurement is first turned into credit on one common scale, and only then weighted and added.

The analogy also shows where the hard part lies. A decathlon's points tables are settled; a quality score has to decide for itself what each measurement is judged against, how much each one counts, and what to do when one is missing. Because buyers and sellers would rely on the result, each of those answers also had to be one that either side could check for itself. That set the shape of the work.

The work began with discovery, not design. A short round of market research asked one question first: do the people who would use a score recognize the problem? It looked at how buyers decide today: on the headline measurement, then on long test reports that are hard to compare, and on internal scorecards that differ from firm to firm. The split runs through the buying firm: the people who buy decide on one number, while the people who use the product judge it on others.

The findings were written up as an internal mock announcement of the score's launch, with answers to the hardest questions a skeptic would ask. That working document set down who the score is for, what it must not try to do, which decisions would be hard to reverse, and which claims the evidence could not yet support, and the rest of the work was checked against it. It set the scope: add to the headline measurement rather than replace it, keep the score small enough to teach, and earn trust before making any claim about price.

Only then was the method specified in full, building on the client's hypothesis: which measurements count, how each one earns credit, how the credits are weighted, and when an item gets no score or is rejected outright. The test data are far from normally distributed, and several measurements are strongly skewed, so a method built on averages and standard deviations would misplace many items. Credit is therefore read from an item's actual position in the reference population, or from a fixed rule on an absolute scale, and turned into value through non-linear utility functions, each studied for its own measurement: more of a good property earns less and less extra credit, some properties have a range that is simply right, and past a limit a value earns nothing. The main weights start from a survey of what market participants value; where the adopted weights depart from it, the paper says so. An item with a missing measurement is not scored, so nobody gains by leaving one out.

The method is one published specification, and anyone can recompute a score independently from it and the population's published reference points. The full dataset is processed offline, never on the website. Every chart and number in the paper is produced directly from the real test data, and the scoring is re-checked on every release.

Before circulation, the paper went through rounds of structured internal review, each from the standpoint of a different reader: buyer, seller, client and outside expert. Each finding was checked against the code, the data or the cited source, and accepted or rejected with a written reason. Uncomfortable findings are stated in the paper's summary, not left in an appendix.

Items that meet the same headline specification separate widely once the other measurements are counted, and most of that spread comes from a few measurements the specification does not cover. The paper reports this as a measurement, not a price. It also marks what the evidence already supports and what is still to be tested, chiefly whether the score tracks performance in use.

Every score can be traced to a published rule and independently recomputed by anyone, and every certificate can be verified as genuine without an account. A buyer can check the arithmetic from the published reference points onward without trusting the client. The reference points themselves rest on a single source of test data, which no outside party can yet re-derive, and the paper names that as its main limitation.