The prototype works. Will the hundredth unit work too?

The first device works, a customer has seen the demonstration, and the team has a bill of materials and manufacturing files. It is tempting to treat the next step as ordering more of the same parts. In practice, the object of the work changes. Instead of making one assembly operate correctly, the team must create conditions under which subsequent units work without personal intervention from the designer.

That transition has three independent layers. The design must tolerate real variation, the process must reproduce the intended product, and inspection must distinguish an acceptable result from a problem. A good test cannot repair a mechanically sensitive design. Consistent assembly cannot compensate for an ambiguous requirement. A green indicator on the fixture is insufficient if the configuration being tested differs from the one delivered to the customer.

We will follow a hypothetical batch of temperature modules. All numbers are synthetic and illustrate a decision process. They are neither measurements of a BBest24 product nor a ready-made qualification procedure for an arbitrary sensor. The practical output is an acceptance record and a workflow that a team can adapt to its own first production batch.

What exactly has to be repeatable?

The article on prototyping asks what a working demonstrator actually proves. Here we assume that the function is useful and technically feasible. The question is whether we can deliver it again, in a defined configuration, with evidence of its quality.

“Measures temperature correctly” is too loose for acceptance testing. For this example, we require an indication error no greater than ±0.50°C at three check points: 10°C, 25°C and 40°C. We test a complete device after stabilization, with specified power, firmware configuration and probe placement. This is an example acceptance requirement. It does not establish accuracy throughout a continuous temperature range or under every environmental condition.

Even this definition exposes decisions. Does stabilization begin when power is applied, when the device enters the chamber, or when the reference reaches its setpoint? Is the radio active, and does its heat affect the reading? Is the lid tightened to the intended torque? If an answer changes the result, it belongs in the method. Leaving the choice to the operator introduces an uncontrolled variant of the product or the test.

Verification relates a result to stated requirements, whereas validation concerns suitability for the intended use. NASA explains this distinction in Product Verification. For a small team, the consequence is practical: passing a factory test does not automatically establish whether the sensor responds quickly enough in the customer’s warehouse.

A specification should also identify what is deliberately outside the acceptance test. This prevents a later reader from interpreting a three-point check as a complete environmental qualification. The difference is not bureaucratic. It determines which claims the team can defend and which questions still require another experiment.

Why conforming components may produce a nonconforming unit

Imagine a circuit board with a probe located close to the lid. Each component is within its own dimensional tolerance, but a particular combination of heights and spacers applies pressure when the enclosure closes. Another possible mechanism is a change in heat flow. The symptom alone cannot establish which mechanism is responsible.

The immediate response should not be to widen the acceptance limits. First separate the user requirement, design tolerances and the process’s ability to reproduce the design. If the product must meet ±0.50°C, supplier variation does not justify exceeding that limit. It may instead show that the design needs more clearance, a different mounting arrangement or control of a particular dimension.

Start a tolerance review with a sketch of the relationships: which dimension affects contact, stress or clearance? Adding maximum deviations describes an extreme combination, but does not establish its frequency. A statistical combination requires defensible assumptions about distributions and dependencies. Parts from the same tool or batch may vary together, making an independence assumption convenient but incorrect.

For an early batch, a controlled experiment can be more useful than a precise-looking reject forecast. Select representative dimensional extremes, compare two assembly settings, and keep the measurement procedure fixed. The objective is to reveal sensitivity in the design. This does not replace observation of production, but directs engineering effort towards a cause that the team can change.

The choice of extremes should be documented. Selecting only parts that are easy to assemble can accidentally reproduce the conditions of the successful prototype. A pilot batch is valuable partly because it can deliberately include combinations that the designer would otherwise avoid without noticing.

Twenty units before and after enclosure assembly

Consider 20 devices tested at one of the check points, 25°C. Board errors before assembly range from −0.14°C to +0.28°C. In the model, we add +0.18°C to the first 12 readings and +0.30°C to the remaining eight. This is an explicitly imposed change, not a fitted explanation of a measured experiment. The assembled devices then range from +0.04°C to +0.58°C.

Four results exceed +0.50°C. If inspection ended at the board stage, every unit would pass that check. This does not make the board test useless: it can detect electronic assembly errors early. It means the test does not cover the effects of operations performed afterwards.

Each vertical bar represents the assumed expanded uncertainty of ±0.08°C. Open points require review under the decision rule discussed below. They do not represent a separate defect category or a probability of failure.

In an actual experiment, we would retain paired readings for the same serial number. Comparing the average of one group of open boards with the average of a different group of assembled devices mixes assembly effects with differences between units. I would also include a time control: leave some units open for the same period to investigate whether fixture warming explains the shift.

The next step would reverse the change and separate factors. A reading that returns after removing the lid supports an association with assembly, but still does not distinguish pressure from temperature. A mechanical spacer, board-temperature monitoring or a change in tightening order may provide further evidence. Changing everything at once may remove the symptom while losing the explanation.

An experimental log should therefore record the order of interventions. Otherwise, a successful final state can become a story about the last component changed, even if an earlier modification was responsible. The useful result is a reproducible relationship between a controlled change and an observed response.

Requirement limits and acceptance limits

A reading of +0.48°C lies inside the ±0.50°C requirement. With expanded uncertainty of 0.08°C, however, the interval associated with that result extends to +0.56°C. The position of its centre does not justify pretending that uncertainty is absent.

For this example, we choose a conservative operational rule: accept when |error| + U ≤ 0.50°C. With U = 0.08°C, the acceptance limits become ±0.42°C. A result whose entire associated interval lies outside the requirement is classified as nonconforming. Remaining results are held for review. This is the policy selected for the example, not a universal rule or a quantitative determination of false-acceptance risk.

The model batch produces 12 accepted results and eight requiring review. None is conclusively nonconforming under this rule: even +0.58°C with U = 0.08°C touches the +0.50°C limit. This does not establish that every device is good. It means the chosen measurement capability cannot resolve the disposition of some units.

Uncertainty must come from a budget covering relevant contributions to the method. Display resolution cannot simply be renamed fixture uncertainty. NIST TN 1297, section 5 describes combination through a measurement model, including correlations. Expanded uncertainty applies a coverage factor whose interpretation depends on assumptions, as explained in section 6.

Establish whether the fixture can distinguish units

If an unchanged device alternately passes and fails, the problem may be in the product, the fixture, or both. Repeating the test until it becomes green only hides the issue. Saving exclusively the last attempt is particularly misleading.

An initial fixture investigation can include several devices spanning the observed results, two operators and repeated measurements after remounting. Mix the order so that operator identity is not confounded with temperature changes over time. Define whether a repeat includes removing the unit, reconnecting leads and waiting for stabilization. Ten readings from an untouched setup answer a narrower question than ten independent mountings.

NIST’s Gauge R&R material distinguishes repeatability, reproducibility, stability and measurement bias. Applied here, these become separate questions about repeated readings, operator differences and changes over time. The detailed study and estimation method must suit the measurement. A small exploratory exercise identifies problems; it does not automatically qualify the fixture.

Control specimens are also useful. One stable device can warn of fixture changes, but cannot exercise the complete measurement range. A specimen with a deliberately introduced, known fault can establish whether a particular test detects that nonconformity. Such specimens need unmistakable identification and physical separation from outgoing products. A golden sample without history and periodic checking may drift along with the process.

Fixture maintenance belongs in this discussion. Worn contacts, contaminated surfaces and replaced cables can alter the measurement without changing the software version. Record interventions that plausibly affect results, then decide which control checks are needed before production resumes. A maintenance log becomes useful evidence when an entire batch appears to shift simultaneously.

Calibration should not conceal an assembly problem

If closing the enclosure creates a constant error, a software correction may appear to be the cheapest solution. First establish whether the error is actually stable. A correction determined at 25°C need not eliminate error at 10°C or 40°C. It cannot repair variable thermal contact, mechanical hysteresis or drift caused by material ageing.

For this example, the sensible sequence is to understand the mechanism, reduce design sensitivity, and then assess whether individual correction is necessary. When justified, a correction needs an identifier, an algorithm version, input data and the conditions under which it was derived. Acceptance should verify performance after saving coefficients and restarting the device, rather than relying only on a temporary value in the calibration tool’s memory.

The permissible correction range matters too. An unusually large correction may indicate the wrong component or damage. If automation can map any reading to the reference value, the test can conceal faults. The process record therefore needs limits on both the final error and the validity of the calibration operation itself.

Calibration data should be associated with the physical unit, not merely its position in a tray. A communication retry or operator moving two devices can otherwise assign a correct coefficient to the wrong product. Read back the serial number and saved data before final acceptance. The point is to verify the state the customer receives, not just the success of a command sent by the fixture.

What must move from the design team to the fixture

Handover documentation should allow someone who did not design the product to perform the test. That is a useful assessment of the instructions: can the operator establish the correct configuration, start the procedure and recognize a stop condition without calling the firmware author?

For the model module, I would prepare the following acceptance record. Times, equipment and values must be approved after developing the method. An empty field should not invite an operator to improvise.

The connection from repair back to testing does not mean erasing previous data. A repaired device keeps its history. If a changed component affects several characteristics, the retest scope should follow an impact assessment, not only the last error message.

Records should survive a fixture-computer failure. A small team can use a simple controlled register and backup, provided it links results to units and distinguishes an unfinished test. Printing a PASS label before durable recording creates an opportunity to ship a device without acceptance evidence.

A practical recovery rule is equally important. After an interrupted run, the fixture should identify whether the test never started, was partially completed or finished without a saved disposition. Automatically treating all three cases as “resume from the next step” risks skipping an operation whose completion was never established.

Do not turn the first batch into impressive-looking statistics

Process-capability indices are attractive because they compress quality into a number. For a stable process under suitable distributional assumptions, Cp compares tolerance width with spread, while Cpk also reflects location relative to the limits. NIST provides the formulas and interpretation conditions. The mixture of two assembly behaviours in our example is not a sound basis for applying such a number without further analysis.

For an early batch, a production-order plot annotated with operator changes, enclosure batches and fixture identity may reveal more. If readings jump after a tool tip is replaced, an average for the whole week obscures the information needed for correction. Useful analysis begins by preserving sequence and context, rather than selecting an impressive metric.

First-pass performance also differs from final yield. Shipping all 20 devices after rework does not establish that production was 100% correct. The cost of eight investigations, repeat tests and delay belongs to the process. Retaining first-attempt results lets the team determine whether the next batch has actually improved.

Changes in the product mix can also distort a comparison. A week of easy configurations should not be celebrated as a process improvement over a week of difficult ones without examining the difference. A modest register with configuration, first result and disposition can be more informative than a dashboard that aggregates away those distinctions.

The cost of one more check

Assume, for planning, that checking the complete assembly adds 90 seconds of fixture occupancy per unit. For 1,000 units, that is 25 hours. It is fixture time, not necessarily 25 hours of operator labour: part may run automatically, and several units may stabilize in parallel. Conversely, waiting for a shared chamber may extend delivery more than the measurement itself.

A decision to shorten a test should be tied to detection capability. A faster surrogate test may be possible, but its relationship to the full procedure must first be assessed using representative units, including borderline and faulty ones. Correlation among good devices alone does not establish that the shortcut detects a rare fault. Saving 30 seconds is unconvincing if it removes the only check of enclosure effects.

Destructive or long-duration tests may need a different strategy from individual-unit acceptance. A sampling plan should state what decision it supports and which risks are accepted. “We checked several devices” cannot become “every device meets the requirement” without additional evidence. Our example retains checks of every unit at the specified points.

Consider the cost of ambiguity as well. If eight units must repeatedly wait for the designer, fixture throughput is not the main bottleneck. Better instructions, more capable measurement or a less sensitive design may reduce this queue. Increasing operator speed cannot resolve an engineering decision that the process has failed to define.

A readiness criterion for the first series

Before starting the next batch, the team should be able to identify the approved configuration, controlled assembly parameters, final-test scope and the rule for borderline results. It also needs an owner for deviation decisions. An operator should not have to resolve the conflict between a shipping deadline and a quality requirement.

A practical review can end with five questions:

1. Are we testing the configuration the user receives, and do we understand operations performed afterwards?
2. Can we distinguish product variation from fixture variation?
3. Does every disposition have a result, conditions, procedure version and serial number?
4. Do repeats and repairs preserve the history of the first nonconformity?
5. Do we know which observation stops the batch and who may restart it?

Answers should point to concrete records and process behaviour. The hundredth unit then stops depending on the memory of the person who commissioned the first. It may still expose a new problem, but the team has a way to detect it, limit its reach and verify whether the corrective action works.

A repeated measurement should retain the original attempt identifier and the reason for repetition. This makes it possible to distinguish an actual product correction from selecting a more favorable result among several attempts.

Sources

• NASA, Systems Engineering Handbook: Product Verification - the purposes of verification and validation, and the relationship between testing and product configuration.
• NIST/SEMATECH e-Handbook: Gauge R&R studies - areas of measurement-process evaluation.
• NIST TN 1297, 1994 edition, section 5: Combined Standard Uncertainty - uncertainty contributions and models.
• NIST TN 1297, 1994 edition, section 6: Expanded Uncertainty - coverage factors and interpretation.
• NIST/SEMATECH e-Handbook: What is Process Capability? - capability indices and their assumptions.

Sources checked on 21 October 2026. The 20-unit batch, acceptance rule and example timings form an original educational model. No hardware experiment was performed; the plot reproduces explicit synthetic data.