A General Tool for Acceptance Testing

The Reliable Quantification Test can be used setting general acceptance standards for production equipment. MS Excel helps calculate MTBF values.

Design and fabrication are critical steps in providing reliable equipment. Many frustrations could be avoided if a “crystal ball” were available to see, before acceptance, how the system will operate 6 months after bringing it on line. If it fails to meet expectations, the new equipment’s performance may be a critical parameter impacting total factory output. This is the reason to take strong action in making the right results happen.

All projects, structured or unstructured, have a defining moment when the equipment is configured and turned on for its first production run. Unstructured approaches often have many surprises and struggles to meet expectations. Even structured methods need to address the step of “Start Up—Functionality Demonstration.” A good tool to help in this situation is a testing formula involving mean time between failures (MTBF). The Reliability Quantification Test (RQT) formula came from Military Standard 781. The formula was championed throughout INTEL Corp. by Dave Troness. The company has been proactive in applying this tool in setting performance expectations with its suppliers.

RQT is my first choice in setting general acceptance standards for future suppliers of typical production equipment. Obviously, for highly critical, dangerous, or life-dependent systems, where greater than 80 percent confidence levels are required, more extensive, designed experiments should be carried out.

RQT can be used for new installations and for new subsystems proposed for existing equipment lines, which is a more-frequent occurrence for maintenance organizations. For proposed subsystems, one of the first steps is to quantify existing performance using on-going data collection. A functional block diagram of the operation is needed that has at least one block outlining the specific subsystem. MTBF information should be generated for each block. This information quantifies the current status and can help set standards for the new subsystem.

The expectations for the proposed subsystem should be part of the requirements document, and suppliers need to understand the assumptions and share the same goals. The purchase order should address the potential outcomes of the acceptance test, and the project timeline needs to account for the testing time, and possible re-testing.

The RQT is an objective way of determining, with high confidence, that the new system is acceptable, or that it needs further improvement, before impacting current operations. It is an appropriate test for the supplier to do before shipping but after “burn in.” Or, for demonstration testing, it can be done in line as long as the ability is provided to immediately go back to the previous configuration if the test fails. For multiple lines of identical equipment, RQT can determine an objective test plan for the first upgrade before migrating the changes to the rest of the lines.

The RQT equation is useful in solving for MTBF or in determining the required test time given an expected MTBF. The formula is:

MTBF = 2T/X² a2r+2
where:
T = time
X² = chi-squared

Chi-squared is a numerical value obtained from a standard chi-squared table, and can be found easily using MS Excel on a personal computer. The two parameters needed to find the proper X²value are alpha, a, and the Degrees of Freedom, DF. The recommendation is to use 80 percent as the desired confidence level (for typical production equipment); therefore a will be 0.2. DF is determined by solving the equation:

Degrees of Freedom = 2r + 2
where
r = number of failures.

Therefore, if the number of failures is 0, 1, 2, 3, …, then the DF is 2, 4, 6, 8,…etc.
With these two parameters, start Excel and click on the function wizard (fx) button. Choose Statistical, and under the function name, select CHIINV. Click OK. The function wizard will ask for probability and Degrees of Freedom. The chi-squared value will show in the value box as soon as the a (0.2 for typical RQT) and the DF are filled in.

Two ways to use RQT
The RQT formula can provide a MTBF value from known data of the number of failures for a given test time. Suppose an existing subsystem was operated for 480 hours and it failed 7 times during that period of time. By solving the RQT formula for MTBF we would be able to say with 80 percent confidence that the MTBF for the existing system is at least 47 hours (MTBF = 2(480)/ 20.47 = 47 hours). You could then require the proposed subsystem to be that level or better so it would not significantly impact factory flow.

By properly evaluating the functional block diagram of the overall system, the influence of the subsystem could be quantified. If the new system were to be improved, the benefit for the overall system also could be estimated.

The second way to use the formula is for a test. Assume that the new subsystem is required to perform with 120 hours MTBF. To determine the testing time and failure expectations, we would first solve the RQT formula for time (T). An acceptance test table can now be developed by solving for T for various numbers of failures.

To make the test relevant, actual factory conditions need to be reproduced, and this may take creativity. Materials, environment, utilities, performance rates, waste rates, etc. need to be replicated as if the system were in production. These conditions may be difficult to replicate, especially at the supplier’s site. They need to be prioritized relative to the critical nature of the proposed system. Projects have had serious difficulty because testing was done with substitute materials and under ideal conditions. Do not discount any factor without carefully weighing the impact of that condition.

Resolve not to accept or go forward with compromised equipment. The project’s business case performance levels could be unattainable if the original design, fabrication, or assembly is not capable.

Equipment changes are part of every operating area and need to be recognized as opportunities for improvement. The RQT formula is a simple way of bringing some objectivity to bear on making the right things happen for successful transition to new equipment systems. If demonstration RQT is done with creativity, successful transitions will result. MT

Robert C. Hansen has more than 20 years experience as an engineering and maintenance department manager for a large manufacturing company. He currently is a consultant on manufacturing productivity and can be reached at R.C. Hansen Consulting, P.O. Box 272427, Ft. Collins, CO 80527; (888) 430-4633; e-mail rch4OEE@aol.com.