Krouwer Consulting
ACT Order 

 

Home
Up
Books
ACT Software
FMEA software
Smart Tag
Pivot Table Log File Analyzer
Training

ACT = Assay Comparison Tool Download the manual

ACT software is now available and costs $25 - Click on Purchase to view requirements and order

The Assay Comparison Tool (ACT) is an Excel add-in that allows one to compare the quality of different assays (e.g., chloride, cholesterol, etc.) using process capability analysis of quality control results. A description of the concept has been published (1).

ACT requires Windows 2000 or later as the operating system and Excel 2000 (part of Office 2000) or later.

Perhaps the best way to see how ACT works is to follow through an example.

Inputs required by the program

One starts with as much quality control data for as many assays as desired. This data must be in an Excel file. The format of the data doesn’t matter, as long as there is a rectangular block of data containing at a minimum, the name of the assay and each quality control value assayed. ACT will analyze control values grouped by assay, lot number, and instrument. ACT asks the user to enter quality control targets and medically acceptable limits for each assay, lot number, and  instrument (if targets and medically acceptable limits are in the original Excel file – this step will have already been done).

Medically acceptable limits – These are not the quality control ranges. Medically acceptable limits are the ± distances from target that are expected not to cause clinical treatment errors, when values are within limits. CLIA limits are a useful starting point, with the exception of ± 3 SD limits, since these limits depend on the data.

Outputs provided by the program

ACT calculates the process capability metric Cpm for each assay, lot number, and instrument. Cpm is a unitless measure of distance from target (the total analytical error scaled by the medically acceptable limits). Because this measure is unitless, different assays may be compared. Higher Cpms are better (ACT flags Cpms that are less than 1). Assays are listed in decreasing order of Cpm. See the example below.

In addition, ACT outputs

  • the number of values outside of medically acceptable limits (either out low or out high)

  • the estimated total analytical error

  • the lower and upper 95% uncertainty intervals (non parametric estimation)

  • the contribution of bias as a percent of total analytical error

  • the contribution of imprecision as a percent of total analytical error

The last two items allow labs to simulate effects of assay improvement.

An example of ACT output

Assay

Lot Number

Instr.

Med. Limits

Cpm

Sum of out high

Sum of out low

Pct from Bias

Pct from Prec.

Total Error

Lower 95% Uncert.

Upper 95% Uncert.

K

16271

CHEM1

0.5

1.821

0

0

0.3%

99.7%

0.092

-0.2

0.1

K

16272

CHEM2

0.5

1.204

0

0

3.7%

96.3%

0.138

-0.3

0.2

CL

16272

CHEM2

5.25

0.946

0

1

22.3%

77.7%

1.849

-4

2

CL

16271

CHEM1

4.25

0.922

1

0

33.7%

66.3%

1.536

-1

4

NA

16272

CHEM2

4

0.917

2

0

18.5%

81.5%

1.454

-2

5

NA

16271

CHEM1

4

0.880

2

1

0.0%

100.0%

1.516

-3

4

Assay

Lot Number

Instr.

Med. Limits

Cpm

Target

Pct from Bias

Pct from Prec.

Total Error

Sim. Cpm

Sim. Bias

Sim. Reps

K

16271

CHEM1

0.5

1.821

4

0.3%

99.7%

0.092

1.821

1

1

K

16272

CHEM2

0.5

1.204

6

3.7%

96.3%

0.138

1.204

1

1

CL

16272

CHEM2

5.25

0.946

105

22.3%

77.7%

1.849

0.946

1

1

CL

16271

CHEM1

4.25

0.922

85

33.7%

66.3%

1.536

1.133

0

1

NA

16272

CHEM2

4

0.917

147

18.5%

81.5%

1.454

0.917

1

1

NA

16271

CHEM1

4

0.880

133

0.0%

100.0%

1.516

1.244

1

2

    Note: Some columns hidden for screen format.

Interpretation

  1. The column labeled Cpm shows that the 2 K assays are acceptable with Cpms greater than 1, no values outside of medically acceptable limits (Med. Limits), and low percent bias (Pct from Bias).

  2. The chloride and sodium assays have lower Cpms with values that are outside of medically acceptable limits and some large biases.

  3. Lower and upper 95% uncertainty intervals are provided.

  4. For the last Cl assay, if one removed the bias as shown in the column Sim. Bias. by setting bias to 0,  the new Cpm would now be over 1.

  5. The effect of increasing replicates (Sim. Reps) on Cpm, is shown for the last Na assay.

  6. These are theoretically possible improvements - real improvements require understanding the root cause of the problem.

Summary

The table below shows the relationship for various medically acceptable limits and quality control limits.

Case

Medically Acceptable
Limit

Quality Control
Limit

Comment

1

Data passes

Data passes

Assay is in control with
medically acceptable results

2

Data passes

Data fails

Assay is not in control but has
medically acceptable results

3

Data fails

Data passes

Assay is in control but has
medically unacceptable results

4

Data fails

Data fails

Assay is not in control with
medically acceptable results

Note the two cases in which the data has failed one of the limit conditions. In case 2, even though the assay is not in control, the results will not cause an increase in treatment errors. Case 3 could occur for assays if the medically acceptable limits are narrower than the quality control limits. An example of this is troponin I, where assay performance has not met consensus (ESC-ACC) imprecision recommendations. Case 3 is an example of a process that is not “capable”. That is, even though results are unacceptable, the process is in control. It would be a mistake to make the quality control limits narrower. One would get frequent alerts but there would be nothing to fix.

Reference

1. Assay Development and Evaluation: A Manufacturer’s Perspective. Jan S. Krouwer, AACC Press, Washington DC, 2002 pp 96-101.