Should biological quality control be replaced?

I’ve been thinking about quality control and quality improvement lately. Mostly this has been about how to go about determining whether the lab has a quality problem with testing and what statistics should be used for this purpose but I was reminded recently about an issue concerning biological quality control that came up a couple months ago on the AARC diagnostics forum. Specifically, one of the participants noted that some of their technicians had refused to perform biological QC on the basis that it violated their HIPAA rights to the privacy of their medical information. Further discussion noted that this was actually a correct interpretation of the HIPAA regulations and that no PFT lab can “force” its technicians to perform biological QC.

I will be the first to admit that I’d never thought about it this way, and I’ve been mulling it over ever since. I’ve performed PFT testing on myself both for formal biological QC and as a quick way to check the operation of a test system for decades but I never thought of my PFT results as being part of my medical information. That’s probably an indication of my own short-sightedness however, and I also realize that over the years I’ve run across a number of testing issues I’d taken for granted up until somebody pointed out a problem with them.

My attitude towards my PFT results may also be due to the fact that I don’t have any notable lung disease. My lab has had technicians who have been asthmatic however, and this has never been a factor in whether they were hired or not (other than not letting them perform methacholine challenges). They’ve usually performed bio-QC on themselves and at the time they seemed to regard it as a way to check on the status of their asthma. In retrospect however, I have to wonder if they were ever concerned that I would use their health status and test information against them in their annual evaluation, or even that the hospital would re-consider their employment because the costs of their health insurance might be higher. Although I don’t think the hospitals I’ve worked for ever thought along these lines, like it or not there are many businesses where this is a factor.

Yesterday I asked myself what would happen if all PFT labs were required to completely end biological quality control because of HIPAA requirements? It didn’t take a lot of thought to realize that there are a number of mechanical test simulators in the marketplace that could do quite well at replacing the biological part of quality control. As importantly, the more I’ve thought about it the more I’ve come to think that biological QC probably isn’t the right way to go about QC in the first place.

First, no matter how experienced we may be there’s still a lot of inherent variability in PFT results, even when we perform the tests on ourselves. This is the nature of any biological system and it’s why bio-QC results are plotted and analyzed using standard deviations on a Levey-Jennings chart.

From Wikipedia

The problem with this is that the “normal” range is based on the standard deviation of the test results themselves, and that the greater the variability these test results have, the larger the “normal” range is going to be as well. This means that a given test system may be considered to be out of range for one individual with a naturally low variability in test results but still within range for another individual who has a naturally high level of variability.

Another problem with this approach is that it only considers whether changes over time are within a specified range of the average, not whether the results are actually accurate. Say, for example, the calibration syringe used by a lab was significantly inaccurate for some reason (the collar on the stem was loose and slipped, for example). Hopefully this is unlikely but nevertheless it would mean that the test equipment would be mis-calibrated. Biological QC might pick up on the fact that a calibration syringe had changed but not if the calibration syringe was already inaccurate when an individual began their QC.

Finally, most individuals working in PFT Labs are likely in reasonably good health with reasonably normal PFT results. Most patients referred to a PFT lab probably have a lung disorder of one kind or another and their results will often be significantly outside the normal range. So, does QC performed on normal(-ish) individuals mean that the test equipment accurately tests patients with abnormal lung function? Maybe yes, maybe no, but think about about patients with severe airway obstruction who have markedly reduced expiratory flow rates and require a prolonged exhalation time in order to get anywhere near a normal FVC. Is the test system really measuring those low flow rates accurately and since most systems tend to be flow-based, is the exhaled volume being accurately integrated from their expiratory flow? Again, maybe yes, maybe no, but biological QC isn’t very likely to tell you this.

So, what can take the place of the biological QC?

Just as automated systems are replacing humans across the job marketplace there are a number of mechanical simulators that can replace the biological component in QC. Realistically however, because the current demand for these devices is presently low, there isn’t a great deal of selection available.

There are a couple of flow-volume simulators (the 691227 from Hans-Rudolph and the PWG-33 from Piston Medical). These can produce any of the 50 different ATS spirometry waveforms which cover the entire range of pulmonary diseases and are used by manufacturers to test their spirometers. Additional waveforms can be imported and edited as well.

There is a DLCO simulator (the 691044 from Hans-Rudolph). I’ve discussed this previously and it can, depending on the gases used, simulate low, medium or high DLCO results. My lab has been involved in several research studies where we were required to regularly test the equipment used on the study’s subjects with a DLCO simulator and its results were uploaded along with the patient test results.

Finally there is a metabolic (i.e., VO2, VCO2, Ve) simulator (the 17050 series from Vacumed). This is able to simulate a wide variety of tidal volumes and respiratory rates as well as VO2 and VCO2.

At the present time there are no lung volume simulators. Despite this lung volumes measurements using helium dilution or nitrogen washout can be simulated (with a bit of care) using a calibration syringe. Morgan Scientific used to sell a simulator for plethysmographs however this product has since been discontinued (but may return depending on customer interest).

Because the market for simulators is so small, they also tend to be built only when an order has been placed. This means there are no economies of scale, which in turn means that they tend to be more expensive than they “should” be and this has discouraged their more widespread use, particularly since they are not required by any regulatory agencies.

But since simulator technology is well understood and their design is a straightforward engineering issue there’s no particular reason that an automated simulator couldn’t be developed that performed spirometry, lung volumes and DLCO. There’s also no particular reason that an automated QC system couldn’t simulate subjects with mild, moderate, severe and very severe airway obstruction as well as subjects with mild, moderate and severe restriction.

At the present time however, despite the fact that an automated QC system could test PFT systems across their entire operational range with a high degree of accuracy and repeatability there is no demand for an automated QC system and it is therefore unlikely to be built. But it’s possible for this situation to change, and perhaps to do so unexpectedly and as much as I’d like to see our field decide to police itself and mandate lab certifications, inspections and QC reporting, it’s more likely that this will be imposed from outside.

One possibility is that a technician could be fired from a PFT lab position, in part allegedly for refusing to perform biological QC. Another is that a physiology student could refuse spirometry (or some other medical-ish) test and then received a poor grade, allegedly because of this. If these, or similar, cases were brought to court, it’s possible the court would rule in favor of the plaintiff on HIPAA grounds and this interpretation of the law could quickly mean the end of biological QC everywhere.

For that matter CMS (Medicare) or SSA (Social Security) could suddenly decide (perhaps in order to limit payments or because they’re tired of questionable PFT results) to require that any PFT results they pay for or that are used in a payment decision could only be performed on test equipment independently verified by QC simulators.

This issue could also slowly come to a head as the word gets around (mea culpa), and more and more technicians decide they don’t have to do biological QC any more. As much as this may appear to cause problems for some lab managers I have a hard time blaming anybody that doesn’t want to perform biological QC because they want to keep any and all of their medical information private. I don’t think of my own PFT results as being particularly concerning but where do you draw the line? If you worked in a chemistry or hematology lab would you feel the same if you had to regularly provide blood or urine samples for QC? Probably not.

The fact is that biological QC is at best a limited way to monitor test equipment. In my experience equipment problems are far more often found during routine calibration and patient testing than they are because of biological QC. It’s a lot better than not performing any quality control at all but it doesn’t (and can’t) insure equipment accuracy and at best it can only detect change over time that is statistically significant. Finally, I’ve never met a technician (or a manager for that matter) that really liked performing biological QC and documenting the results.

Maybe it’s time to look past biological QC to other ways of ensuring the proper operation of our test equipment. An automated simulator would make it a lot easier to routinely monitor test system accuracy and performance and does not violate any HIPAA regulations. Current simulators have a limited range of abilities, are relatively expensive and are available from only a small number of manufacturers but if simulator QC became a requirement this would likely change very quickly.

Creative Commons License
PFT Blog by Richard Johnston is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License

8 thoughts on “Should biological quality control be replaced?

  1. Hi Richard,

    Great discussion!

    I would caution anyone using the Hans Rudolph 691044 system. I used this for a 5 year research program for the Mannkind Corporation and found a major flaw in the system. If there’s a leak on the inspiratory side of the system it won’t detect it.

    The reason is that the QC system has two separate chambers that don’t communicate with each other. The exhalation side of the QC system is filled with a separate gas and, therefore, will NOT detect a leak in the system when it simulates dilution during inspiration. With the nSpire system we had, if there was a leaky, or ruptured balloon, the QC device wouldn’t pick it up,

    After the study was finished we developed a simple, and very cheap, system for simulating DLCO.

    If you purchase a Hans Rudolph 7 Liter syringe and calculate the expected values for CO, methane, etc., when diluted at different volumes, you can be confident in your machine. You must also test the syringe for leaks before using. We used dilutions of 2liters, 5 liters, and 7 liters. If the expected values don’t match up, you have a problem. The usual ATPS and STPS should be included in the calculations. Also, when filling at the different levels, allow 2-3 seconds for mixing.

    It’s time-consuming, but it works. I must say, however, that if the daily DLCO calibration finds a problem, it almost always shows up in the 7 liter QC method.

    Hope this is of some use. I agree with you, however, that we need a standardized method because, someday, it’ll bite us in the “you know where!”

    All the best,

    Ira

    • Ira –

      Kudos, of course, for being so meticulous. Most labs however, have neither the time, resources or expertise to do this in any kind of a routine manner. That’s why an automated QC simulator would be so valuable.

      – Richard

    • Hi Ira and Richard,

      I’d love to contribute to this discussion. I agree with the above syringe dilution method in TLCO quality assurance – and the counter-argument that it can or may be too time-consuming for some.

      I’d argue, however, that it is the linearity, rather than the accuracy, of the gas analysers, that is the more important, based on the second part of the TLCO equation itself: Ln(FATRxFICO/FITRxFACO).

      Therefore, using a syringe dilution, one could then just plot the ratio of expired/inspired CO (e.g. FECO/FICO) against the ratio of expired/inspired tracer (e.g. FECH4/FICH4) at various arbitrary dilution volumes. Assuming this relationship stays linear, one can be fairly confident about the gas analyser linearity – without needing to worry about any further (more complex!) calculations.

      • Ron –

        I’d agree that linearity is more important than accuracy but in order to mix a volume of gas with a calibration syringe you also have to consider the deadspace of the outlet port of the syringe as well as that of any tubing or valves you use. That’s at least one of the reason why the Hans Rudolph DLCO simulator uses pre-mixed gases. I’d also ask how you’re accessing the gas analyzer signals. The test systems that are in my lab do not have any utility software that lets us look at the analyzer outputs directly (I’m pretty sure the service tech does, but he brings it with him). We’d have to go through a simulated DLCO just to look at the gas analyzer results. Not impossible but a bit tricky.

        Regards, Richard

        • Perhaps I’m entirely off-base, but that’s what I’m getting at – a series of simulated TLCO manoeuvres using a calibration syringe, arbitrarily varying the “residual volume” of the syringe by varying how much air is left in the syringe prior to then filling the syringe with the TLCO test gas.

          This is somewhat similar to what the 2017 ERS/ATS standards suggest part-way through point 5 of the Equipment calibration and quality control section: “With approximately 1 L of air in the syringe, the test begins by filling the remaining volume with test gas. Following a 10 s “breath-hold” the syringe is then emptied. The calculation of VA must be within 300 mL of 3 L with the syringe dead-space being used for the anatomic dead-space in the VA calculation. The absolute value of TLCO must be <0.166 mmol·min−1·kPa−1 or <0.5 mL·min−1·mmHg−1."

          As above, if you repeat these simulated manoeuvres, but arbitrarily alter the amount of air left in the syringe for each manoeuvre (i.e. not just 1 L), you can then plot FECO/FICO vs FEtracer/FItracer from each of these simulated TLCO trials to better assess gas analyser linearity.

          The additional dead space of the outlet port and/or tubing should not matter since we are not assessing the calculated VA. Rather, we are assuming that as CO and the tracer gas dilute to different but proportional extents at each varying dilutional volume (of which knowing the exact volumes is not so important), and that relationship of FECO/FICO vs FEtracer/FItracer, when plotted, should therefore remain linear across these varying dilutional volumes if the analysers are working linearly.

          Happy to be corrected if I'm looking at this entirely the wrong way!

          • Ron –

            I don’t see any reason why it shouldn’t work. You’d probably need at least three different dilutions to be able to determine linearity. You might be able to assume that the analyzer zeros were accurate and get by with only two but I’d be concerned about drift. The procedure’s a bit involved so it’s probably not something you’d want to do all that often. If you do try it I be interested to hear how it works for you.

            What I’m not sure about is how often analyzer linearity is really a problem. Too often the problems are with tubing, valves and sampling lines, not the analyzer. I’m concerned about the how effectively the entire system works which is why I’m holding out for an automated simulator.

            Regards, Richard

  2. Just a thought – but why not have your lab personal perform a biological QC but enter a different height, dob etc and use this consistently for the calculations. I.e I am 173 cm and DOB 1982 ; but use 165 cm and DOB 1975. That way the percent predicted is not actually my medical result etc.

    Also – I’ve always longed for a heated syringe arrangement where I could use the syringe to test/simulate under BTPS conditions.

    • Clinton –

      I don’t think that de-identifying bio-QC in the way you proposed would meet HIPAA requirements since the lab manager would still know what the results were.

      A heated syringe would not simulate BTPS conditions since the air inside would not be saturated with water vapor, it would just be heated air. Both of the flow-volume simulators have optional attachments that allow them to provide heated air fully saturated and even with physiological gases (5% CO2, 17% O2).

      – Richard

Leave a Reply

Your email address will not be published. Required fields are marked *