In my last blog on personal spirometers I mentioned that one of the university projects has developed software that assesses spirometry results for ATS-ERS standard including cough, glottal closure and early termination of exhalation. The specific project that claimed this also claimed to have attained a 70% detection rate using waveforms from the NHANESIII study. This should be taken as a challenge by the pulmonary function equipment manufacturers to improve their existing software.
The test systems I am familiar with are pretty much limited to being able to determine back-extrapolation and whether end-expiratory flow rates meet ATS-ERS end-of-test criteria. So far they have not attempted to assess spirometry efforts for the cough or glottal closure that are also part of the ATS-ERS criteria for test quality. Here is a good example of glottal closure:
The FEV1 for this effort (unfortunately also the best the patient was able to perform) is underestimated due to the short sub-1 second pause. Taken at face value, the FEV1 and FEV1/FVC ratio would have indicated the patient had mild airway obstruction. Reading between the lines if there hadn’t been a pause the results might have been within normal limits.
Even though the manual for our test systems states that its “software complies with the recommendations set forth within ATS/ERS 2005 Standardisation of Lung Function Testing guidelines” our test system passed this spirometry effort with flying colors. I am sure that someone can give me a lawyer’s interpretation of the word “complies” that shows the software is somehow actually in compliance but to me it says that our equipment manufacturer is being selective about which ATS-ERS criteria it actually meets. In some ways this is not a fair statement to make because there are a number of ATS-ERS criteria for different pulmonary function tests that can only be met by actually observing the patient during testing, but that also means that a blanket statement like that shouldn’t be in the manual in the first place.
I will admit to having a certain level of frustration in dealing with equipment manufacturers. My PFT lab is both busy and relatively sophisticated and we run into oddball problems not all that infrequently. There are always going to be computer glitches and patients that find new ways to do tests incorrectly, but trying to determine what is causing a problem can be difficult in large part because the documentation that comes with our different test systems is inadequate. Getting answers from the manufacturers isn’t particularly easy either. Its one thing when you have a broken test system and can call tech support, its another when you have a system doing something odd intermittently and you don’t know if it is a feature or a bug. We have passed what we thought were reasonably critical problems to a manufacturer and then had to wait a couple weeks to get an answer back. Sometimes we’re told that yes, there is a bug and it may be fixed in the next software release. Sometimes we’re told that we’re misunderstanding the problem and when they explain it, it makes sense, but then why wasn’t it in the manual in the first place?
Anyway, I am not sure why detecting glottal closure and coughs during exhalation isn’t being attempted. I realize that a software algorithm for detecting glottal closure and coughs can’t be perfect and that is because even experienced human observers can occasionally miss them as well. My programming skills are years out of date but even so, detecting the pause seen in the spirometry example should be a relatively trivial problem. And if a small bunch of undergraduate college students can do it (on a smartphone no less), why isn’t it being done by equipment manufacturers?
My concern about this is that I have seen a trend towards less and less well-trained staff performing spirometry. I know of several local clinics that have medical assistants performing spirometry that have been, at best, trained by another medical assistant. Often more time is spent teaching them how to correctly enter the patient’s demographics into the computer than in actually performing the test.
Many physicians are also not as aware of testing errors as they should be and only look at the numbers on the reports. We occasionally have teaching sessions with hospital residents and fellows about pulmonary function testing. These sessions are not about interpretation but how to evaluate test results for inconsistencies and errors. We have used spirometry efforts like this as an example of misleading results and so far this particular error is only rarely noticed. Having said that, presently this is an error that can only be seen on a graph and only if it is understood what the graph is saying, but since the graphics that come with spirometry reports are often quite small (gotta cram everything on one page, after all) an error like this may be too subtle to be easily noticed in the first place.
I have mixed feelings about the increasing reliance on computers. I started in this field when all the equipment was manual and all the records were paper and I will be the first to admit that a computer can’t be beat for calculations and data management. On the other hand, I see that people often let the computer do their critical thinking and “if the computer said it, it must be so” is all too common an attitude. Even more unfortunately the thought “if a computer doesn’t say it, it must not be so” is a common corollary to this as well. The reality, like it or not, is that we will continue to become more dependent on computers, not less and this puts a burden of responsibility onto the equipment manufacturers that when they say they meet ATS-ERS standards, that they actually do without having to lawyer-up.
PFT Blog by Richard Johnston is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.