Information Management

Critical care quality systems – benchmarking and the Internet

Aarno Kari, M.D., Ph.D., Managing director
Intensium Oy
PO BOX 1188
FIN-70211 Kuopio, Finland
Phone +358 17 264 9670

Email: aarno.kari@intensium.fi

The article also available in PDF: 55 KB

Information technology and quality systems

Clinical information systems provide the users with powerful tools to support formal quality systems. Kuopio University Hospital applies a certified ISO-9002 system throughout the institution. The ICUs use the deioCliniSoft information system for the automated production of the reports on non-conformities according to patient groups (Figure 6.). These reports are used when the decisions of corrective and preventive actions are made according to the quality system.

Figure 6. The use of the CIMS as a part of ISO-9002 quality system.

Benchmarking

A tremendous amount of effort has been made to build up benchmarking databases for intensive care. At present, the number of active databases including more than 50,000 admissions is probably between seven and ten. Most of these databases have been used for more than five years. The penetration rates, with the exceptions of two or three countries are poor and we know surprisingly little about the real benefits of the benchmarking databases.

Some constraints of traditional databases:

1. The dataset may be too extensive
2. Tools for data capture and transmission may be underdeveloped
3. Availability and timeliness of feedback
4. Failure to provide users with information instead of data

The most important reason for the poor acceptance is certainly the "obesity" of the dataset. If the set is so large that an ICU has to hire a dedicated data collector, the possible benefits of benchmarking may not be realized. If the number of variables collected and never reported is high, the motivation of the staff to do the job may decrease.

Data capture on paper forms is time consuming both to the nurses and secretaries who enter the data into a computer. The transmission of data on diskettes or via e-mail is also time consuming, and is prone to errors.

Data cleaning and preparation of the reports takes time, and the end-users may not know when the next reports will be available.

Finally, we may not know enough about the real needs of the end-users. What is really relevant information when they want to know how they are doing compared to others?

Benchmarking and Internet

The internet has tremendous potential as a benchmarking network tool. The user only needs internet access and a standard browser. Data can be entered directly to the web database and updated reports are always available. The database and case report forms can be easily maintained to meet the users’ requirements.

An internet-based benchmarking system for intensive care has been used in Finland for more than two years. We have a network of more than 20 ICUs. Most of them collect data on paper forms and allow secretaries to enter the data into our database using a web browser. The ICUs are provided with continuously updated, predefined reports and an opportunity to design specific reports for their own purposes.

The technical stability of the system has been good, and the internet response times within Finland have been acceptable.

Some questions related to use of internet

The opponents of an internet-based approach have listed at least the following potential disadvantages:

1. Not all ICUs have access to the internet (mostly because of firewall arrangements within the hospital data network).
2. Data transmission rate may cause problems in interactive data entry during "the internet rush hours".
3. Unmanageable risks of data safety and security in the open internet environment.
4. Regulations restrict the transfer of medical data out of the hospital or out of the country.

The first argument is valid for the moment, but attitudes may change over time when safer solutions will be available. There are already some technical solutions to protect hospital network when internet based systems are used (e.g. Virtual Private Network solutions).

Alterations of data transmission rates are a problem, especially in Europe. We measured the response times for predefined steps required to navigate on our data collecting forms in 12 Finnish ICUs and in nine randomly selected sites in other European countries. The mean response times to move from one step to another varied in Finland between 1.0 and 9.6 seconds whereas the range in other European sites was 8.3 – 23.6 seconds. Although we can expect that the response time may become shorter in the near future, the limitations of the data transmission rate remain a problem in data entry.

Data security and safety have been considered major issues when evaluating the possibilities of the Internet in benchmarking processes. The risks seem to be roughly overestimated. This is obvious when comparing the data management steps of a distributed and a web-based system (Table 1.). A paper-based system combined with distributed PC software may be even more vulnerable to data safety and security violations than a properly designed internet-based system is.

An essential prerequisite to an Internet-based quality reporting system is a totally computerized data cleaning procedure operating at the level of data entry. Data should be clean enough at the point of data entry to be included in continuously updated reports. Additional cleaning procedures may be needed afterwards, if the data are used for scientific purposes.

Table 1. The steps of data management in a distributed and in a web based benchmarking system.

Reporting – Information instead of data

The dual purpose of a database (scientific and benchmarking) may sometimes cause problems in reporting. The fine-tuning of the classification of patients with sepsis is essential to control case-mix in research, but results in categories with zero to five patients per year in a small ICU. This information is not useful for benchmarking where the emphasis lies in the comparisons.

We have developed a Four-Dimensional Quality Index, which is simple enough and still provides the users with essential information about how they are doing compared with others (Table 2.). The first dimension is the quality of the collected data: "Are we measuring things correctly?". The second dimension gives the answer to the questions: "Are we doing the right things?" Do we admit patients to the ICU who benefit the most from the treatment and do we discharge them at the right time. The third dimension describes the outcomes: Are we doing the right things right? Finally, the fourth dimension tells us how much of the resources we are consuming to achieve our goals. Each dimension is presented as an index ranging from zero to more than100. It shows the rank order of an ICU among other members of the network. The index value 100 indicates that the ICU has been ranked as the second best in this dimension of quality and value 50 indicates the second worst performance. The indexes are presented quarterly, and the user immediately gets an idea how his or her ICU has been performing. The user can drill original data into any of the indexes to see what the underlying reason for the present ranking is.

Table 2. Components of Four-Dimensional Quality Index for Intensive Care

Even very basic information can be successfully used for benchmarking. In anesthesiology, the throughput times in the OR are the most trivial information of the quality systems. They become, however, very interesting when they are compared with the corresponding results from another hospital. In the pilot using a Finnish Anesthesiology database, we could show that there are remarkable differences in the throughput times even in homogeneous groups of cases (e.g. scheduled cesarean section). The difference was mainly because of variances in the time between the patient’s admission to the OR and the surgical incision.

Next steps

There are several significant improvements in the quality management, which can be achieved by the applications of new information technology:

Data capture should be brought to the bedside either by using a handheld computer for data entry and transfer to a web database or by using a CIMS interfaced to a validation workstation and the web database.
The development of the point-of-care clinical information system will make it possible to re-engineer care processes. Standardized care protocols have already been shown to improve the quality of care and reduce the variation of the quality (14-16). When the CIMS support the design and use of the protocols quality management can be extended inside the processes instead of the current view focused only on inputs and outputs. This will open totally new perspectives for quality management.

References

1. Knaus WA, Draper EA, Wagner DP, Zimmerman JE.: APACHE II: a severity of disease classification system. Crit Care Med 1985 Oct;13(10):818-29
2. Knaus WA, Wagner DP, Draper EA, Zimmerman JE, Bergner M, Bastos PG, Sirio CA, Murphy DJ, Lotring T, Damiano A, et al.: The APACHE III prognostic system. Risk prediction of hospital mortality for critically ill hospitalized adults. Chest 1991 Dec;100(6):1619-36
3. Le Gall JR, Lemeshow S, Saulnier F.: A new Simplified Acute Physiology Score (SAPS II) based on a European/North American multicenter study. JAMA 1993 Dec 22-29;270(24):2957-63
4. Cullen DJ, Civetta JM, Briggs BA, Ferrara LC.: Therapeutic intervention scoring system: a method for quantitative comparison of patient care. Crit Care Med 1974 Mar-Apr;2(2):57-60
5. Lerou JG, Dirksen R, van Daele M, Nijhuis GM, Crul JF.: Automated charting of physiological variables in anesthesia: a quantitative comparison of automated versus handwritten anesthesia records. J Clin Monit 1988 Jan;4(1):37-47
6. Kari A, Ruokonen E, Takala J.: Comparison of acceptance and performance of automated and manual data management systems in intensive care. Int J Clin Monit Comput 1990 Jul;7(3):157-62
7. Abenstein JP, De Vos CB, Abel MD, Tarhan S: Eight year's experience with automated anesthesia record keeping: lessons learned--new directions taken. Int J Clin Monit Comput 1992;9(2):117-29
8. Pierpont GL, Thilgen D.: Effect of computerized charting on nursing activity in intensive care. Crit Care Med 1995 Jun;23(6):1067-73
9. Weinger MB, Herndon OW, Gaba DM.: The effect of electronic record keeping and transesophageal echocardiography on task distribution, workload, and vigilance during cardiac anesthesia. Anesthesiology 1997 Jul;87(1):144-55
10. Hammond J, Johnson HM, Varas R, Ward CG.: A qualitative comparison of paper flowsheets vs. a computer-based clinical information system. Chest 1991 Jan;99(1):155-7
11. de Keizer NF, Stoutenbeek CP, Hanneman LA, de Jonge E.: An evaluation of Patient Data Management Systems in Dutch intensive care. Intensive Care Med 1998 Feb;24(2):167-71
12. Bosman RJ, Oudemane van Straaten HM, Zandstra DF: The use of intensive care information systems alters outcome prediction. Intensive Care Med 1998 Sep;24(9):953-8
13. Suistomaa M, Kari A, Ruokonen E, Takala J.: Sampling rate causes bias in APACHE II and SAPS II scores. Intensive Care Med 2000 Dec;26(12):1773-8
14. Brook AD, Ahrens TS, Schaiff R, Prentice D, Sherman G, Shannon W, Kollef MH: Effect of a nursing-implemented sedation protocol on the duration of mechanical ventilation. Crit Care Med 1999 Dec;27(12):2609-15
15. Morris AH: Computerized protocols and bedside decision support. Crit Care Clin 1999 Jul;15(3):523-45
16. Marrie TJ, Lau CY, Wheeler SL, Wong CJ, Vandervoort MK, Feagan BG: A controlled trial of a critical pathway for treatment of community-acquired pneumonia. CAPITAL Study Investigators. Community-Acquired Pneumonia Intervention Trial Assessing Levofloxacin. JAMA 2000 Feb 9;283(6):749-5
    

Last updated: 1 January 2002Created
Legal notice	© GE Healthcare 2008 ISSN 1795-6269 (Web) ISSN 1795-6277 (CD)	Webmaster