Data Envelope Analysis: A Tool for Measuring Cost Effectiveness in Today’s Challenging Healthcare Environment

Description of DEA

As an analysis tool, DEA traces its development back approximately 75 years ago during the 1950s [13], with formal usage beginning in the decade of the 1980s. It is a non-parametric quantitative analysis tool (makes no assumptions about the distributions of data) that measures relative performance within a cohort of competing units (i.e., healthcare service providers) [13]. In the 12-year span between 1978 and 1990, an estimated 400+ publications used this tool. In the year of 2009, there were 700+ publications produced using DEA, and cumulatively to that point 4000+ publications had utilized this analysis tool [14]. To date, DEA has been used extensively over the past almost 45 years and its use is increasing [14].

DEA derives from the discipline of operations research, and as such, uses linear programming and engages in an optimization process as the underlying mechanism of action supporting the analyses [15]. It computes productivity/ efficiency (technical efficiency) or effectiveness within a group of performance units (healthcare service providers) also called decision-making units (DMUs) [16, 17, 18]. In the DEA output, the DMUs are visually situated along an efficiency frontier, also called an envelope or a facet [16]. Further, DEA models typically are of two types: Constant Returns to Scale (CRS) or Variable Returns to Scale (VRS) [19]. The earlier DEA models utilized CRS, and the later models used either one. CRS models allow each DMU in the DEA analysis to operate at optimal scale efficiency, while VRS introduces some constraints to the underlying analysis procedure. Returns to scale, an economic concept [20] refers to part of the underlying mathematical processing inherent in the DEA analysis that speaks to how the output/input scales of DMUs are calculated, e.g., by adding weight constraints or not, and a full discussion of such is well beyond the scope of this elementary description of DEA.

The output for a DEA analysis results in a ratio of outputs divided by inputs. This ratio approximates productivity/ efficiency, cost effectiveness, or a benefit to cost ratio for the measured healthcare service. The outputs normally represent the measures of productivity that derive from a healthcare service (i.e., the yield), and are typically the units of the service delivered, and sometimes can represent some degree of achievement of a beneficial service impact on service recipients (not always the case). Alternatively, inputs equate to the resources devoted to producing the outputs or service delivery yield, and are, for example, staff time, physical plant operational costs, or other expenses. Inputs can also be thought of as the operational resources that a healthcare service invests to generate its outputs.

With DEA analysis output, the value of 1.0 (Iota score) represents an optimum output to input ratio, which is perfect performance, at least in a relative sense, within the cohort of performing entities (i.e., the comparative group of healthcare service providers/DMUs). When the resulting Iota score is less than 1.0, the extent that performance can be improved in terms of cost effectiveness is the difference between the actual lota score and 1.0. For example, an achieved lota value of .52 means a .48 or 48% improvement in cost effectiveness is possible if there are adjustments to the outputs and inputs.

One absolute requirement of DEA is appropriate theoretical specification in order to produce results that are meaningful [16]. That is, the theoretical specification of outputs and inputs in each DEA model must be conceptually sound, demonstrating alignment with the real-world outputs (sometimes outcomes) and the cost dimensions (inputs) included in the healthcare service. DEA pinpoints the best performers within a healthcare service cohort, and does not offer comparison to a performance absolute or theoretical standard [13, 16, 17, 18].

Additionally, DEA is capable of handling complex mixes of output and input variables with no need to match different units of measurement across the variables. The DEA analysis output identifies the extent of improvement needed for a more optimal output/input ratio, and targets the specific variables that are ripe for modification. These variables can be outputs or inputs and are called slack variables, that if so adjusted can result in enhanced cost effectiveness of the healthcare service.

**Limitations of DEA**

Every type of data analysis tool has limitations, and DEA is no different. First, DEA is sensitive to extreme outlier values in the outputs or inputs [16]. Secondly, DEA does not provide a root cause analysis in identifying the reasons for low performance [13]. The third limitation is that DEA only provides a relative perspective on cost effectiveness, identifying the best performer in the comparison group of healthcare service providers (i.e., DMUs along the envelope/face/efficiency frontier), with all other performers compared to that standard [13]. Finally, in order for DEA to provide useful results, there must be appropriate theoretical specification that differentiates outputs from inputs conceptually in the ratio equation. Such conceptualization must be clear and represent an intuitive match between the real-world healthcare service outputs and operational resources (inputs).

**Two Illustrations of DEA's Potential Use in Healthcare**

What follows are two illustrations of DEA's potential use in healthcare. The input and output data along with the lota scores are fictious. The goal is to provide two conceptual examples of the use of DEA to measure cost effectiveness of healthcare services to illuminate its utility as a tool. Please also note that these illustrations are purposely parsimonious with the inclusion of only a few program outputs and inputs in the DEA equation. It is recognized that in actuality, there are additional costs and benefits that would necessarily be included before running the analysis to examine cost effectiveness. Keep in mind that the illustrations that follow are primarily conceptual; and therefore, simplistic in the inclusion of outputs and inputs for optimal clarity of the power of a DEA analysis.

**DEA Healthcare Illustration-1 (Outpatient Mental Health Crisis Counseling Service)**

There are two outpatient mental health crisis counseling programs in this illustration (agency A and B). Agency A is located in a large urban city with over half a million in population. Agency B is a rural service provider located 50 miles away from the next urban population center and serves a population of 15,000. Both entities provide the two main services of crisis counseling (one-time acute counseling intervention to prevent a suicide or homicide), and brief counseling stabilization services of up to a one-month duration to prevent a situation from escalating to a point where suicide or homicide is imminent. The outputs in this illustration are units of crisis counseling (hours), and units of brief counseling stabilization services (hours). The inputs are total number of full-time employees (FTEs), total number individuals served, and total expenditures in dollars to support the delivery of these two services for a three-month comparison period.

Here is the DEA output/input equation for agency A (urban provider) (This is a hypothetical example with dummy data and normally the DEA software would run the analysis and produce the output tables.):

$$\frac{(\text{crisis counseling hours}) \times (\text{brief counseling stabilization hours})}{(\text{total # of FTEs}) \times (\text{total # of individuals served}) \times (\text{total expenditures for the three services})} = \frac{(600) \times (15,000)}{(8) \times (30) \times ($50,000)$} = 75 \text{ cost effective out of a possible 1.0 lota score}$$

For agency B, please see the DEA output below (rural provider) (This is a hypothetical example with dummy data and normally the DEA software would run the analysis and produce the output tables.):

$$\frac{(\text{crisis counseling hours}) \times (\text{brief counseling stabilization hours})}{(\text{total # of FTEs}) \times (\text{total # of individuals served}) \times (\text{total expenditures for the three services})} = \frac{(100) \times (300)}{(1) \times (2) \times ($17,500)$} = 86 \text{ cost effective out of a possible 1.0 lota score}$$ For agency A, the Iota score is .75. The percentage modification needed among all outputs and inputs for optimum cost effectiveness is .25 or 25%. Modifications of specific variables in the output/input equation for DEA are typically called "slack" variables [13, 18]. There are two options for specific slack variable adjustments to make this outpatient community mental health crisis counseling program (agency A) optimally cost effective. There could be an increase in the outputs totaling 3,000,000 in either or across both the units of crisis counseling, and brief counseling stabilization services or a decrease in one or any combination of the three inputs again totaling 3,000,000. Keep in mind that this sense of optimum is relative to the other comparison entity (agency B), and there is usually at least several other healthcare service entities/DMUs to comprise an actual cohort/group of comparators as opposed to a binary comparison like in this illustration.

For agency B, the Iota score is .86. This means the percentage change needed among all outputs and inputs for optimum cost effectiveness is .14 or 14%. Modification of outputs and/or inputs would need to yield a total of 5000 more output units. Keeping in mind that in a real analysis, one would run a DEA cost effectiveness analysis with a group of comparison entities (e.g., five or more comparison entities/DMUs to comprise an actual cohort). The DEA output typically identifies the best targets for slack variable (output or input variable) adjustments.

DEA Healthcare Illustration–2 (Comparing the cost effectiveness of two locations of an Occupational Therapy practice group for three months with one delivering in-person services and the other delivering services via telehealth)

The occupational therapy (OT) practice group in location C delivers in-person services with the practice group in location D delivering services using telehealth technology. The outputs are talk-based primary intervention services and family/home/work support consultative services (both measured in hours of service). The inputs are salary costs, non-salary overhead costs associated with operating an occupational therapy practice, and the cost of the purchase of specialized supplemental clinical services outside of the occupational therapy practice, and all three are measured in dollars. The timeframe for measuring cost effectiveness for both practice locations/service delivery modalities is three months.

Here is the DEA output/input equation for practice group location C, OT services delivered in a traditional in-person/face-to-face manner (This is a hypothetical example with dummy data and normally the DEA software would run the analysis and produce the output tables.):

$$\frac{(10,000) \times (11,000)}{(2,000) \times (1,000) \times (\$370)} = 0.15 \text{ cost effective out of a possible 1.0 Iota score}$$

For practice group location D, please see the DEA output below, OT services delivered via telehealth (This is a hypothetical example with dummy data and normally the DEA software would run the analysis and produce the output tables.):

$$\frac{(11,000) \times (10,000)}{(2,100) \times (150) \times (\$690)} = 0.51 \text{ cost effective out of a possible 1.0 Iota score}$$

For OT practice location C, the Iota score is .15. The percentage modification needed among all outputs and inputs for optimum cost effectiveness is .85 or 85%. There are two options for specific slack variable adjustments to make this in-person OT practice more cost effective and that would be to modify the output/input ratio by a total of 630,000,000 units by either increasing the outputs, decreasing the inputs, or some combination of the two. Keep in mind that for a real DEA analysis, there would be a cohort of healthcare service providers/DMUs/comparators rather than this binary comparison in this illustration.

For OT practice location D, the Iota score is .51. The percentage modification needed among all outputs and inputs for optimal cost effectiveness is .49 or 49%. That is, the total output/input ratio change in slack variables required to approach optimum cost effectiveness is 107,350,000 units by either increasing the outputs, decreasing the inputs, or some combination thereof. The DEA output typically identifies the best targets for slack variable (output or input variable) changes. Again, a real DEA analysis would include an actual cohort of healthcare service providers/DMUs/comparators instead of this binary comparison.

Beyond the two aforementioned illustrations, here are several conceptual DEA output/input equations for measuring cost effectiveness in various other healthcare services as additional examples:

• Hospital-Based Pre-Discharge Planning Service (Outputs/Inputs) Outputs (number of discharge placement options identified) x (number of discharge placement options that align with patient’s interest) Inputs (cost of social worker’s salary) x (cost of non- personnel overhead to support the pre-discharge planning operations)
• Healthcare Case Management Service (Outputs/ Inputs) Outputs (number of service linkages made) x (number of advocacy encounters) x (number of new resources identified) Inputs (cost of case manager’s salary) x (cost of non-personnel overhead to support case management operations) x (cost of transportation for client)