Learning by Doing

Learning by doing is viewed as an important determinant of success for many professions requiring high skill. Over the years, researchers have come to realize that teams and firms can also exhibit learning by doing. Even in cases where annual output does not increase over time, a firm can experience reductions in unit costs or improvements in quality that cannot be attributable to economies of scale, but cumulative experience instead. The presence of learning can have important implications for overall growth in a nascent industry. Differential learning across workers and firms can also have important implications for competition in the market. Health economists have been particularly interested in learning, because current and emerging medical technologies are complex, requiring both individual and team-based skills which are likely to benefit from experience.

Social scientists have been examining the impact of learning by doing on production technology for several decades. The concept of a learning curve was first described in 1936, when a study determined that as the quantity of manufactured units doubled, the number of direct labor hours required to produce an individual unit decreased at an uniform rate (Wright, 1936). Another early study concluded that the aircraft industry`s rate of learning, or reduced labor requirement, was 80% between doubled quantities of airframes (Alchian, 1963).

The standard equation that is used in the literature to characterize a learning curve takes the form:

where y represents the resources (hours or costs) required to produce the ith unit, a is the amount of resources required to produce the first unit, x is the cumulative number of units produced through the current time period i, and β is the learning rate (Argote, 1999). Taking natural logs yields a regression that can be readily estimated:

Most economic studies of learning by doing have employed this general framework by estimating the effectiveness of cumulative output production in reducing average costs. For example, it has been determined that on average each doubling of plant scale was accompanied by a 11% reduction in unit costs in the chemical industry (Lieberman, 1984). In the health economics literature, learning by doing has been tested for insurance plans, hospitals, and doctors. One study determined that clinical costs decline 10–15% with each doubling of experience for insurers administering managed behavioral health plans (where experience is measured as the cumulative number of managed care claims processed in a state by a particular health plan) (Sturm, 1999). However, most health economic studies have attempted to measure the effects of cumulative experience on patient outcomes (primarily mortality) rather than unit costs.

Hospital-Level Studies

Hospital-level studies of learning by doing have examined specific complex operations or procedures performed on patients. These studies focus on specific procedures in order to control for heterogeneity across treatments provided in hospitals. Patient outcomes (mostly patient mortality) are the dependent variable of interest, and regressions include both cumulative output and annual output as explanatory variables. Cumulative output is hypothesized to represent learning by doing, whereas annual output is hypothesized to reflect economies of scale. Because many of these studies do not have access to patient data dating back to when a particular operation was initially introduced for medical care, these studies often proxy for cumulative output using lagged values (e.g., number of operations performed 1, 2, or 3 years ago at a particular hospital) as a proxy for cumulative output.

Most published studies of learning by doing at the hospital level are based on two procedures for heart disease: Coronary artery bypass graft (CABG) surgery, and percutaneous transluminal coronary angioplasty (PTCA) (Gaynor et al., 2005; Ho, 2002; Pisano et al., 2001; Sfekas, 2009). CABG is a form of open heart surgery in which the rib cage is opened and a section of a blood vessel is grafted from the aorta to the coronary artery. PTCA is a procedure performed to improve blood supply to the heart. A balloon-tipped catheter is inserted into an artery in the groin or shoulder and threaded to the blocked artery. The balloon is then inflated to flatten atherosclerotic plaque against the artery wall, reopening the artery. Health economists have focused on these procedures, because heart disease is the leading cause of death in the United States. Therefore, these procedures are performed frequently by many hospitals, so that data are readily available. The data are usually derived from one or multiple US states, which allow researchers access to detailed data from hospital discharge abstracts for all admissions for several years. These data specifications are required, so that researchers can accurately count the cumulative and annual number of procedures performed for each hospital. CABG and PTCA have also been the focus of interest for learning by doing studies, because there is a large body of medical literature that specifies the information that is necessary to control for patient characteristics that influence patient mortality and other outcomes for these two procedures. The required variables, which include multiple demographic and clinical characteristics, are also available in hospital discharge datasets.

Multiple studies find no support for learning by doing at the hospital level for either CABG or PTCA. Lagged volume or cumulative volume tends not to be statistically significant in explaining mortality for patients who undergo these procedures. This conclusion has been found for studies analyzing data from Arizona, California, and Maryland for various sample periods spanning the years 1983 through 2001 (Gaynor et al., 2005; Ho, 2002; Sfekas, 2009). All of these studies include hospital specific fixed effects (dummy variables for each hospital in the sample) in the regression specifications in order to control for unobserved heterogeneity across hospitals which are constant over time. For example, some hospitals may benefit from exceptional and long-tenured nursing staff, or highly talented administrative staff. These factors can influence patient outcomes, but they are not observable in hospital discharge abstracts. The inclusion of hospital fixed effects means that the regressions estimate the effects of increases or decreases within the hospital in procedure volume over time, rather than the effects of differences in cumulative volume across hospitals on patient mortality. The fixed effect specification will more accurately capture the learning by doing effect which is hypothesized in the underlying economic model. However, precise estimation requires a sample of hospitals with data from a sufficient number of time periods in order to observe significant variation in procedure volume across time. Thus, these prior studies may have failed to precisely estimate a learning by doing effect. The samples in these studies contained data from only one or two states. Samples of this size may not have enough hospitals that experienced noticeable changes in procedure volume across years.

One study of 16 institutions that began performing a new procedure for minimally invasive heart surgery found that the amount of time required to perform the operation declined as the cumulative number of procedures increased (Huckman and Pisano, 2006). This result is tangible, even with the inclusion of hospital fixed effects in the regression models. The patient-level data were collected during the first 2 years after which the procedure was first approved by the Food and Drug Administration. Thus, this study may have been able to detect an institution-specific learning by doing effect, because the analysis was performed just after the technology was introduced; when the greatest amount of learning most likely occurs.

Although health economists have had little success identifying a tangible effect of cumulative volume on patient outcomes, a large literature in both medical and health services research journals finds a significant association between procedure volume and patient mortality. In addition to CABG and PTCA, this ‘volume-outcome’ relationship has been documented for a wide range of procedures and treatments, including carotid endarterectomy, hip replacement, lung cancer resection, liver transplantation, and neonatal intensive care (Halm et al., 2003; Luft et al., 1979; Birkmeyer et al., 2002). When this relationship was first identified in the medical literature, learning by doing was mentioned as a likely explanation for this finding. Researchers suggested that if experience was the underlying source of the volume-outcome effect, then complex operations should be ‘regionalized,’ so that patients would benefit from improved outcomes at a select number of facilities that would be able to gain greater experience. The absence of a significant effect of cumulative volume on patient mortality for CABG and PTCA casts doubt on the learning by doing hypothesis, particularly for common cardiac procedures.

One other challenge faced by health economists trying to identify a learning by doing effect is that cumulative volume and annual volume are highly correlated. Hospitals that perform a large number of procedures in 1 year tend to do so in subsequent years. In at least one instance, an analysis of hospital data for PTCA could not explicitly test for the effect of learning by doing on average costs per patient, because inclusion of both cumulative and annual volume as explanatory variables led to multicollinearity (Ho, 2002). This issue might be resolved if researchers were able to analyze data from multiple states simultaneously, with data stretching over many years. Gathering a much larger sample would increase the likelihood that one could find hospitals that experienced sufficient variation in volume (e.g., due to entry or exit of competitors), which would weaken the collinearity between cumulative and annual volume.

It is also interesting to note that learning by doing studies in the economics literature have tended to focus on patient outcomes as the dependent variable of interest rather than costs. This focus contrasts with the general industrial organization literature, where there is much less research on the relationship between learning by doing and product quality. Some research has analyzed data on nuclear power plants (Lester and McCabe, 1993). Both reactor-specific learning and spillovers across reactors have been found to be important determinants of nuclear reactor performance. Learning by doing as measured by cumulative output has also been associated with fewer complaints in the aircraft production industry (Argote, 1993).

There may be fewer studies of the effect of learning by doing on costs in the health economics literature, because it is difficult to obtain datasets that provide both detailed information on patient outcomes and the costs of care. Hospital discharge abstracts often contain information on the total charges for a patient admission. These data can be linked with hospital cost reports that contain the cost-to-charge ratio for each hospital, so that an estimate of costs per patient admission can be calculated. However, the saliency of patient mortality as a dependent variable of interest may have led to the greater focus of learning by doing studies on health outcomes for patients.

Physician-Level Studies

Many fewer published studies have attempted to estimate the volume-outcome effect at the surgeon level. The lack of studies stems in part from the fact that it is difficult to identify hospital datasets that provide consistent identifiers of physicians across patients and time. Only one published study included cumulative surgeon volume as an explanatory variable to explain patient mortality for CABG, and it finds no evidence of learning by doing (Huesch, 2009). In fact, this study also finds no association between annual surgeon volume and patient outcomes, although a small number of studies in the medical literature find that surgeons who perform more complex operations achieve lower mortality rates. Another study of approximately 4000 patients who received LASIK surgery in the early 2000s in the country of Colombia also found no effect of cumulative surgeon procedure volume on patient outcomes (Contreras et al., 2011). The presence of learning by doing effects at the hospital versus the individual doctor level is likely to vary by medical intervention. For some operations, the surgeon0 s technical skill and discretion over specific intraoperative processes are likely important determinants of patient outcome. In other operations, hospital-based services (intensive care, pain management, respiratory care, and nursing care) are more likely to determine inpatient mortality.

Endogeneity

One may be concerned that the absence of a learning by doing effect may reflect endogeneity in the volume-outcome effect. There may be factors that are unobservable to the researcher, which influence both procedure volume and patient outcomes, leading to an observed association between these two variables in a regression model. For example, some facilities may be more quick to invest in newer surgical devices, which allow them to treat more patients and achieve better outcomes simultaneously. Endogeneity may also result from selective referral. The reputation of higher quality hospitals or surgeons may become well known in the community, attracting more patients seeking care.

Some learning by doing studies have accounted for potential endogeneity using instrumental variables techniques (Gaynor et al., 2005). The variables that are hypothesized to influence procedure volume but are otherwise uncorrelated with patient outcomes include: The number of patients residing within a fixed geographical radius of a hospital, the number of other hospitals offering the same procedure within a fixed geographical radius of a hospital, and the predicted number of patients to choose a hospital for treatment, based on distance from the patients0 residences to each particular hospital. These instruments are significant predictors of patient volume, but specification tests cannot reject the null hypothesis that procedure volume is exogenous in explaining patient outcomes. Therefore, concerns regarding the potential endogeneity of procedure volume are not supported by current empirical analyses.

Forgetting

The general industrial organization literature has also tested for the presence of forgetting in firm production (Benkard, 2000; Thompson, 2007). This literature considers the possibility that productivity gains from learning can depreciate over time. More flexible regression specifications capture the fact that cost per unit of output can rise during significant production troughs that may occur in the life cycle of a product. Only one published paper has attempted to test for forgetting in the health economics literature, and it found almost complete forgetting from prior experience among recently trained surgeons performing CABG (Huesch, 2009). More studies need to be performed to validate this finding. The industrial organization literature identified forgetting in the context of airplane manufacturing, where there can be noticeable declines in production in the life cycle of a particular model of airplane. In contrast, most hospitals are not likely to experience noticeable troughs in the performance of a procedure. It would be useful to identify a large sample of hospitals that had experienced the entrance of a nearby competitor for the same procedure to precisely estimate a forgetting effect. Determining the extent to which forgetting exists in the performance of complex medical treatments has important implications for patient care. If there is little depreciation in learning, then one can be more certain that hospitals or surgeons who are currently high quality will remain so in the future. If forgetting does exist, further studies would be needed to determine why learning depreciates. Quality could depreciate over time, because the skill set of surgeons could depreciate with lack of use, or because multidisciplinary teams of caregivers become less coordinated if they treat fewer patients.

Other Forms Of Learning

Past industrial organization studies have also identified forms of learning other than learning by doing. For instance, in a study of the semiconductor industry, firms learn three times more from an additional unit of their own cumulative production than from an additional unit of another firm’s cumulative production (Irwin and Klenow, 1994). The reductions in unit costs associated with increases in other firms’ cumulative production or industry cumulative output are referred to as spillover effects. In this context, a firm`s own learning by doing is referred to as proprietary learning. It is plausible that spillover learning could occur in the context of complex medical procedures. For example, a hospital performing a small volume of procedures in a city with several large facilities nearby may have better outcomes than a comparatively small facility in a rural area. The small urban hospital may be able to benefit from nearby expertise.

Cost reductions associated with calendar time rather than production quantity have been referred to as ‘learning by watching.’ Hospitals may be able to improve outcomes by learning from the experience of other facilities. For example, a hospital which began performing 50 PTCAs per year in 1996 is likely to have better outcomes than a comparable hospital in 1986, because the former facility could benefit from the knowledge and experience gained over the previous decade. One study that found little evidence of learning by doing based on the cumulative number of PTCAs performed by hospitals over time found substantial evidence of learning by watching for this procedure (Ho, 2002). Outcomes improved year by year for all hospitals, regardless of the cumulative number of angioplasty procedures they performed. Learning by watching has also been identified for the performance of LASIK (Contreras et al., 2011). Significant improvements in outcomes were observed at two points in the sample period analysis when all physicians in a practice performing this procedure met to update surgical plans based on patient characteristics.

Determining the relative magnitude of learning by doing, spillover learning, and learning by watching is important for assessing the relative success of small versus large firms. If most learning is nonproprietary and few economies of scale exist, then small firms can more easily compete with large firms. Health economics lacks a comprehensive set of studies that test for learning by doing for a range of procedures and for hospitals or physicians in multiple states. The studies so far find little evidence for learning by doing, whereas there is more convincing evidence for learning by watching. These findings suggest that there is little support for ‘regionalizing’ complex surgical procedures at a select number of high volume hospitals that would benefit from greater experience.

Conclusion

Researchers have identified learning by doing that reduced unit costs in industries ranging from chemical processing to semiconductors. And there are hundreds of papers in the medical literature finding an association between higher hospital or surgeon procedure volume and lower mortality rates. However, most rigorous econometric analyses of health care data have been unable to formally identify learning by doing. Perhaps health economists lack sufficient data to distinguish between annual and cumulative output measures when testing for learning by doing in mortality and/or costs. Analysis of a wider range of newly emerging medical treatments, as well as more detailed data on costs would help to explain the role of learning in influencing the costs and quality of medical care.

In the meantime, policy makers should be cautious of recommendations to centralize complex surgical procedures based on existing volume-outcome studies. Although larger providers tend to yield better patient outcomes, making them even larger will not likely lower hospital mortality rates further. More research is required to determine the underlying reasons for the volume-outcome relationship. One should also keep in mind that learning by watching effects appear to be significant in health care. All providers tend to improve over time, regardless of volume. Given the potential beneficial effects of competition in maintaining quality and lower costs, patients may in fact be better off without centralization of complex treatment.

References:

1. Alchian, A. (1963). Reliability of progress curves in airframe production. Econometrica 31(4), 679–693.
2. Argote, L. (1993). Group and organizational learning curves: Individual, system and environmental components. British Journal of Social Psychology 32, 31–51.
3. Argote, L. (1999). 1st Organizational learning: Creating, retaining and transferring knowledge. Norwell, MA: Kluwer Academic Publishers.
4. Benkard, C. L. (2000). Learning and forgetting: The dynamics of aircraft production. American Economic Review 90(4), 1034–1054.
5. Birkmeyer, J. D., Siewers, A. E., Finlayson, S. R., et al. (2002). Hospital volume and surgical mortality in the United States. New England Journal of Medicine 346(15), 1128–1137.
6. Contreras, J. M., Kim, B. and Tristao, I. M. (2011). Does doctors’ experience matter in lasik surgeries? Health Economics 20(6), 699–722.
7. Gaynor, M., Seider, H. and Vogt, W. B. (2005). The volume-outcome effect, scale economies, and learning-by-doing. American Economic Review 95(2), 243–247.
8. Halm, E. A., Chassin, M. R., Tuhrim, S., et al. (2003). Revisiting the appropriateness of carotid endarterectomy. Stroke. 34(6), 1464–1471.
9. Ho, V. (2002). Learning and the evolution of medical technologies: The diffusion of coronary angioplasty. Journal of Health Economics 21(5), 873–885.
10. Huckman, R. S. and Pisano, G. P. (2006). The firm specificity of individual performance: Evidence from cardiac surgery. Management Science 52(4), 473–488.
11. Huesch, M. D. (2009). Learning by doing, scale effects, or neither? Cardiac surgeons after residency. Health Services Research 44(6), 1960–1982.
12. Irwin, D. A. and Klenow, P. J. (1994). Learning-by-doing spillovers in the semiconductor industry. Journal of Political Economy 102(6), 1200–1227.
13. Lester, R. K. and McCabe, M. J. (1993). The effect of industrial structure on learning by doing in nuclear power plant operation. RAND Journal of Economics 24(3), 418–438.
14. Lieberman, M. B. (1984). The learning curve and pricing in the chemical processing industries. RAND Journal of Economics 15(2), 213–228.
15. Luft, H. S., Bunker, J. and Enthoven, A. (1979). Should operations be regionalized? An empirical study of the relation between surgical volume and mortality. The New England Journal of Medicine 301(25), 1364–1369.
16. Pisano, G. P., Bohmer, R. M. J. and Edmondson, A. C. (2001). Organizational differences in rates of learning?: Evidence from the adoption of minimally invasive cardiac surgery. Management Science 47(6), 752–768.
17. Sfekas, A. (2009). Learning, forgetting, and hospital quality: An empirical analysis of cardiac procedures in Maryland and Arizona. Health Economics 18(6), 697–711.
18. Sturm, R. (1999). Cost and quality trends under managed care: Is there a learning curve in behavioral health carve-out plans? Journal of Health Economics 18(5), 593–604.
19. Thompson, P. (2007). How much did the liberty shipbuilders forget? Management Science 53(6), 908–918. Available at: http://mansci.journal.informs.org/cgi/doi/10.1287/mnsc.1060.0678 (accessed on 19 February 2013).
20. Wright, T. P. (1936). Factors affecting the cost of airplanes. Journal of Aeronautical Sciences 3(4), 122–128.
21. Halm, E. A., Lee, C. and Chassin, M. R. (2002). Is volume related to outcome in health care? A systematic review and methodologic critique of the literature. Annals of Internal Medicine 137(6), 511–520.
22. Huesch, M. D. and Sakakibara, M. (2009). Forgetting the learning curve for a moment: How much performance is unrelated to own experience? Health Economics 18(7), 855–862.