| Home | > | Outcomes Measurement in the Human Services, 2nd Edition |
|
| ||||
 |
Outcomes Measurement in the Human Services, 2nd Edition
Cross-Cutting Issues and Methods in the Era of Health Reform
Jennifer L. Magnabosco and Ronald W. Manderscheid, Editors
ISBN: 978-0-87101-422-1. 2011. Item #4221. 424 pages.
ISBN: 978-0-87101-422-1. 2011. Item #4221. 424 pages.
NASW Members
Save 10% (print edition): $44.09
You MUST enter member number to receive member discount at checkout.
Outcomes Measurement in the Human Services: Cross-Cutting Issues and Methods in the Era of Health Reform, 2nd Edition, updates the original volume released in 1997. The second edition’s 25 chapters are divided into five sections:
Each chapter includes discussions and recommendations for how outcomes measurement can more effectively inform research, practice, policy, and advocacy agendas so that the diversity of human service needs in the United States can be met fairly and with cultural sensitivity.
The book provides readers with both macro- and micro-perspectives on the topics of outcomes measurement; incorporates practice, policy, and research perspectives; and examines current and long-standing issues within the human services field with regard to outcomes and performance measurement. With numerous authors from the field of social work, this new edition of Outcomes Measurement in the Human Services also boasts contributions from professionals in the fields of psychology, psychiatry, policy research, administration, and numerous other social sciences. With its thorough, authoritative coverage of the issues of outcomes measurement and accountability, this book is appropriate for and directly relevant to courses in evaluation research, clinical practice, social policy, and administration.
- Overarching Issues and Methods
- Health
- Mental and Behavioral Health
- Child and Family Services
- Special Topics in the Human Services
Each chapter includes discussions and recommendations for how outcomes measurement can more effectively inform research, practice, policy, and advocacy agendas so that the diversity of human service needs in the United States can be met fairly and with cultural sensitivity.
The book provides readers with both macro- and micro-perspectives on the topics of outcomes measurement; incorporates practice, policy, and research perspectives; and examines current and long-standing issues within the human services field with regard to outcomes and performance measurement. With numerous authors from the field of social work, this new edition of Outcomes Measurement in the Human Services also boasts contributions from professionals in the fields of psychology, psychiatry, policy research, administration, and numerous other social sciences. With its thorough, authoritative coverage of the issues of outcomes measurement and accountability, this book is appropriate for and directly relevant to courses in evaluation research, clinical practice, social policy, and administration.
Foreword
Acknowledgements
Chapter 1: Introduction
Jennifer L. Magnabosco and Ronald W. Manderscheid
Chapter 2: Outcomes Measurement in the Human Services: Lessons Learned from Public and Private Sectors
Harry P. Hatry
Chapter 3: Qualitative Approaches to Outcomes Evaluation
Michael Quinn Patton and Jean Gornick
Chapter 4: Quantitative Approaches to Outcomes Measurement and Evaluation: A Commentary
Brian T. Yates
Chapter 5: Outcomes Measurement in the Human Services: The Role of Evidence-based Practice
Bruce A. Thyer
Chapter 6: Building Capacity for Outcomes Measurement in Human Service Organizations: The Getting to Outcomes Method
Joie Acosta and Matt Chinman
Chapter 7: Leadership in the Human Services: Models for Outcomes-driven Organizational Cultures
Thomas Packard and Richard H. Beinecke
Chapter 8: The Patient-centered Medical Home: Implications for Behavioral Health and the Measurement of Integrated Care
John Bartlett
Chapter 9: Health and Quality-of-Life Outcomes: The Role of Patient-reported Measures
Dasha Cherepanov and Ron D. Hays
Chapter 10: Outcomes Measurement and Gerontology: What the United States Needs as It Grows Older
Barbara Berkman and Daniel B. Kaplan
Chapter 11: Complementary Alternative, and Integrative Medicine: Current Challenges for Outcomes Measurement
Ian D. Coulter and Raheleh Khorsan
Chapter 12: Research-to-Practice Example: Screening and Outcomes Assessments in Cancer Care
James R. Zabora
Chapter 13: New Directions for Outcome-oriented Mental Health System Transformation
Timothy A. Kelly
Chapter 14: Toward Integrated Outcome Measurement for Mental Health and Substance Use Services and Systems
James Siemianowski and Thomas A. Kirk Jr.
Chapter 15: Rapid Assessment Instruments as Outcomes Measures: Past and Current Uses
Kevin Corcoran and Nikki Hozack
Chapter 16: Outcomes Measurement and the Mental Health Consumer Movement: Research, Recovery and Recognition in the Human Services
Jean Campbell
Chapter 17: Research-to-Practice Example: Not by Outcomes Alone - Using Peer Evaluation to Ensure Fidelity to Evidence-based Assertive Community Treatment Practice
Gregory B. Teague and Maria Monroe-DeVita
Chapter 18: Outcomes Measurement for Child and Family Services: Successful Approaches for Practice, Management, and Policy
Rose M. Ethridge, Cynthia Klein, Jacqui LaCoste, and Julie T. Marks
Chapter 19: Cost and Outcomes Analysis of Child Well‑Being
M. Rebecca Kilburn
Chapter 20: Practice-to-Research Example: Implementation of Child and Family Services Reviews: Progress and Challenges
Brenda G. McGowan and Elaine M. Walsh
Chapter 21: The Veterans Health Administration: Lessons Learned from Implementing Performance Measurement, Electronic Health Records, and Evidence-based Practice
Joseph Francis
Chapter 22: Measuring Outcomes in Criminal Justice
Susan Turner
Chapter 23: U.S. Faith-based Human Services: The Current State of Outcomes Assessment
Richard Flory, Hebah Farrag, Brie Loskota, and Tobin Belzer
Chapter 24: Social, Health, and Mental Health Outcomes: Global Linkages
Vijay Ganju
Chapter 25: Concluding Comments
Jennifer L. Magnabosco and Ronald W. Manderscheid
Acknowledgements
Chapter 1: Introduction
Jennifer L. Magnabosco and Ronald W. Manderscheid
PART I: Overarching Issues and Methods
Chapter 2: Outcomes Measurement in the Human Services: Lessons Learned from Public and Private Sectors
Harry P. Hatry
Chapter 3: Qualitative Approaches to Outcomes Evaluation
Michael Quinn Patton and Jean Gornick
Chapter 4: Quantitative Approaches to Outcomes Measurement and Evaluation: A Commentary
Brian T. Yates
Chapter 5: Outcomes Measurement in the Human Services: The Role of Evidence-based Practice
Bruce A. Thyer
Chapter 6: Building Capacity for Outcomes Measurement in Human Service Organizations: The Getting to Outcomes Method
Joie Acosta and Matt Chinman
Chapter 7: Leadership in the Human Services: Models for Outcomes-driven Organizational Cultures
Thomas Packard and Richard H. Beinecke
PART II: Outcomes Measurement in Health
Chapter 8: The Patient-centered Medical Home: Implications for Behavioral Health and the Measurement of Integrated Care
John Bartlett
Chapter 9: Health and Quality-of-Life Outcomes: The Role of Patient-reported Measures
Dasha Cherepanov and Ron D. Hays
Chapter 10: Outcomes Measurement and Gerontology: What the United States Needs as It Grows Older
Barbara Berkman and Daniel B. Kaplan
Chapter 11: Complementary Alternative, and Integrative Medicine: Current Challenges for Outcomes Measurement
Ian D. Coulter and Raheleh Khorsan
Chapter 12: Research-to-Practice Example: Screening and Outcomes Assessments in Cancer Care
James R. Zabora
PART III: Outcomes Measurement in Mental and Behavioral Health
Chapter 13: New Directions for Outcome-oriented Mental Health System Transformation
Timothy A. Kelly
Chapter 14: Toward Integrated Outcome Measurement for Mental Health and Substance Use Services and Systems
James Siemianowski and Thomas A. Kirk Jr.
Chapter 15: Rapid Assessment Instruments as Outcomes Measures: Past and Current Uses
Kevin Corcoran and Nikki Hozack
Chapter 16: Outcomes Measurement and the Mental Health Consumer Movement: Research, Recovery and Recognition in the Human Services
Jean Campbell
Chapter 17: Research-to-Practice Example: Not by Outcomes Alone - Using Peer Evaluation to Ensure Fidelity to Evidence-based Assertive Community Treatment Practice
Gregory B. Teague and Maria Monroe-DeVita
PART IV: Outcomes Measurement in Child and Family Services
Chapter 18: Outcomes Measurement for Child and Family Services: Successful Approaches for Practice, Management, and Policy
Rose M. Ethridge, Cynthia Klein, Jacqui LaCoste, and Julie T. Marks
Chapter 19: Cost and Outcomes Analysis of Child Well‑Being
M. Rebecca Kilburn
Chapter 20: Practice-to-Research Example: Implementation of Child and Family Services Reviews: Progress and Challenges
Brenda G. McGowan and Elaine M. Walsh
Part V: Special Topics in Outcomes Measurement in the Human Services
Chapter 21: The Veterans Health Administration: Lessons Learned from Implementing Performance Measurement, Electronic Health Records, and Evidence-based Practice
Joseph Francis
Chapter 22: Measuring Outcomes in Criminal Justice
Susan Turner
Chapter 23: U.S. Faith-based Human Services: The Current State of Outcomes Assessment
Richard Flory, Hebah Farrag, Brie Loskota, and Tobin Belzer
Chapter 24: Social, Health, and Mental Health Outcomes: Global Linkages
Vijay Ganju
Chapter 25: Concluding Comments
Jennifer L. Magnabosco and Ronald W. Manderscheid
Much has transpired since the first edition of Outcomes Measurement in the Human Services appeared in 1997. In this second edition, Jennifer Magnabosco and Ron Manderscheid have performed a valuable service to the fields of health, human services, and social work by reconvening many of the original contributors to the 1997 edition and asking them to reflect on how outcomes measurement has changed and evolved over the past nearly 15 years.
The 1997 edition appeared at a time when health, human services, and social work were still dealing with the relatively unknown concept of outcomes. Consequently, the contributors to the first edition had to deal with such issues as: What is an outcome? How do you measure outcomes? Why should programs and agencies measure outcomes? At the time, there was little agreement on the relationship of outcomes and outcomes measurement to the emerging concept of government performance measurement as mandated by the federal Government Performance and Results Act of 1993. In addition, there was both concern and suspicion about outcomes and government performance measurement on the part of the scholarly evaluation community.
This second edition of Outcomes Measurement in the Human Services appears at a time when outcomes and outcomes measurement has become a major field of scholarly research and practice in health, human services, and social work. The chapters in this second edition make some important contributions to conceptualizing and defining outcome measures, and address a more nuanced set of issues that reflect the maturation of the field. The book helps to set the outcomes agenda for health, human services, and social work for the next decade.
For many of the issues identified by the contributing authors, there seems to be both an upside and a downside. Many funding sources, such as the United Way, foundations, most government agencies, and even whole countries now mandate the collection of outcome data, leading to the creation of large and potentially meaningful data sets. Yet there is a real concern on the part of providers, professionals, and consumers that funder-mandated quantitative outcomes infringe on their prerogatives. This practice raises a number of important subsidiary issues, including encroachment on the freedom of nonprofit human service organizations to experiment and innovate with new service delivery approaches. Likewise, evidence-based practice, focusing on greater attention to scientific considerations in decision making, is considered in some ways antithetical to the prescriptive specification of outcomes by funding sources. On another issue, qualitative research has demonstrated its effectiveness in producing valuable insights. A great deal of useful information is missed when only quantitative approaches to outcomes measurement are used, yet effective methods for incorporating qualitative findings have yet to be discovered.
One encouraging note is the recognition by several contributing authors of the many valuable outcome instruments that are now available in several fields, such as health, mental health, child and family services, substance use care, and others. The challenge now is not only to measure the accomplishment of outcomes, but also to better understand how and why they took place.
Another important topic addressed by several authors is the issue of linking outcomes, costs, and compensation. The ability of health, human services, and social work to compute their costs, specifically the costs of achieving outcomes, is increasingly important as government and private funding sources continue to move toward pay for performance by utilizing performance-based contracts and grants. With respect to the health field, outcomes and outcome measures in conjunction with evidenced-based practice hold the potential to focus not only on what works, but also at what cost. These trends have been codified, for example, in the Patient Protection and Affordable Care Act of 2010, which is intended to extend insurance coverage and improve the quality of care delivery.
The state of the art, as presented by the contributors to this volume, clearly demonstrates that significant progress in outcomes measurement has been made over the last 15 years. Yet many challenges still exist, and will continue to persist, as human service sectors embark upon major policy and practice reforms. It is clear that all stakeholders are not exactly on the same page. Funders, government agencies, nonprofit organizations and the general public all want to know that dollars spent for health, human services, and social work accomplish results, not just process. Those closer to consumers and direct service professionals want to be assured that treatment prerogatives are respected. This is all to say that the outcomes and performance measurement movements are perceived and judged on the basis of one’s location within the system, which in effect makes it piecemeal. What these movements need to become are well-integrated systems that continually contribute to making service delivery more efficient and effective while improving productivity and maximizing the use of technology. We hope that fulfilling this need can be part of the agenda for the next decade.
All in all, this second edition does an excellent job of bringing the topics of outcomes and performance measurement up to date! Readers will understand not only where the outcomes movement is today, but also how we got where we are.
Lawrence L. Martin
Professor of Public Affairs
University of Central Florida
Peter M. Kettner
Professor Emeritus
Arizona State University
The 1997 edition appeared at a time when health, human services, and social work were still dealing with the relatively unknown concept of outcomes. Consequently, the contributors to the first edition had to deal with such issues as: What is an outcome? How do you measure outcomes? Why should programs and agencies measure outcomes? At the time, there was little agreement on the relationship of outcomes and outcomes measurement to the emerging concept of government performance measurement as mandated by the federal Government Performance and Results Act of 1993. In addition, there was both concern and suspicion about outcomes and government performance measurement on the part of the scholarly evaluation community.
This second edition of Outcomes Measurement in the Human Services appears at a time when outcomes and outcomes measurement has become a major field of scholarly research and practice in health, human services, and social work. The chapters in this second edition make some important contributions to conceptualizing and defining outcome measures, and address a more nuanced set of issues that reflect the maturation of the field. The book helps to set the outcomes agenda for health, human services, and social work for the next decade.
For many of the issues identified by the contributing authors, there seems to be both an upside and a downside. Many funding sources, such as the United Way, foundations, most government agencies, and even whole countries now mandate the collection of outcome data, leading to the creation of large and potentially meaningful data sets. Yet there is a real concern on the part of providers, professionals, and consumers that funder-mandated quantitative outcomes infringe on their prerogatives. This practice raises a number of important subsidiary issues, including encroachment on the freedom of nonprofit human service organizations to experiment and innovate with new service delivery approaches. Likewise, evidence-based practice, focusing on greater attention to scientific considerations in decision making, is considered in some ways antithetical to the prescriptive specification of outcomes by funding sources. On another issue, qualitative research has demonstrated its effectiveness in producing valuable insights. A great deal of useful information is missed when only quantitative approaches to outcomes measurement are used, yet effective methods for incorporating qualitative findings have yet to be discovered.
One encouraging note is the recognition by several contributing authors of the many valuable outcome instruments that are now available in several fields, such as health, mental health, child and family services, substance use care, and others. The challenge now is not only to measure the accomplishment of outcomes, but also to better understand how and why they took place.
Another important topic addressed by several authors is the issue of linking outcomes, costs, and compensation. The ability of health, human services, and social work to compute their costs, specifically the costs of achieving outcomes, is increasingly important as government and private funding sources continue to move toward pay for performance by utilizing performance-based contracts and grants. With respect to the health field, outcomes and outcome measures in conjunction with evidenced-based practice hold the potential to focus not only on what works, but also at what cost. These trends have been codified, for example, in the Patient Protection and Affordable Care Act of 2010, which is intended to extend insurance coverage and improve the quality of care delivery.
The state of the art, as presented by the contributors to this volume, clearly demonstrates that significant progress in outcomes measurement has been made over the last 15 years. Yet many challenges still exist, and will continue to persist, as human service sectors embark upon major policy and practice reforms. It is clear that all stakeholders are not exactly on the same page. Funders, government agencies, nonprofit organizations and the general public all want to know that dollars spent for health, human services, and social work accomplish results, not just process. Those closer to consumers and direct service professionals want to be assured that treatment prerogatives are respected. This is all to say that the outcomes and performance measurement movements are perceived and judged on the basis of one’s location within the system, which in effect makes it piecemeal. What these movements need to become are well-integrated systems that continually contribute to making service delivery more efficient and effective while improving productivity and maximizing the use of technology. We hope that fulfilling this need can be part of the agenda for the next decade.
All in all, this second edition does an excellent job of bringing the topics of outcomes and performance measurement up to date! Readers will understand not only where the outcomes movement is today, but also how we got where we are.
Lawrence L. Martin
Professor of Public Affairs
University of Central Florida
Peter M. Kettner
Professor Emeritus
Arizona State University
In 1997, the year the first edition of Outcomes Measurement in the Human Services was published, the United States had just begun to implement more systematic accountability of the human services through legislation (such as the Government Performance and Results Act), more contemporary approaches to human services administration that were geared toward performance measurement (such as total quality management), and more stringent approaches to fiscal accountability through evolving strategies such as managed care. At the time of this second edition’s writing, our nation has once again just completed a thorough examination of how our health, mental health, and child and family services and systems are to be delivered and managed. Having just recently witnessed the passing of national health care reform, mental health parity, and new procedures to overhaul state child and welfare accountability systems, citizens in the United States are living in an unprecedented time of systemic change within our human services. Such historic change again directs much attention to human service outcomes and their measurement, and in the 21st century, new contexts for care such as prevention, consumer-, and recovery-driven (please see chapter 16 for a general definition) and strengthsbased (approaches to recovery that shift from a problem-focused, targeting of deficiencies perspective, to a focus on "elevating" and "magnifying strengths" perspective) care (Seligman, 2002; Seligman & Csikszentmihalyi, 2000).
This volume is intended to be in sync with information presented in the first edition. Both volumes:
Many authors have contributed chapters to both editions, and in this edition, several authors again provide social work perspectives on outcomes measurement in the three main human service areas covered: health, mental/ behavioral health, and child and family services. This volume’s design and content are also grounded in a set of guiding questions similar to those of the first edition:
Keeping these guiding questions in mind, authors in the second edition of Outcomes Measurement in the Human Services present an updated discussion of issues and methods in outcomes measurement (Mullen & Magnabosco, 1997). Where possible, and fitting, discussion regarding the effect that demands for accountability have had and will continue to have on social work practice is also included.
This introduction sets the overall context for the book, summarizing each chapter and providing a descriptor of its content in italics. Parts I through IV of the book model the first edition’s overall structure. Part I addresses general overarching issues and cross-cutting themes of outcomes measurement, and parts II, III, and IV examine outcomes measurement in three select fields of human services: health, mental and behavioral health, and child and family services. Whereas one section in the first edition featured practice-research examples, in this edition we provide three such examples, one in each of the fields of practice, to illustrate the development and/or application of outcomes measurement in real-life human service scenarios. Part V is a new section that features particular areas in the human services that cut across the three main fields of practice featured and are timely and representative of important domains of interest in the 21st century. We close the book with concluding comments and recommendations for future action.
Part I "examines the overarching issues and methods that apply to all human service disciplines" (Mullen & Magnabosco, 1997, p. xxi). The themes found in these beginning chapters are echoed and particularly contextualized by chapters in parts II-V.
Orientation to Outcomes and Performance Measurement (Chapter 2)
Hatry updates the current status of outcomes and performance measurement in the public and private sectors, discussing major recent activities that have shaped the direction of such measurement in the United States and internationally. In this chapter (as in the first edition), he distinguishes among types of outcomes measurement and approaches and discusses the need for human service organizations (HSOs) to incorporate outcomes management in everyday organizational activities. He questions whether the typically collected outcomes and performance data are valid and comprehensive and raises concerns about how such data are being used. He advocates for the need to balance and integrate regular outcomes measurement and program evaluation, especially in light of the need to address disparities (inequities), and discusses various strategies being used to effectively meet accountability demands from funders, government, specific communities, and the public at large.
Qualitative and Quantitative Approaches and Outcomes Evaluation (Chapters 3 and 4)
The next two chapters provide perspectives on qualitative and quantitative approaches to outcomes measurement and outcomes evaluation. This dual perspective echoes Hatry, highlighting the necessity of both approaches and their unique contributions to the development, implementation, monitoring, and provision of outcomes and performance measurement, as well as to accountability, practice, and research purposes. In chapter 3, Patton and Gornick discuss the development, trends, and impact that qualitative data – specifically, qualitative outcomes evaluations – have had in the outcomes measurement movement and in meeting demands for accountability in the human services. They illustrate the value that qualitative methods offer in a case example evaluation of a residence for persons with long-term alcohol and drug use conditions. Patton and Gornick conclude with comments on some of the major issues in the future of outcomes evaluation, including the need to develop a common language for types of evaluation and outcome measures, as well as how increasingly important concepts, such as strengths-based care and prevention, require understanding that there may be "different meanings in different contexts."
In chapter 4, Yates provides a commentary on quantitative approaches to outcomes measurement and evaluation. On the basis of his 35 years researching and evaluating HSO programs, he reflects on issues that are common to measuring outcomes quantitatively across HSOs, such as the idea that "reliability and validity of quantitative outcomes are not just based on science anymore," and the fact that the process of selecting and quantifying outcomes for, and by, funders is not standardized. He then discusses implications for outcomes measurement in this era of change, and offers the RAPOA (Resource-Activity-Process-Outcome Analysis) model as a tool that can help HSOs measure the range of outcomes related to improving services and systems.
Controversies in Evidence-based Practice and Outcomes (Chapter 5)
Thyer discusses controversies in the measurement of outcomes and evidence-based practice (EBP). He shares views that are critically needed during this time of turbulent change in the human services: He explains why it is important to pause and reconsider the fact that EBP is really a five-step process, and that EBP "does not tell clinicians and others involved with an EBP process what to do." He thoroughly discusses the relationship between the EBP process and outcomes measurement, how to locate and assess credible evidence, and various stakeholder approaches to outcomes measurement, as well as the implications different perspectives have for human services and EBP. He concludes with thoughts on how to balance approaches to outcomes measurement and EBPs and ends on an empowering note to consumer and advocacy groups, suggesting that they become more involved with promoting EBPs in service and government organizations.
A Process for "Getting to Outcomes" (Chapter 6)
Acosta and Chinman describe a method for HSOs to build capacity for outcomes called "Getting to Outcomes" (GTO). The authors discuss how the 10-step GTO model is "more than a process – [it] is an intervention that builds the capacity of users to use outcome data for continuous quality improvement, not just performance measurement." They illustrate the use of GTO in a current project designed to implement positive youth development programs by an adolescent coalition in Maine. They conclude with a discussion about barriers associated with measuring outcomes in HSOs and how GTO can effectively help navigate the development of capacity and outcomes.
Leadership and Outcomes Assessment (Chapter 7)
Packard and Beinecke discuss leadership development in the human services and the implications leadership can have for outcomes at multiple levels. They make it clear that human services leadership is facing a serious challenge over the next decade as many experienced human service professionals retire. They also review the main models of leadership, tenets of leadership versus management, organizational change, and pertinent studies that have shown links between leadership and outcomes in the human services. Packard and Beinecke offer a conceptual model for organizational change that can be used as a framework to develop and measure progress toward outcomes-based cultures in HSOs. They conclude with suggestions for promoting "change leaders" and outcomes measurement systems and cultures in HSOs.
In 1997, Mullen and Magnabosco wrote, "perhaps no field of practice of human service practice has been more affected by outcomes measurement than health." (p. xxviii) This remains true today.
Integrated Service Delivery (Chapter 8)
Bartlett describes a current solution to integrated healthcare – the patient-centered medical home – and the implications it has had during health care reform and for the United States in general. Chronicling the history of primary care delivery in this country and its relation to current health care reform tenets, Bartlett describes the challenges to providing cost-effective and needed care, and to measuring integrated health care outcomes between primary care and other sectors, especially mental health and substance use. He stresses the unmistakable need to integrate health and mental health and describes several strategies that may help to bring us closer to reasonable outcomes, such as implementing the "use of brief, valid, and reliable tools" (like Zabora in chapter 12 and Corcoran and Hozak in chapter 15) and "thinking globally but acting locally" (Geddes, 1915) – that is, measuring health outcomes at all levels, with an eye toward improved population health.
Patient-reported Measures (Chapter 9)
Cherepanov and Hays provide a comprehensive discussion of measures that are critical to monitoring health and care: patient-reported outcome measures, health-related quality-of-life measures, and patient evaluations of care. The authors discuss the various types of measures in detail and their applications to practice; in particular, they discuss the Consumer Assessment of Healthcare Providers and Systems (CAHPS) Program to illustrate patient evaluation of care measures. Like other authors in the book, Cherepanov and Hays chronicle the changes made in measurement and note the available resources. In their conclusion, they predict that patient-reported outcome measures may gain greater significance during health care reform.
Assessing Services for an Aging Population (Chapter 10)
Berkman and Kaplan discuss one of the most pressing health concerns the United States will face in the next decade: our aging population. They describe the characteristics of this population and how they will affect the context of health care delivery, regardless of health care reform. Berkman and Kaplan review the state of outcomes measurement in aging and provide a range of instruments and measures that are especially relevant and useful. Like other authors in the book, they highlight the role that social workers will have as the face of health care delivery is changed by this population’s needs and by health care reform.
Complementary and Alternative Medicine (CAM) (Chapter 11)
Coulter and Horsan discuss an aspect of health care that is widespread but not routinely included in mainstream discussions: the use of CAM and integrative medicine (IM). The authors explain the nature and modalities (e.g., acupuncture, chiropractic medicine) of CAM and IM, their paradigms, the controversies and challenges associated with integrating CAM and IM, and the challenges associated with conducting research and outcomes measurement. They conclude with recommendations and comments about useful approaches (e.g., observational studies) and theoretical models (e.g., systems theory), the principles upon which CAM outcomes measurement should be based (e.g., "elevate the notion of person-centered care and patient-centered care"), and an agenda for outcomes measurement in CAM that can be used for comparative effectiveness research and other health care reform efforts.
Research to Practice Example: Chronic Care and Psychological Distress (Chapter 12)
Zabora illustrates the importance of using appropriate and comprehensive screening and outcomes assessment tools to determine the range of needs chronic care patients, especially oncology patients, have during the course of care and over the longer term. Zabora focuses on screening and assessments for psychological distress – one of the most serious effects of chronic illness – in the context of hospital settings. This focus is of particular note because screening for such distress often goes unperformed, largely because of a lack of comprehensive tools and measures. He concludes with a discussion of barriers to such assessments, how particular members of the health care team – such as social workers – are better poised to take on this responsibility, and how such instruments and measures can be especially useful in health care reform efforts that seek quick and valid approaches.
The contributors to part III examine mental and behavioral health services outcomes measurement. As in the first edition, the terms mental health and behavioral health are sometimes used interchangeably when discussing outcomes measurement. In 1997, the term behavioral health typically was used to designate services that applied to people with substance abuse and mental health problems in a managed care context (Freeman & Trabin, 1994; Mullen & Magnabosco, 1997). Today, behavioral health carries a similar meaning, with two differences: First, substance use is used in place of substance abuse. Second, the overall meaning of substance use is not determined as much by the managed care context. Rather, it is informed by the fact that consumers, and the wide range of stakeholders in the fields of mental health and substance use, have come to accept a broader and more integrated approach to the spectrum of mental health and substance use care. Although the use of the terms "substance use" and "substance abuse" largely depend on ideology, practice, and societal context, the former represents recognition that persons with this disorder have a real illness and that the issues associated with this illness are multidimensional, taking into account the holistic individual, not just his or her drug using or drug "abusing." Hence, in the United States, we have more readily come to use the term substance use to "connote measurability or an emphasis on behavioral functioning" (Mullen & Magnabosco, 1997).
Transformation and Outcomes (Chapter 13)
Kelly examines mental health policy in the United States, factors leading to the recent emphasis on mental health transformation or reform, and recommendations for what is needed to transform the country’s mental health system into an outcomes-oriented culture of care. Based on his recent book, this chapter, like Thyer’s, provides the author’s views on the use of EBPs and how various instruments can support the measurement of service effectiveness, as well as different stakeholder views on the use of EBPs and outcomes measurement. He concludes with a "blueprint for change" that can help the United States transform to a national outcomes-based mental health system, including the caution that we need to correctly implement "outcome data for all parties" that help guarantee community-based recovery for persons with mental illness.
Integrated Measures for Mental and Substance Use Care (Chapter 14)
Siemianowski and Kirk provide an overview of identity and integration issues for the field of mental health and substance use generally, the implications of these issues for integrated outcomes measurement for the two fields, and the specific implications for persons with co-occurring disorders (persons with mental health and substance use conditions). They use an example of multilevel integration that has occurred in the state of Connecticut to illustrate how transformation in organizational culture, strong leadership, and working with a variety of stakeholders – especially consumers – have resulted in a system that is driven by recovery, outcomes, and performance measurement. They conclude with recommendations for outcomes that can reflect an integrated system of mental health and substance use care under health care reform, and that can support integration with primary care, so that those in need can receive the comprehensive services required for their recovery.
Emergence of Rapid Assessment Instruments (RAIs) (Chapter 15)
This chapter updates Corcoran’s 1997 chapter on RAIs with expanded commentary with his colleague Hozak. Just as Corcoran discussed the importance of RAIs as an important extension of outcomes measurement during managed care and in social work clinical practice, Corcoran and Hozak describe how RAIs can be useful in health care reform today. Although these tools have been in use for a long time, in routine practice they have not superseded reliance on wisdom, clinical judgments, and intuition in decision making. Corcoran and Hozack believe this will continue. However, they think that RAIs can provide evidence about medical necessity and be used as quick and valid measures of treatment outcomes, or as estimations of the clinical or real-life significance of usual treatment and/or interventions. They conclude with cautions about properly linking different populations to standardized and normed instruments and measures, as well as endorse appropriate uses of RAIs to meet various clinical and reform goals.
Consumer-operated Service Programs (COSPs) (Chapter 16)
Campbell describes the evolution of the consumer movement since the late 1990s, with specific emphasis on contributions of consumers, consumer researchers, and other stakeholders in the study and integration of COSPs within the continuum of behavioral health care in the United States. She describes the range of COSP service models and highlights salient research that has investigated their effectiveness, including results from the only national survey on COSP services, completed in 2002. Much of this chapter also focuses on rating the evidence of COSP studies – in the form of a mini-research synthesis – to question the appropriateness and types of evidence-based lenses through which to assess COSPs, and to set directions for future research and outcomes measurement in this area.
Research to Practice Example: Assertive Community Treatment Measures (Chapter 17)
Teague and Monroe-DeVita describe the evolution of measurement for one EBP – Assertive Community Treatment (ACT) – and provide an example of a live project that is testing refined aspects of ACT to improve service quality and outcomes for persons with severe mental illness. This example illustrates the relationship between fidelity, quality, outcomes, and effectiveness of ACT services, concluding with empowering comments – partially inspired by Kelly’s arguments about mental health system transformation – regarding the need to promote outcomes measurement, quality improvement, and outcomes-based culture.
Part IV examines the common and unique aspects of outcomes measurement in child and family services. Although the move toward outcomes measurement has been most evident in health and mental and behavioral health since 1997, the field of child and family services has been making great strides. Child and family services share many of the same contexts that have shaped outcomes measurement in other human service fields. The chapters in this section speak to these commonalities and, at the same time, highlight some of the unique measurement issues faced by child and family services.
Multilevel Measurement Approaches in Children’s Programs (Chapter 18)
Ethridge and colleagues describe how history, current policy changes, and systemwide reforms in the field of child and family services have shaped existing and emerging approaches to measuring outcomes and performance measures in this field of practice. They discuss outcomes and performance measurement domains (which are also discussed in chapter 20) and the challenges associated with using certain methodologies, measures, and research in daily management, practice, policy, and advocacy. They illustrate how these challenges are being addressed in an outcomes measurement process that was developed and implemented across child and family services in Kent County, Michigan, in the United States. The authors conclude with recommendations for improving the delivery and outcomes of child and family services, including the need for cross-system collaboration and the development of infrastructures and working practices that enhance the sharing of information across human service sectors.
Linking Costs and Outcomes for Children (Chapter 19)
Kilburn describes the challenges and solutions to measuring cost and outcomes of childhood services and programs and family well-being. She discusses why it is important to incorporate cost and outcomes analysis into human services research and decision making, describing the different methods by which this is possible: cost analysis, cost-effectiveness analysis, and cost-benefit analysis. Kilburn illustrates the use of these methods with examples from the child services literature, but she makes it clear in her discussion that these methods are usable in any human services domain. She concludes with implications for how cost and outcomes analysis can enhance policy, research, and practice in the child and family services field.
Research to Practice Example: Standardization of Process and Outcomes in Child Service and Welfare Evaluations (Chapter 20)
McGowan and Walsh discuss lessons learned from child services and welfare evaluations and the push for standardization of process and outcomes (for safety, permanency, and child and family well-being) at the state level. They describe how the new results-oriented approach in accountability first came into play in 2000, its subsequent statewide assessment review process, and how this movement is helping to address some of the "methodological shortcomings" (Wulcyzn, 1997) in outcomes measurement and promoting new approaches to child welfare practice.
Part V brings together four topical areas not typically represented in general discussions of outcomes measurement in the human services that can be helpful as we begin to think more holistically about outcomes measurement, performance measurement, and outcomes evaluation across the human services.
Transformation of the Veterans Health Administration (VHA) (Chapter 21)
Francis discusses lessons learned from the VHA during its transformation into a Cabinet-level government agency determined to incorporate innovative and evidence-based clinical, administrative, and management practices to provide better health care to U.S. veterans. He describes how the VHA evolved its outcomes and performance measurement system and its integrated electronic health record, as well as how instituting a systemwide Quality Enhancement Research Initiative has improved the implementation of EBPs and performance measurement in everyday practice. He concludes by noting the importance of accountable leadership in forging change in measurement and quality-improvement systems and why this process, which includes the determination of measures, needs to be more patient centered.
Outcomes in Criminal Justice (Chapter 22)
Turner provides an overview of outcomes measurement in the U.S. criminal justice system and the challenges associated with measuring outcomes at multiple levels: individual, agency, and state. She bases her discussion in two main outcomes measurement domains: crime rates and recidivism. She discusses how new measurement systems, such as CompStat, have helped to improve correctional institutions’ internal processes and outcomes, and in turn, community outcomes in both of these domains. Turner concludes with some lessons learned from the field of criminal justice that other human services may find useful in their quest to become more automated and collaborative with the various sectors involved in providing comprehensive care.
Outcomes in Faith-based Organizations (FBOs) (Chapter 23)
Flory and colleagues discuss the role that FBOs and services play in the United States and how more formalized programming and funding of such services have evolved over the past 15 years. They discuss the research upon which effectiveness evaluations have been based and directions for research that can help to better measure important components of such services, such as how the "faith factor" may influence clients and play a special role in measurement. They conclude by suggesting the development of a typology and language for FBOs and of forums that can help to better share lessons learned.
International View of Outcomes (Chapter 24)
Ganju provides an international perspective on health, mental health, and social outcomes. Like several other authors, he highlights the importance of linking several life domains and their associated outcomes together to better understand how we can improve the quality of life globally. In particular, he discusses a conundrum: Although a "natural imperative" exists that links mental health and substance use to broader societal and community goals (such as reduction in poverty, increased employment), we have yet to comfortably and effectively recognize, prioritize, and equalize mental health and substance use outcomes when evaluating global welfare. Ganju discusses the tensions underlying this and offers recommendations to help promote an international outcomes agenda for health, mental health, and substance use.
Concluding Comments
Since 1997, human services has grown in its "wealth" (Mullen & Magnabosco, 1997) of approaches, instruments, and types of outcomes and performance measures, as well as its capacity to integrate research and measurement processes into everyday clinical and administrative practice, policymaking, and advocacy. Regardless, much still needs to be done to critically assess the field of outcomes measurement and how effective its evidence bases are in meeting the myriad demands that clients, collaborators, communities, funders, and other stakeholders place on HSOs. This volume brings together an array of information that can be used as a starting point toward that end. We will return to this theme in our conclusion. We hope your reading of this volume will provide a contemporary understanding of some of the most salient crosscutting and field-specific, issues, methods, and tools in outcomes measurement in the human services today.
This volume is intended to be in sync with information presented in the first edition. Both volumes:
- Bring together prominent thinkers, researchers, practitioners, administrators, . . . policymakers [and consumers] to discuss issues relevant to the current impetus for accountability and measuring outcomes in health, mental and behavioral health, and child and family services
- Include chapters written by human service specialists with expertise in practice, policy and research, and advocacy
- Give students and professionals information to begin creating a comprehensive framework and to better understand the implications of outcomes measurement for the human services.
Many authors have contributed chapters to both editions, and in this edition, several authors again provide social work perspectives on outcomes measurement in the three main human service areas covered: health, mental/ behavioral health, and child and family services. This volume’s design and content are also grounded in a set of guiding questions similar to those of the first edition:
- Changed context and response since 1997: How have the contexts of outcomes measurement changed? How has outcomes measurement been usefully reconceptualized? What are the legislative and public policy contexts of outcomes measurement? Have they changed? If so, how? How have outcomes been addressed to meet legislative and public policy demands?
- New and established approaches to outcomes measurement since 1997: What approaches to outcomes measurement are being promulgated, and how have they changed since 1997? What can be said about the reliability, validity, and quality of existing approaches to outcomes measurement, as well as their relevance for social work and other human services?
- Future Implications: What are the implications of the increasing attention to outcomes measurement for social programs and for practitioners, policymakers, advocates, consumers, and researchers? What are the implications for future research, practice, policy, and advocacy?
Keeping these guiding questions in mind, authors in the second edition of Outcomes Measurement in the Human Services present an updated discussion of issues and methods in outcomes measurement (Mullen & Magnabosco, 1997). Where possible, and fitting, discussion regarding the effect that demands for accountability have had and will continue to have on social work practice is also included.
The Book’s Organization
This introduction sets the overall context for the book, summarizing each chapter and providing a descriptor of its content in italics. Parts I through IV of the book model the first edition’s overall structure. Part I addresses general overarching issues and cross-cutting themes of outcomes measurement, and parts II, III, and IV examine outcomes measurement in three select fields of human services: health, mental and behavioral health, and child and family services. Whereas one section in the first edition featured practice-research examples, in this edition we provide three such examples, one in each of the fields of practice, to illustrate the development and/or application of outcomes measurement in real-life human service scenarios. Part V is a new section that features particular areas in the human services that cut across the three main fields of practice featured and are timely and representative of important domains of interest in the 21st century. We close the book with concluding comments and recommendations for future action.
Part I: Overarching Issues and Methods
Part I "examines the overarching issues and methods that apply to all human service disciplines" (Mullen & Magnabosco, 1997, p. xxi). The themes found in these beginning chapters are echoed and particularly contextualized by chapters in parts II-V.
Orientation to Outcomes and Performance Measurement (Chapter 2)
Hatry updates the current status of outcomes and performance measurement in the public and private sectors, discussing major recent activities that have shaped the direction of such measurement in the United States and internationally. In this chapter (as in the first edition), he distinguishes among types of outcomes measurement and approaches and discusses the need for human service organizations (HSOs) to incorporate outcomes management in everyday organizational activities. He questions whether the typically collected outcomes and performance data are valid and comprehensive and raises concerns about how such data are being used. He advocates for the need to balance and integrate regular outcomes measurement and program evaluation, especially in light of the need to address disparities (inequities), and discusses various strategies being used to effectively meet accountability demands from funders, government, specific communities, and the public at large.
Qualitative and Quantitative Approaches and Outcomes Evaluation (Chapters 3 and 4)
The next two chapters provide perspectives on qualitative and quantitative approaches to outcomes measurement and outcomes evaluation. This dual perspective echoes Hatry, highlighting the necessity of both approaches and their unique contributions to the development, implementation, monitoring, and provision of outcomes and performance measurement, as well as to accountability, practice, and research purposes. In chapter 3, Patton and Gornick discuss the development, trends, and impact that qualitative data – specifically, qualitative outcomes evaluations – have had in the outcomes measurement movement and in meeting demands for accountability in the human services. They illustrate the value that qualitative methods offer in a case example evaluation of a residence for persons with long-term alcohol and drug use conditions. Patton and Gornick conclude with comments on some of the major issues in the future of outcomes evaluation, including the need to develop a common language for types of evaluation and outcome measures, as well as how increasingly important concepts, such as strengths-based care and prevention, require understanding that there may be "different meanings in different contexts."
In chapter 4, Yates provides a commentary on quantitative approaches to outcomes measurement and evaluation. On the basis of his 35 years researching and evaluating HSO programs, he reflects on issues that are common to measuring outcomes quantitatively across HSOs, such as the idea that "reliability and validity of quantitative outcomes are not just based on science anymore," and the fact that the process of selecting and quantifying outcomes for, and by, funders is not standardized. He then discusses implications for outcomes measurement in this era of change, and offers the RAPOA (Resource-Activity-Process-Outcome Analysis) model as a tool that can help HSOs measure the range of outcomes related to improving services and systems.
Controversies in Evidence-based Practice and Outcomes (Chapter 5)
Thyer discusses controversies in the measurement of outcomes and evidence-based practice (EBP). He shares views that are critically needed during this time of turbulent change in the human services: He explains why it is important to pause and reconsider the fact that EBP is really a five-step process, and that EBP "does not tell clinicians and others involved with an EBP process what to do." He thoroughly discusses the relationship between the EBP process and outcomes measurement, how to locate and assess credible evidence, and various stakeholder approaches to outcomes measurement, as well as the implications different perspectives have for human services and EBP. He concludes with thoughts on how to balance approaches to outcomes measurement and EBPs and ends on an empowering note to consumer and advocacy groups, suggesting that they become more involved with promoting EBPs in service and government organizations.
A Process for "Getting to Outcomes" (Chapter 6)
Acosta and Chinman describe a method for HSOs to build capacity for outcomes called "Getting to Outcomes" (GTO). The authors discuss how the 10-step GTO model is "more than a process – [it] is an intervention that builds the capacity of users to use outcome data for continuous quality improvement, not just performance measurement." They illustrate the use of GTO in a current project designed to implement positive youth development programs by an adolescent coalition in Maine. They conclude with a discussion about barriers associated with measuring outcomes in HSOs and how GTO can effectively help navigate the development of capacity and outcomes.
Leadership and Outcomes Assessment (Chapter 7)
Packard and Beinecke discuss leadership development in the human services and the implications leadership can have for outcomes at multiple levels. They make it clear that human services leadership is facing a serious challenge over the next decade as many experienced human service professionals retire. They also review the main models of leadership, tenets of leadership versus management, organizational change, and pertinent studies that have shown links between leadership and outcomes in the human services. Packard and Beinecke offer a conceptual model for organizational change that can be used as a framework to develop and measure progress toward outcomes-based cultures in HSOs. They conclude with suggestions for promoting "change leaders" and outcomes measurement systems and cultures in HSOs.
Part II: Outcomes Measurement in Health
In 1997, Mullen and Magnabosco wrote, "perhaps no field of practice of human service practice has been more affected by outcomes measurement than health." (p. xxviii) This remains true today.
Integrated Service Delivery (Chapter 8)
Bartlett describes a current solution to integrated healthcare – the patient-centered medical home – and the implications it has had during health care reform and for the United States in general. Chronicling the history of primary care delivery in this country and its relation to current health care reform tenets, Bartlett describes the challenges to providing cost-effective and needed care, and to measuring integrated health care outcomes between primary care and other sectors, especially mental health and substance use. He stresses the unmistakable need to integrate health and mental health and describes several strategies that may help to bring us closer to reasonable outcomes, such as implementing the "use of brief, valid, and reliable tools" (like Zabora in chapter 12 and Corcoran and Hozak in chapter 15) and "thinking globally but acting locally" (Geddes, 1915) – that is, measuring health outcomes at all levels, with an eye toward improved population health.
Patient-reported Measures (Chapter 9)
Cherepanov and Hays provide a comprehensive discussion of measures that are critical to monitoring health and care: patient-reported outcome measures, health-related quality-of-life measures, and patient evaluations of care. The authors discuss the various types of measures in detail and their applications to practice; in particular, they discuss the Consumer Assessment of Healthcare Providers and Systems (CAHPS) Program to illustrate patient evaluation of care measures. Like other authors in the book, Cherepanov and Hays chronicle the changes made in measurement and note the available resources. In their conclusion, they predict that patient-reported outcome measures may gain greater significance during health care reform.
Assessing Services for an Aging Population (Chapter 10)
Berkman and Kaplan discuss one of the most pressing health concerns the United States will face in the next decade: our aging population. They describe the characteristics of this population and how they will affect the context of health care delivery, regardless of health care reform. Berkman and Kaplan review the state of outcomes measurement in aging and provide a range of instruments and measures that are especially relevant and useful. Like other authors in the book, they highlight the role that social workers will have as the face of health care delivery is changed by this population’s needs and by health care reform.
Complementary and Alternative Medicine (CAM) (Chapter 11)
Coulter and Horsan discuss an aspect of health care that is widespread but not routinely included in mainstream discussions: the use of CAM and integrative medicine (IM). The authors explain the nature and modalities (e.g., acupuncture, chiropractic medicine) of CAM and IM, their paradigms, the controversies and challenges associated with integrating CAM and IM, and the challenges associated with conducting research and outcomes measurement. They conclude with recommendations and comments about useful approaches (e.g., observational studies) and theoretical models (e.g., systems theory), the principles upon which CAM outcomes measurement should be based (e.g., "elevate the notion of person-centered care and patient-centered care"), and an agenda for outcomes measurement in CAM that can be used for comparative effectiveness research and other health care reform efforts.
Research to Practice Example: Chronic Care and Psychological Distress (Chapter 12)
Zabora illustrates the importance of using appropriate and comprehensive screening and outcomes assessment tools to determine the range of needs chronic care patients, especially oncology patients, have during the course of care and over the longer term. Zabora focuses on screening and assessments for psychological distress – one of the most serious effects of chronic illness – in the context of hospital settings. This focus is of particular note because screening for such distress often goes unperformed, largely because of a lack of comprehensive tools and measures. He concludes with a discussion of barriers to such assessments, how particular members of the health care team – such as social workers – are better poised to take on this responsibility, and how such instruments and measures can be especially useful in health care reform efforts that seek quick and valid approaches.
Part III: Outcomes Measurement in Mental and Behavioral Health
The contributors to part III examine mental and behavioral health services outcomes measurement. As in the first edition, the terms mental health and behavioral health are sometimes used interchangeably when discussing outcomes measurement. In 1997, the term behavioral health typically was used to designate services that applied to people with substance abuse and mental health problems in a managed care context (Freeman & Trabin, 1994; Mullen & Magnabosco, 1997). Today, behavioral health carries a similar meaning, with two differences: First, substance use is used in place of substance abuse. Second, the overall meaning of substance use is not determined as much by the managed care context. Rather, it is informed by the fact that consumers, and the wide range of stakeholders in the fields of mental health and substance use, have come to accept a broader and more integrated approach to the spectrum of mental health and substance use care. Although the use of the terms "substance use" and "substance abuse" largely depend on ideology, practice, and societal context, the former represents recognition that persons with this disorder have a real illness and that the issues associated with this illness are multidimensional, taking into account the holistic individual, not just his or her drug using or drug "abusing." Hence, in the United States, we have more readily come to use the term substance use to "connote measurability or an emphasis on behavioral functioning" (Mullen & Magnabosco, 1997).
Transformation and Outcomes (Chapter 13)
Kelly examines mental health policy in the United States, factors leading to the recent emphasis on mental health transformation or reform, and recommendations for what is needed to transform the country’s mental health system into an outcomes-oriented culture of care. Based on his recent book, this chapter, like Thyer’s, provides the author’s views on the use of EBPs and how various instruments can support the measurement of service effectiveness, as well as different stakeholder views on the use of EBPs and outcomes measurement. He concludes with a "blueprint for change" that can help the United States transform to a national outcomes-based mental health system, including the caution that we need to correctly implement "outcome data for all parties" that help guarantee community-based recovery for persons with mental illness.
Integrated Measures for Mental and Substance Use Care (Chapter 14)
Siemianowski and Kirk provide an overview of identity and integration issues for the field of mental health and substance use generally, the implications of these issues for integrated outcomes measurement for the two fields, and the specific implications for persons with co-occurring disorders (persons with mental health and substance use conditions). They use an example of multilevel integration that has occurred in the state of Connecticut to illustrate how transformation in organizational culture, strong leadership, and working with a variety of stakeholders – especially consumers – have resulted in a system that is driven by recovery, outcomes, and performance measurement. They conclude with recommendations for outcomes that can reflect an integrated system of mental health and substance use care under health care reform, and that can support integration with primary care, so that those in need can receive the comprehensive services required for their recovery.
Emergence of Rapid Assessment Instruments (RAIs) (Chapter 15)
This chapter updates Corcoran’s 1997 chapter on RAIs with expanded commentary with his colleague Hozak. Just as Corcoran discussed the importance of RAIs as an important extension of outcomes measurement during managed care and in social work clinical practice, Corcoran and Hozak describe how RAIs can be useful in health care reform today. Although these tools have been in use for a long time, in routine practice they have not superseded reliance on wisdom, clinical judgments, and intuition in decision making. Corcoran and Hozack believe this will continue. However, they think that RAIs can provide evidence about medical necessity and be used as quick and valid measures of treatment outcomes, or as estimations of the clinical or real-life significance of usual treatment and/or interventions. They conclude with cautions about properly linking different populations to standardized and normed instruments and measures, as well as endorse appropriate uses of RAIs to meet various clinical and reform goals.
Consumer-operated Service Programs (COSPs) (Chapter 16)
Campbell describes the evolution of the consumer movement since the late 1990s, with specific emphasis on contributions of consumers, consumer researchers, and other stakeholders in the study and integration of COSPs within the continuum of behavioral health care in the United States. She describes the range of COSP service models and highlights salient research that has investigated their effectiveness, including results from the only national survey on COSP services, completed in 2002. Much of this chapter also focuses on rating the evidence of COSP studies – in the form of a mini-research synthesis – to question the appropriateness and types of evidence-based lenses through which to assess COSPs, and to set directions for future research and outcomes measurement in this area.
Research to Practice Example: Assertive Community Treatment Measures (Chapter 17)
Teague and Monroe-DeVita describe the evolution of measurement for one EBP – Assertive Community Treatment (ACT) – and provide an example of a live project that is testing refined aspects of ACT to improve service quality and outcomes for persons with severe mental illness. This example illustrates the relationship between fidelity, quality, outcomes, and effectiveness of ACT services, concluding with empowering comments – partially inspired by Kelly’s arguments about mental health system transformation – regarding the need to promote outcomes measurement, quality improvement, and outcomes-based culture.
Part IV: Outcomes Measurement in Child and Family Services
Part IV examines the common and unique aspects of outcomes measurement in child and family services. Although the move toward outcomes measurement has been most evident in health and mental and behavioral health since 1997, the field of child and family services has been making great strides. Child and family services share many of the same contexts that have shaped outcomes measurement in other human service fields. The chapters in this section speak to these commonalities and, at the same time, highlight some of the unique measurement issues faced by child and family services.
Multilevel Measurement Approaches in Children’s Programs (Chapter 18)
Ethridge and colleagues describe how history, current policy changes, and systemwide reforms in the field of child and family services have shaped existing and emerging approaches to measuring outcomes and performance measures in this field of practice. They discuss outcomes and performance measurement domains (which are also discussed in chapter 20) and the challenges associated with using certain methodologies, measures, and research in daily management, practice, policy, and advocacy. They illustrate how these challenges are being addressed in an outcomes measurement process that was developed and implemented across child and family services in Kent County, Michigan, in the United States. The authors conclude with recommendations for improving the delivery and outcomes of child and family services, including the need for cross-system collaboration and the development of infrastructures and working practices that enhance the sharing of information across human service sectors.
Linking Costs and Outcomes for Children (Chapter 19)
Kilburn describes the challenges and solutions to measuring cost and outcomes of childhood services and programs and family well-being. She discusses why it is important to incorporate cost and outcomes analysis into human services research and decision making, describing the different methods by which this is possible: cost analysis, cost-effectiveness analysis, and cost-benefit analysis. Kilburn illustrates the use of these methods with examples from the child services literature, but she makes it clear in her discussion that these methods are usable in any human services domain. She concludes with implications for how cost and outcomes analysis can enhance policy, research, and practice in the child and family services field.
Research to Practice Example: Standardization of Process and Outcomes in Child Service and Welfare Evaluations (Chapter 20)
McGowan and Walsh discuss lessons learned from child services and welfare evaluations and the push for standardization of process and outcomes (for safety, permanency, and child and family well-being) at the state level. They describe how the new results-oriented approach in accountability first came into play in 2000, its subsequent statewide assessment review process, and how this movement is helping to address some of the "methodological shortcomings" (Wulcyzn, 1997) in outcomes measurement and promoting new approaches to child welfare practice.
Part V: Special Topics in the Human Services
Part V brings together four topical areas not typically represented in general discussions of outcomes measurement in the human services that can be helpful as we begin to think more holistically about outcomes measurement, performance measurement, and outcomes evaluation across the human services.
Transformation of the Veterans Health Administration (VHA) (Chapter 21)
Francis discusses lessons learned from the VHA during its transformation into a Cabinet-level government agency determined to incorporate innovative and evidence-based clinical, administrative, and management practices to provide better health care to U.S. veterans. He describes how the VHA evolved its outcomes and performance measurement system and its integrated electronic health record, as well as how instituting a systemwide Quality Enhancement Research Initiative has improved the implementation of EBPs and performance measurement in everyday practice. He concludes by noting the importance of accountable leadership in forging change in measurement and quality-improvement systems and why this process, which includes the determination of measures, needs to be more patient centered.
Outcomes in Criminal Justice (Chapter 22)
Turner provides an overview of outcomes measurement in the U.S. criminal justice system and the challenges associated with measuring outcomes at multiple levels: individual, agency, and state. She bases her discussion in two main outcomes measurement domains: crime rates and recidivism. She discusses how new measurement systems, such as CompStat, have helped to improve correctional institutions’ internal processes and outcomes, and in turn, community outcomes in both of these domains. Turner concludes with some lessons learned from the field of criminal justice that other human services may find useful in their quest to become more automated and collaborative with the various sectors involved in providing comprehensive care.
Outcomes in Faith-based Organizations (FBOs) (Chapter 23)
Flory and colleagues discuss the role that FBOs and services play in the United States and how more formalized programming and funding of such services have evolved over the past 15 years. They discuss the research upon which effectiveness evaluations have been based and directions for research that can help to better measure important components of such services, such as how the "faith factor" may influence clients and play a special role in measurement. They conclude by suggesting the development of a typology and language for FBOs and of forums that can help to better share lessons learned.
International View of Outcomes (Chapter 24)
Ganju provides an international perspective on health, mental health, and social outcomes. Like several other authors, he highlights the importance of linking several life domains and their associated outcomes together to better understand how we can improve the quality of life globally. In particular, he discusses a conundrum: Although a "natural imperative" exists that links mental health and substance use to broader societal and community goals (such as reduction in poverty, increased employment), we have yet to comfortably and effectively recognize, prioritize, and equalize mental health and substance use outcomes when evaluating global welfare. Ganju discusses the tensions underlying this and offers recommendations to help promote an international outcomes agenda for health, mental health, and substance use.
Concluding Comments
Since 1997, human services has grown in its "wealth" (Mullen & Magnabosco, 1997) of approaches, instruments, and types of outcomes and performance measures, as well as its capacity to integrate research and measurement processes into everyday clinical and administrative practice, policymaking, and advocacy. Regardless, much still needs to be done to critically assess the field of outcomes measurement and how effective its evidence bases are in meeting the myriad demands that clients, collaborators, communities, funders, and other stakeholders place on HSOs. This volume brings together an array of information that can be used as a starting point toward that end. We will return to this theme in our conclusion. We hope your reading of this volume will provide a contemporary understanding of some of the most salient crosscutting and field-specific, issues, methods, and tools in outcomes measurement in the human services today.
Jennifer L. Magnabosco, PhD, currently is co-principal investigator and projects director at the Center for Implementation Research and Practice Support and health science research specialist at the Center of Excellence, Center for the Study of Health Care Provider Behavior, Veterans Health Administration, Department of Veterans Affairs, North Hills, CA. Concurrently she is management consultant and principal investigator for the Los Angeles County Department of Mental Health; adjunct professor in the Department of Social Work, California State University, Los Angeles; and part-time research faculty member at Yo San University of Traditional Chinese Medicine, Los Angeles. Previously she was associate director and senior research associate at the Thomas and Dorothy Leavey Center for the Study of Los Angeles at Loyola Marymount University, Los Angeles; associate policy researcher at the RAND Corporation, Santa Monica, CA; director of administration and operations at the Center for the Study of Social Work Practice at Columbia University, New York; and staff psychotherapist at the Postgraduate Center for Mental Health, New York. Dr. Magnabosco is co-chair of the Research Interest and Emerging Leader groups of ACMHA – The College for Behavioral Health Leadership – and a governing council member and member of the action board for the Mental Health Section of the American Public Health Association. She has also held several alumni association volunteer positions, including vice president of the University of Chicago National Alumni Board of Governors and board member of the Columbia University School of Social Work National Alumni Board of Directors.
Ron Manderscheid, PhD, currently serves as executive director of the National Association of County Behavioral Health and Developmental Disability Directors, Washington, DC. The Association represents county and local authorities and provides a national program of technical assistance and support. Concurrently, he is adjunct professor in the Department of Mental Health, Bloomberg School of Public Health, Johns Hopkins University, Baltimore; a member of the Secretary of Health and Human Services’ Advisory Committee on Healthy People 2020; and president of ACMHA - The College for Behavioral Health Leadership. Dr. Manderscheid serves on the boards of the Employee Assistance Research Foundation, the Danya Institute, the FrameWorks Institute, and the Council on Quality and Leadership. Previously, he served as the director of mental health and substance use programs at the Global Health Sector of SRA International, Rockville, MD. Throughout his career, he has emphasized and promoted consumer and family concerns.
Ron Manderscheid, PhD, currently serves as executive director of the National Association of County Behavioral Health and Developmental Disability Directors, Washington, DC. The Association represents county and local authorities and provides a national program of technical assistance and support. Concurrently, he is adjunct professor in the Department of Mental Health, Bloomberg School of Public Health, Johns Hopkins University, Baltimore; a member of the Secretary of Health and Human Services’ Advisory Committee on Healthy People 2020; and president of ACMHA - The College for Behavioral Health Leadership. Dr. Manderscheid serves on the boards of the Employee Assistance Research Foundation, the Danya Institute, the FrameWorks Institute, and the Council on Quality and Leadership. Previously, he served as the director of mental health and substance use programs at the Global Health Sector of SRA International, Rockville, MD. Throughout his career, he has emphasized and promoted consumer and family concerns.
