Disability employment service systems in Australia have no appreciable way of measuring the impact of employment interventions on individuals who access these services. The Australian Federal Government’s flagship disability employment system: the Disability Employment Services (DES), focuses on process and provider success, with little understanding of the impact of employment practice on the individual.

The Disability Employment Services (DES) exists to support people with a disability to gain meaningful employment on par with other members of society, often seen through the lens of economic citizenship. For a person with a disability, this is often wrapped up in the issue of rights in the context of meeting the United Nations Convention on the Rights of Persons with Disabilities 2006 (UNCRPD) obligations and often, Human Rights and Discrimination legislation. This proposition is not new, having been espoused in the UN Declaration of Human Rights in 1948 that supported the ideal of the right to choose work with just and fair conditions (Parmenter, 1986). The Disability Employment Reference Group established in 2021 as part of the Disability Employment Service (DES) review, is led by the Australian Disability Discrimination Commissioner, from the Australian Human Rights Commission.

The success of the current DES system focuses on the capacity of the disability employment service provider to find and sustain the client in employment settings measured at 13, 26 and 52 weeks; weighted towards outcomes at 26 weeks to receive outcome payments and are known to measure provider compliance against the system (DES, 2021; Smith, 2018).

The current system of key performance indicators (KPIs), with their focus on efficiency, effectiveness and service quality takes no account of the impact of employment or the quality of the placement on the individual, nor account for practice methods. Bellamy, Rhodes, and Albin (1986) highlighted the need to measure employment outcomes across the broader impact on the individual not simply hours and wages earned. We still have not progressed in our construct of measuring the success of employment support programs beyond simple placement hours and dollars earned. The current disability employment review in Australia proposes improvements towards the quality KPI, however there is no suggestion of measuring the impact of employment on the individual or the mandating of evidence-based practices.

Current measures focus primarily on the institutional level, i.e., the efficiency of service provision in securing employment by the provider system in response to policy settings. Employment for people with a disability has historically been in low skills and low wages jobs, which does not sit comfortably with the intent of the landmark Disability Services Act 1986 and the National Disability Insurance Scheme Act 2013 (NDIS), both of which support the goal of a healthy life in the community. The legislation is consistent with a whole of life capacity building approach that includes access and support to pursue quality employment, complete with wage justice. Mallett, Brown & Finnis (2021) highlighted the need to take a life course view of building the capacity of a person with a disability by intervening early to ensure that education and transition to employment are aligned to a real job in the community. However, the fragmented nature of transition and employment services continues to hamper successful transition for young people (Bond & Keys, 2020). Despite policy initiatives in Australia and legislation that supports employment opportunities for people with a disability, research has shown only a limited number (n=17) of projects being undertaken that support employment interventions for people with a disability using evidence-based practices (Brown & Mallett, 2021). School to post-secondary education and employment is hampered by an absence of policy and practice alignment at this vital life stage by competing policy objectives that challenges the traditional boundaries of state and federal level governments (Shelvin et al., 2020).

Consistent with experience in Australia, researchers overseas have highlighted the disconnect between various agencies such as education, vocational rehabilitation, and lifestyle support (Luecking & Certo, 2002). The outcome of this disconnect is low paid employment and poverty, with employment often in centre-based services such as sheltered workshops. Shultz & Carter (2022) reported that transition education should have employment as a central goal along with connecting youth with disabilities to jobs that meet their needs and offer career pathways. An examination of the provision of disability employment services worldwide shows various approaches, but increasingly reflecting the prevailing free marketideol-

ogy and culture of the respective states which has seen a separation of employment provision from other support services. The past decade has seen an acceleration of governments to tender out services they have traditionally provided, loosely based on adherence to Friedman’s ideology that market-based solutions offer better service provision than government organizations (Harris, 2001).

Employment services are one area that has seen governments abandon service provision in favor of utilizing external parties to deliver services. According to Struyven (2004) countries which decide to involve the private sector in the re-integration of long-term unemployed people face a dual challenge of not only tackling unemployment but also implementing market competition.

Struyven (2004) identified two critical issues for governments choosing this pathway: creating space for market competition and the need for positive creaming, which encourages service providers to focus on the most profoundly disadvantaged job seekers. Conversely in the Australian context, there is evidence of "creaming and parking" of clients, where providers focus on job-ready clients (the most able of the disabled) to exclude the more disadvantaged (Dept of Social Services, 2014). This risk is more closely associated with programs that focus on rewarding providers for job outcomes.

The development of Employment First policy initiatives in the USA, while highlighting the capacity of people with a disability to work in integrated settings has shown that policy settings need further development to improve open employment outcomes (Domin & Butterworth, 2013). This suggests that policy and legislation alone will not be sufficient to change open employment opportunities (Martinez, 2013), with a recent report by Grossi and Andresen (2022) into disability employment outcomes in the state of Indiana highlighting largely static employment outcomes, despite policy initiatives.

Francis, Gross, Turnbull, and Turnbull (2014) examined barriers to employment for people with intellectual disabilities and individualized support needs in the USA, who participated in the Family Employment Awareness Training (FEAT). Participants reported that agencies and programs did not provide appropriate or effective support and in general used a generic view of disability in service provision. A generic approach to service delivery is often a response to policy settings and may be a contributing factor to outcomes reported in the USA (Grossi & Andresen, 2022). An example of a generic simple approach to the demand side of employment is the use of quotas and fines in China to drive business to meet China’s UNCRPD obligations. A report jointly produced by The Conference Board, China and the Yang-Tan Institute on Employment and Disability, ILR School, Cornell University (2018) highlighted how this type of simplistic approach to achieving policy outcomes does not address the issues employers face in employing people with a disability.

Research has illustrated a number of factors that contribute to successful school to work transition such as self-determination, a factor crucial to gaining employment (Sigstad & Garrels, 2022). At the same time, the absence of planning for post school inclusive higher education and employment increases the likelihood of being excluded from the normative pathways available to young people without a disability (Uditsky & Hughson, 2012). The outcome of this is higher levels of exclusion and the inability to develop career pathways and identities. Perri et al., (2021) noted that legislation in Canada to support school to work transition is needed so that policies are developed to ensure that young people with a disability can access career and employment planning that support long-term employment.

Beyer et al (2010a) examined supported employment in Europe, and while not covering employment support across all European countries did conclude that supported or open employment is a foundation tool for economic inclusion for people with intellectual disabilities. The study focused on funding and the use of supported employment services. Gustafsson, Peralta and Danermark (2018) reviewed supported or open employment in Sweden. The study highlighted the role of individual factors in successful employment outcomes, providing support for an individualized approach that values the individual and the impact of employment on their life beyond simple employment and income measures.

Stainton et al (2011) examined the literature surrounding social and economic outcomes for supported employment from a cost benefit point of view concluding that supported or open employment has a higher cost benefit than sheltered employment of day programs. The literature search examined data from Canada, Australia, the United States and United Kingdom. Research tends to focus on systematic issues, policy and what service providers and staff do with little understanding of the personal outcomes of employment on the individual. An examination of and the development of an understanding of the impact of employment on the individual may go some way to understanding what processes, policies and practices are necessary to achieve real growth in disability employment and full economic citizenship that meets our obligations under the UNCRPD conventions. A comprehensive literature review that examines the benefits of employment from the perspective of its impact on the individual is needed.

A search utilizing EBSCO host databases was conducted to find evidence of alternative approaches to measuring disability employment outcomes at an individual impact level.

The initial search revealed over 500,000 peer-reviewed papers using the inclusion/exclusion criteria noted in Fig 1.It was based on using keywords and snowballing more keywords into the search to refine the possible literature that may fall into the search category. This approach enabled the reduction of available peer-reviewed papers to a manageable level, with the final keywords considered appropriate for the nature of the research. This produced four hundred and fifty-four papers that met the keyword criteria.

A review of all 454 papers highlighted a paucity of research into outcome measures; indeed, the papers reviewed overwhelming focused on what services did and could do better, along with the quality-of-life outcomes for several disability types. A significant number focused on analyzing longitudinal data to highlight employment outcomes post-school and interventions such as Vocational Rehabilitation (VR) and Supported Employment (SE). Following this process, subsequent rounds of screening were undertaken to reduce the selected papers to a final group that focused on outcomes measures at a personal level.

Fig 1. Flow chart illustrating selection process

Only eleven papers met the criteria (Table 1). The search highlighted two recent papers that looked at outcome measures for specific populations; Taylor and Seltzer’s (2012) Vocational Index for Adults with ASD, and Di Rezze et al. (2018), which reviewed outcome measures for adults with neurodevelopmental disabilities (NDD). Taylor and Seltzer produced an eleven-category scale based on data collected over twelve years from approximately 350 participants that measures change over time. They believed that it would be a suitable adjunct to other measures rather than a stand-alone measure. In that sense, while a helpful tool, it would be limited in its application.

Di Rezze et al. (2018) restricted their study to neurodevelopmental disabilities (NDD), with the explicit aim to identify standardised instruments and procedures used in employment used over the past forty-five years. They noted that a list of outcome measures might better inform VR professionals. Their inclusion criteria resulted in 45 peer-reviewed articles identifying 64 measures for review. An examination of the articles and instruments identified found that overwhelmingly, they focused on job-site activities, workplace integration, behaviour support, job service support, personality traits, work preferences and wages.

Table 1. Result summary table

Table 1. (continued)

Table 1. (continued)

Beyer et al. (2010b) examined quality of life (QOL) outcomes across different employment services using existing QOL scales and the Work Environment Scale. That research focused on the quality of outcomes from the different service types rather than employment impact, highlighting the need for more research with better control of critical variables such as gender and adaptive behavior. Research by Dutta et al. (2008) could be classified as similar in its intent. It focused on longitudinal data from participants in VR services with various disabilities to determine which disability type achieved the more significant employment outcomes from this service type. The primary outcome of this study was that state VR services are associated with employment outcomes when used aggressively with medical interventions. In some ways this is similar to the Wehman et al (2014) study that looked at program outcomes, in this case highlighting the effectiveness of internships as a transition to work support.

Fleming et al. (2013) conducted a similar study into the effectiveness of VR services when using the World Health Organization, (2001) International Classification and Functioning and Health (ICF), concluding that the ICF improves predictors of QOL. Earlier research by Gardner contributed to the development of the Council on Quality and Leadership’s Outcome Based Performance Measures published in 1993, now known as Personal Outcomes Measures (POMs) which is a person-centered outcome measure that is widely used for organizational certification in person-centered service delivery (Gardner & Carran 2005). It does not single out employment or measure the impact of employment on the individual. It shares characteristics with the Quality of Supported Employment Implementation Scale (QSEIS) developed by the Individual Place and Support (IPS) community that focuses on measuring the implementation of supported employment programs for people with severe mental illness (Bond, Picone, Mauer, Fishbein & Stout, 2000), a measure of service delivery rather than the impact of employment. Later research by Friedman (2018) focused on the validity of the POMs measure, which while agreeing with its validity, did not extend it to encompass the impact of employment beyond simple quality of life measures. Schalock (2020) proposed a Quality of Life Supports Model that combined the quality-of-life concept with the supports model as a framework for policy development, however it was proposed as a model that supported service provision, organization and systems change. It made no significant mention of employment, despite being rooted in the personal well-being framework of quality of life.

Overall, the search failed to highlight any measure that focused on the impact of employment on the individual beyond quality-of-life measures. Not discounting the importance of quality-of-life measures, it is our view that a measure that looks at the impact of employment would more closely align with the intent of the government policy. That is, it supports employment as a fundamental human right and individualized funding that supports choice and control. This type of measure would have significant potential to improve the quality of employment outcomes for people with a disability by shifting the focus from service provision to the quality of the outcome and its benefit to the individual. Successful service provision is often seen as a proxy for quality service by governments; however, at a time of increasing focus on individualization in both funding and practice, measures that exclude the quality of the outcome are seen as out of step with society and personal views. The current Australian Star Rating measurement system is a measure of inputs and reflects system compliance and processes rather than changes in the clients’ domains commonly associated with the benefits of employment: social inclusion, social cohesion, socio-economic security, and self-empowerment. These domains align with the definition of social quality as being “the extent to which people are able to participate in social relationships under conditions which enhance their well-being, capacity and individual potential” (Lin & Herrmann, 2015).

The growing movement that links social quality to citizenship offers a new framework on which to base the outcome measure with its focus on the impact of policy on the individual. van der Maesen and Walker (2005) defined social quality as "the extent to which people can participate in social, economic and cultural life and the development of their communities under conditions that enhance their well-being and individual potential, which enables them, in turn, to influence the conditions of their existence" (p.11). This definition illustrates a strengths-based approach to the empowerment of the individual and sits comfortably with the personal values of self-control, choice and personal advocacy, values seen as inherent to the success of any program that supports individualisation. Social Quality Theory (SQT) originated in Europe within the social policy idiom, commonly viewed as an attempt to push back against the prevailing neoliberal policy construct that heavily influenced Western policy development (Beck et al., 1998).

In defining the proposed domains, it is crucial to recognize that the outcome measure measures the impact of employment on the individual or self (figure 2). Disability employment systems through their policy settings, have primarily focused on measuring the system success through provider placement rates and employment retention up to and including fifty-two weeks of employment in the context of the Australian disability employment system (DES, 2021).

Figure 2. Outcome Dimensions of the Self

With the development of the NDIS in Australia, the focus has progressively moved towards the individual to purchase disability supports from the open market. This shift presents an opportunity to focus on the client and the impact of various supports on them. In that sense, the individual is now the center and focus of government systems, reducing the provider to that of supplier of services at the beck and call of the client – actual person-centered choice and control. It also highlights the focus of NDIS supports on building the clients capacity to lead an independent life with employment a major aspect.

This shift provides the impetus for the development of an outcome measure that measures the impact of employment on the self. A necessary part of this is the defining of the domains as they relate to the intent of the measure to bring clarity and purpose to the measure.

In examining the literature on social cohesion, it is apparent that it is still the subject of robust debate between academia and government on policy objectives.

A literature search revealed an extensive collection of peer-reviewed papers that addressed social cohesion from a wide variety of subjects ranging from communities, ethnic diversity, social capital, streetscape design, town planning, civil war, sexual behavior and even cohesion between female marmots. Researchers argue that it is a characteristic of society and not an individual trait (Schiefer & van der Noll, 2017). Schiefer and van der Noll (2017) go so far as to quote Durkheim's (1893) view that "Social cohesion is not a by-product of individual behavior but rather based on solidarity, shared loyalties, cooperation and mutual action"(pp. 584). However, it is our view that it is constituted on individual attitudes, communities constructed of individuals and, at the highest level, organizations of individuals.

This leads to the necessary conclusion that higher inclusion levels are a requirement of increasing levels of cohesion within society (Oxoby, 2009). We argue that Durkheim's view of excluding individual agency precludes the development of cohesion as an outcome of self-actualization or realization, and the capacity of the self-actualized to create higher levels of social capital as a condition of cohesion within society. Much of the earlier research into social cohesion focused on social order or the integration of individuals and groups from different cultures living in an integrated society. In recent years, the discussion regarding social cohesion has shifted towards government policy outcomes as countries deal with mass immigration, which reinforces the absence of a formal definition (Oxoby, 2009).

Various researchers have proposed differing numbers of constituents that make up social cohesion, with Bottoni (2017) proposing a seven-constituent model comprising, interpersonal trust, social support, the density of social relationships, openness, participation, institutional trust, and legitimacy of institutions. Schiefer & van der Noll (2017) proposed six dimensions: Social relations, identification, orientation towards the common good, shared values, quality of life, and (in) equality. While they share some commonalities, they are different and again highlight the absence of consensus on cohesion.

Within the social quality literature, social cohesion is expressed as; "the extent to which social relations, based on identities, values and norms, are shared" (van der Maesen & Walker, 2005, p12). It focuses on forming a collective identity created by relationships in daily life. In a sense, it is the creation of social capital (norms) that binds communities of common interest together. This was highlighted in Bourdieu's (1986) definition of social capital as; An aggregate of the actual or potential resources which are linked to possession of a durable network of more or less institutionalized relationships of mutual acquaintance and recognition which provides each of its members with the backing of the collectively owned capital. (pp. 248) How then do we define cohesion? The van der Maesen and Beck (2005) definition; "the extent to which social relations, based on identities, values and norms, are shared" articulates it well; however, given our focus on the impact of employment, we propose that social cohesion is: The capacity of the self-actualized to develop relationships of mutuality and reciprocity that support access to community institutions that alleviate economic exclusion and create conditions that support the individual’s economic participation in society. In this definition, we highlight the link between inclusion and cohesion and how social capital is a vital ingredient in healthy societies (Oxoby, 2009). From our perspective, social capital is an outcome of mutuality and reciprocity, without which social capital has no value.

Developing an understanding of social inclusion through the literature provides an exciting glimpse of its nature. The literature displays a number of research papers that discuss social inclusion through the lens of sport, transport, pets, and immigration. However, there is a paucity of literature focusing purely on defining social inclusion, something that is apparent across other areas of research into "Social" (Bigby, 2012). In a sense, research into social inclusion focuses on activities that are "inclusive", with Murphy (2009) concluding that social inclusion is often considered a substitute for the quality of life in discussions (Bigby, 2012). The Australian Federal Government constituted Social Inclusion Board in 2012 defined social inclusion as:

Being socially included means that people have the resources, opportunities, and capabilities they need to: Learn (participate in education and training); Work (participate in employment, unpaid or voluntary work including family and carer responsibilities); Engage (connect with people, use local services, and participate in local, cultural, civic, and recreational activities); and have a voice (influence decisions that affect them) (p. 12). This definition has more in common with activities and is consistent with Murphy (2009) and Bigby’s (2012) observations.

This highlights the multi-dimensional nature of social inclusion with researchers, suggesting that it covers both a psychological and physical/material state (Wilson & Secker, 2015). Often discussion on inclusion focus on exclusion, framed around the narrative that we know what exclusion looks like, so it is the opposite. This excludes the possibility that exclusion is a different setting altogether. Secker (2010) proposed that exclusion is a structural factor, whereas inclusion functions are individual. This is consistent with Oxoby (2009), who observed that discussions regarding exclusion tend to focus on barriers to social and political institutions. In that sense, the discussion could be framed along the exact dimensions as cohesion versus inclusion, which we have stated earlier are two states of the same individual defined by their state of self-actualization.

Bigby (2012) observed that social inclusion is an aspect of their existence for people with an intellectual disability and challenging behavior that only happens after they are made ready for social inclusion. In a sense, it is the result of accepting an intervention, implying that some people in society may have to be deemed ready for social inclusion. Beck et al. (2012) defined social inclusion as the extent to which people have access to and are integrated into the different institutions and social relations that constitute everyday life (pp 62). This seems to have a closer relationship to Secker (2010) and Oxoby (2009) and their observations regarding exclusion being a structural setting as opposed to the personal subjective highlighted by Hacking et al., (2008). Is it possible that the structural elements are verging on cohesive factors?

In developing their social inclusion index; Hacking et al., (2008) classified the domains of inclusion as; social isolation, social relations, social acceptance, and individual factors. It clearly articulated the personal or individual subjective nature of social inclusion.

How then do we define social inclusion in the context of the individual? Our view is that inclusion must be a product of the capacity to exercise choice and control, something that programs such as the NDIS promote as their remit. We propose that Social Inclusion is: The capacity of the individual to self-actualize and, in doing so, exercise choice and control over the manner that they lead an ordinary life in the community, whether through active participation or not as they so choose.

The definition of Social Inclusion feeds into the next domain of SQT; Social Empowerment. Beck et al., (2012) defined social empowerment as; "the extent to which the personal capabilities of individual people and their ability to act is enhanced by social relations" (pp. 62). It is stated that this domain is about human capabilities and their development, along with the formation of collective identities, which is a factor in social cohesion. We contend that self-determination is a more appropriate personal construct to measure in the context the measure. In his early work developing vocational programs, Wehmeyer (2003) defined self-determination as; the right and capacity of people to exert control over and direct their lives (p. 68). Wehmeyer (2003) observed that in vocational rehabilitation or employment, enhanced self-determination and choice led to better employment outcomes, mainly when applied to the school to employment transitions. Self- Determination in employment is seen as a factor in enabling people with disabilities to be actively involved in their career development by promoting self-management, self-instruction, self-employment, and self-supported decision making (Kilsby & Beyer, 2002), which support better employment outcomes. This signifies a level of independence.

We argue that the Beck et al., (2012) definition, which includes the capacity to act independently being enhanced by social relations while intrinsically correct, precludes the possibility that self-determined people may and can act on their own volition without being in a relationship or the need to seek permission to act volitionally.

In SQT, social empowerment has within it, knowledge, labor markets, openness and supportiveness of institutions, public space, and personal relationships as primary domains. The indicators within these domains are not exclusively focused on the individual in that they address broader issues such as childcare, housing, government factors, public advocacy, and dissent, among many, most of which imply a level of permission from the state to act or access. Not all of these would be seen as relevant to employment or self-determination at an individual level and seem more akin to European notions of community activation.

Self Determination contains the possibility of acting volitionally and autonomously and the experience of choice (Gagne & Deci, 2005). This would be consistent with Causal Agency Theory (Shogren et al., 2015) that proposes self-determined people are causal agents in one's life and pursue freely chosen goals. Shogren et al., (2015) noted that an agentic person is the "origin of his or her actions, has high aspirations, perseveres in the face of obstacles, sees more and varied options for action, learns from failures, and overall, has a greater sense of well-being" (Little et al., 2002, p. 390). Casual Agency Theory is a framework for developing supports that promote the development of self-determination.

Social empowerment in the context of SQT is not concerned with the individual (Herrmann, 2012) but with institutions and active citizenship and its capacity to utilize public budgets to support social action. Self-determination may be enhanced through social empowerment; measuring self-determination, or the development of this characteristic in the individual receiving employment support, has greater coherence with social inclusion and social cohesion and would illustrate the impact of employment. This is consistent with the idea that persons with higher levels of the personal agency have a higher sense of empowerment (Chang et al., 2017).

We propose that self-determination within the measure be defined as:

The development of the capacity to act as a causal agent with volition and autonomy in exercising choice and control in the pursuit of career and work choices.

The final pillar of the measure is socio-economic security. Ward et al., (2013) defined socio-economic security as:

The extent to which individual people or groups of people have access to and utilization of successful outcomes related to various resources (including finances, housing, healthcare, employment, and education) throughout and over time (p. 3).

These can be seen as outcomes of participation in a society that would be enhanced by employment. With the advent and rise of the "gig" economy, employment security as it is traditionally understood is changing, along with increasing casualization of employment and the workforce. This increasingly makes employment security something of a mirage and signals the potential for a widening disparity of income differences across households, genders, and age groups, threatening workers' socio-economic security and capacity.

Much of the literature has focused on the idea of flexicurity, a contraction of flexibility or successful transitions and security, which is about equipping people with sufficient skills to successfully improve their working lives and find new employment during these transitions (Keune & Serrano, 2014). This places a greater emphasis on the individual attaining economic security through their labours, highlighting the importance of employment.

How then do we define socio-economic security in the context of measuring the impact of employment? The lens of flexicurity would focus on how an individual successfully negotiates employment transitions, a vital transition but not necessarily broad enough to encapsulate employment outcomes that may include access to social welfare as part of an employment pathway. Abbott and Wallace (2012) defined socio-economic security as; "ensuring that people have the resources over time to be able to cope with daily life, enjoy a dignified lifestyle and take advantage of the opportunities available to citizens (pp. 156).

" They stated that it is about employment that pays a decent wage and having access to welfare services to support this over the life course.

Building on these definitions, we propose that the definition of socio-economic security for the measure be defined as:

Having secure employment that pays sufficient income that a person can afford to live a lifestyle that secures access to resources that are available to all citizens and access to welfare that supports successful employment transitions throughout the life course.

The definition context is consistent with the International Labor Organization (ILO) 2004 statement that highlighted seven elements as requisites for decent work. They are labour market security, employment security, work security (OHS), job security, skill reproduction security (skills development), income security and representation security (trade and employer representation). This framework will be used to influence the socio-economic domain within the measure.

The authors have no relevant financial or non-financial interests to disclose.