The Essen Climate Evaluation Schema - EssenCES - A Manual and More

[8][9]3

Administration and Scoring

Matthew Tonkin

Introduction

This chapter discusses issues relating to the administration and scoring of the EssenCES. It covers why the EssenCES should be completed, who it should be given to, how many completed questionnaires are needed to gain a reliable estimate of social climate, how the EssenCES should be filled out, how to score the EssenCES, and how to deal with missing responses. The limitations of using the EssenCES with certain populations (e.g., those with an intellectual disability) are also discussed.

Why Should the EssenCES Be Used?

The very first decision that must be made when implementing the EssenCES in practice is why the monitoring of social climate is needed. Generally speaking, there are two basic situations in which the EssenCES can be used to inform the[10] functioning of secure units (Tonkin & Howells, 2011). First, the EssenCES can be used to evaluate the impact of a given intervention. For example, a secure unit might adopt a new method of working that involves significant staff and resident restructuring. Alternatively, the intervention might be a particular treatment program that is designed to improve the interpersonal relationships between residents and staff. In both instances, the EssenCES can be administered before and after the intervention to determine whether any change in social climate has occurred. Thus, the EssenCES can be used as an indicator of change/treatment success.

Second, the EssenCES can be used to inform the functioning of secure units as a means of monitoring the social climate over a period of months or years (Tonkin & Howells, 2011). In this context, the EssenCES would be completed at regular intervals to determine whether and how the social climate of a given unit fluctuates over time. This would help to identify particular wards/wings where the climate is not conducive to the safe and successful rehabilitation of offenders (Tonkin et al., 2012). The EssenCES profile of that particular ward/wing could then be used to guide the development of interventions that are designed to improve the social climate.

Thus, the EssenCES can be completed as part of either a short-term or a longterm evaluation of secure units. That is, it can be used to monitor how safe, secure, and supported residents and staff feel within a given unit over time, or it can be used in a more targeted manner to evaluate the specific outcomes of an intervention.

Who Should Complete an EssenCES Questionnaire?

Once the reason for monitoring the social climate has been determined, one of the next issues is to whom the EssenCES should be administered. To some extent (as with most of the issues discussed in this chapter), the answer depends on why the EssenCES is being used in the first place. Nevertheless, in most instances the key issue is to ensure that a variety of people with different roles within the unit complete the EssenCES because different people will have varied attitudes towards and perceptions of the “same” social climate (Britton, 1997; Wright & Saylor, 1992). This means that both residents and staff should participate in the assessment, including staff from various professional backgrounds, such as nursing, psychology, medical, and social work. This will help to ensure that a varied and balanced view of the social climate is obtained (Tonkin & Howells, 2011).

Despite the importance of diverse staff and residents completing the EssenCES, it is also vital to exclude raters who may not be able to give a reliable assessment of the social climate. Certain individuals (e.g., cleaning or secretarial staff) may not have sufficient contact with a ward/unit to offer well-founded assessments of the social climate, for example.

[11]Is the EssenCES Questionnaire Appropriate for Everybody?

The questionnaire was designed for use with patients in forensic institutions, who frequently have little education and average or below average intelligence. Therefore, questionnaire items were developed with a view to being easy to read and understand. However, a basic level of thought and language capabilities is essential in order to provide appropriate EssenCES responses. The use of the EssenCES questionnaire (in its current form, at least) with intellectually disabled populations or with individuals who have experienced brain injury is, therefore, not recommended until sufficient evidence of its validity has been published. Readers who are interested in the suitability of the EssenCES for intellectually disabled individuals are referred to several recent publications (Alderman & Groucott, 2012; Milsom et al., 2014; Quinn, Thomas, & Chester, 2012).

How Should the EssenCES Be Administered to Staff and Residents?

Once the decision has been made to use the EssenCES, the method of administering the questionnaire needs to be addressed. For example, should the questionnaire be sent to staff/residents through the unit’s internal mail system or should they be approached in person? Should the questionnaire be handed over by someone the participants know or by someone unfamiliar to them? Again, there are no right or wrong answers to these questions, and the most appropriate approach will depend on the purpose of the monitoring, the type of residents being surveyed, and other logistical/staffing issues. It might also be helpful to keep in mind that assessments may be more realistic and valid if staff and residents participate upon invitation, rather than being pushed to do so.

However, it is important that all participants are allowed to complete the EssenCES questionnaire anonymously and confidentially. The importance of this principle is clearly highlighted by Moos (1975), who found substantial differences in the perceived social climate between participants who were allowed to respond anonymously and those who were asked to give their name. Typically, anonymous responders had a more negative perception of the climate than those who were clearly identifiable. While it is understandable that there might be pressure to obtain high EssenCES scores from managers or other policy makers, it is of paramount importance that participants are assured that they can be honest about their feelings and perceptions. If a unit does not ensure assessment anonymity and confidentiality, it may invalidate the monitoring of its social climate.

[12]How Many Completed EssenCES Questionnaires Are Needed for an Assessment?

Another important question is how many completed questionnaires are required in order to get a reliable estimate of the social climate. For an effective assessment, it is not necessary that all patients/inmates and members of staff participate in the evaluation. In discussions of other questionnaire measures of social climate, researchers have suggested that between 25% and 50% of individuals within a given unit should complete a questionnaire (Moos, 1975). For the EssenCES, it has been proposed that 7–10 completed questionnaires per ward/wing and per group (i.e., patients/staff) are necessary to provide a sufficiently reliable assessment of social climate. It must be pointed out, though, that this suggestion has not yet been fully tested. Furthermore, it does not take into account differences in ward/ wing size, which would impact on the number of completed questionnaires that are needed. Therefore, the most valid recommendation at present is for administrators to obtain as many completed questionnaires as possible within the limits of time and cost. This is particularly important where the number of staff and residents on a given ward/wing is small.

What to Do With the EssenCES Questionnaires Once They Have Been Completed?

Once the questionnaires have been completed they must be scored and interpreted. This section deals with the various issues that occur at this stage of an evaluation.

First, each item on the EssenCES questionnaire must be given a score ranging from 0 to 4 (see the Appendix for scoring details). With the exception of items 1 and 17, which are unscored filler items, each item on the questionnaire corresponds to one of the three EssenCES subscales (Patients’/Inmates’ Cohesion, Experienced Safety, or Therapeutic Hold/Support). There are five items per subscale. A subscale score is calculated by adding the five relevant responses, which creates a score that ranges from 0 to 20. High total scores are indicative of a positive social climate, whereas low scores indicate a negative climate.

As the EssenCES is a very short questionnaire, the problem of missing responses sometimes encountered with more extensive measures does not occur very frequently. However, in individual instances a participant may not provide responses to all items.There are explicit guidelines for dealing with missing responses. Provided that there is no more than one response missing from each of the three subscales, the mean score obtained across the other four questions on that subscale can be used as a substitute for the missing value (see Box 3.1).