Acceptability Engineering: The Forgotten Part of Intracorporate UX Design

Social Systems and Resistance to Organizational Change

As the Organisation Development Centre also points out, if an organization’s social system neither experiences nor anticipates change, it appears to be relatively stable. Individuals are familiar with the variety of norms and rules guiding the system, and they have developed means of exploiting these to best meet their own particular needs.

Lewin, a famous social psychologist, referred to this situation as quasi-stationary equilibrium, in which a social system only appears to be stable. There is always movement because of the vast number of tacit, or unconscious, negotiations that allow the system to serve the interests of its individuals, but the totality of all movement equals zero—that is, the system remains in what appears to be a state of balance. Another way of describing these negotiations is as forces whose aim is to stabilize the system—that is, forces that resist changes in the organization.

While it can be useful to consider a social system as a single entity, ultimately, social systems comprise individuals, and organizational psychologists have proven that this quasi-stationary equilibrium serves important psychological functions for these individuals—even if the system is neither satisfactory for everyone nor completely satisfactory for anyone. Further, the work of Macefield and Mellor and others on psychodynamics in organizational contexts explains how some individuals may perceive changes in this equilibrium as a very serious threat, extending even to their physical survival. Importantly, they note that, while individuals are often conscious of such threats—which they may experience as increased work stress—they are unlikely to be consciously aware of the full severity of the perceived threat and are even less likely to express this overtly.

This means that individuals within social systems are typically willing to commit considerable mental resources in mounting robust defenses against changes in social systems. Often, they will not verbalize their defenses—possibly not even in their own conscious thoughts. Thus, the key point here is that resistance to change in social systems can be, and often is, extremely potent, tacit in nature, and difficult to predict!

Socio-technical Impact Patterns

Based on extensive research, Eason, a professor of cognitive ergonomics and a leading socio-technical design thinker, identified a pattern of actions and reactions to the way a new technical (IS) system, business process, or social system tends to play out in situations where there is business-driven change. This is illustrated in Figure 1.

Key and Explanations

A. New business drivers result in a new set of strategic business goals. These translate into a requirements specification for the new technical information system.

B. Once implemented, the new technical system immediately affects the business processes—quite possibly as the organization intended. The immediacy of this reaction is such that, in many cases, the organization designs A and B to act simultaneously, within an approach known as ISE-BPR (Information Systems Enabled Business Process Re-engineering.

C. Soon after, the technical system begins to affect the social system by altering power relationships and levels of individual discretion.

D. A further assault on the social system follows, resulting from associated changes in business processes.

E. There is then a backlash from the social system that affects, or resists, both the technical system and the business process, with the intent of restoring the original state of the social system—that is, to restore the original power relationships and levels of individual discretion.

The intensity and nature of the backlash (E) varies across both individuals and organizations as a whole, but the extremities range from physical sabotage of IT equipment and widespread industrial action through mostly harmless expressions of dissatisfaction and relatively minor reductions in work effort. Common reactions include

• withholding help and encouragement in using the new information system and associated business processes—This is essentially a passive-aggressive approach—that is, aggression that results from not doing something—and often occurs in the interrelationships between front-line workers and their immediate, junior management.
• disrupting the power base—For example, working to gain or regain control or passing on inaccurate or misleading information—either about the system or via the system—through management and the supply chain.
• operating dual systems—This often means keeping a personal system as well as using the new, official system, with the intent of regaining power and discretionary control in the work environment.

Ignoring the Backlash

Understandably, many business professionals get seduced into thinking that they can easily address this backlash by taking an approach that is typified by such comments from project sponsors as these: “We’ll just make them use the new system.”, “Make them do things the new way.” And “We’ll just sack anyone who fails to comply.” Business people making such comments typically justify this approach by citing the theoretical business benefits of using the new system and doing things the new way. Unfortunately, while their thoughts about the potential benefits are often accurate, unlike business processes and the information systems that serve them, a desire for business benefits does not typically drive social systems—nor are they necessarily rational in nature!

It is true that some organizations have successfully addressed this issue by essentially dismantling the social system—usually along with the obsolete business processes. This typically involves radical steps such as breaking up the business and re-employing both new and original staff members, in new or different roles, and on new types of projects—along with a deliberate re-engineering of the corporate culture. We can find some examples of this approach within former, public-sector monopolies that have moved to privatization—for example, British Telecom in the UK. However, this approach typically carries considerable business risk and requires vast cash reserves and highly skilled change agents. To summarize, it is sometimes possible, indeed necessary, to bludgeon your way through a social system, but this is not a viable option in many instances.

Perhaps more importantly, it is naive to believe that an individual can be made to use an information system. This assumption implies that system usage or misusage is a binary condition. In contrast, modern information systems are complex, subtle, and powerful in nature. Therefore, an individual can be using the system, but doing so in an inefficient, slow, or inaccurate way—for example, entering information that is subtly incorrect.

Organizations can, and should, use modern IT systems creatively to exploit new business advantages. Therefore, simply failing to apply such creativity can constitute misusage of the system. In summary, it is usually difficult to conclude definitively that an individual is misusing a system, even more difficult to prove that misuse is taking place, and more difficult still to prove that this is a conscious, deliberate act on an individual’s part.

Addressing the Backlash

Given that some kind of backlash from the social system is inevitable in most cases, it is easy to argue that the best way forward is to acknowledge, accept, and work with the backlash rather than just ignoring or fighting it. This presents two challenges:

• Assessing the nature and intensity of the backlash as early and accurately as possible.
• Using this feedback to adjust the technical development and take deliberate steps to influence the social system in a way that minimizes the negative aspects of the backlash and, indeed, even maximizes any unforeseen opportunities that the disruption in the social system may bring.

Assessing the Backlash

Clearly, assessing such potential backlashes at any point prior to implementation is a challenging problem, but as Eason pointed out, a useful starting point is to consider that a new information system impacts the social system at different levels, as shown in Figure 2.

Figure 2 illustrates that there is a point of contact between acceptability engineering and usability engineering at the interface between the technical system and the user. However, it is important to recognize that acceptability and usability are qualitatively different things that we should not confuse. A system may be highly usable, but totally unacceptable!

Using this framework, Eason developed a technique now known as Eason’s User-Cost Benefit Analysis, for the purpose of assessing the potential backlash from a social system upon the implementation of a new information system.

Eason’s User-Cost Benefit Analysis

Eason’s User-Cost Benefit Analysis consists of a process comprising 3 stages.

Stage 1: Defining the Changes

Based on the new information system’s intended functionality and the associated business-process changes, we can define what we believe will be the key changes that will occur for key individuals, in terms of the following hierarchy:

• Tasks that a particular IS initiative defines:
  
  • Key task 1
  • Key task 2
  • Key task 3 …
• Job security
  
  • Security within the organization
  • Employability outside the organization
• Job content
  
  • Task variety
  • Effort necessary
  • New skills necessary
  • Old skills lost
  • Work pace and deadlines
  • Deadlines
  • Satisfaction
• Organizational
  
  • Discretion and autonomy
  • Standardization and formality
  • Privacy
  • Communication
• Personnel policies
  
  • Basic pay
  • Other rewards
  • Career prospects

Stage 2: Presenting the Impact

We can then present these changes to the individuals who are concerned. There are several key points to note at this point:

• The presentation needs to take a form that is suitable to the audience. Typically, this means it must be free of esoteric formalisms and notations. Demonstrations of functional prototypes can be greatly beneficial.
• Different presentations may be necessary for different audiences, which we can often define in terms of the different personas that we used during the system’s design.
• Eason’s research in applying this technique indicates that the way in which an organization presents these changes is often a key influencer of how well the social system reacts to the changes. Thus, this is a classic case of how the act of measuring something can also affect what is being measured.

The delivery of the presentation to individuals is typically viable only as a group exercise. Sampling individuals who are representative of each persona is often a practical requirement. Similarly, it may also be necessary to abstract those personas that we predict will be key to the success of an initiative and subject to the greatest change resulting from the introduction of the new information system.

Stage 3: Analyzing the Impact

Following the presentation, there are two complementary methods of collecting feedback. The first is via group discussions and interviews immediately following the presentation. This qualitative feedback is important or two reasons:

• It carries the benefit of flexibility and, therefore, affords the possibility exploring key issues.
• It demonstrates that the management team conducting the acceptability testing is truly engaging with the concerns of the social system, which in turn, may render the acceptability testing an intrinsically positive influencer of the whole initiative.

However, the lack of anonymity and the sensitivity of the issues can mean that such forums are unlikely to produce much of the key information that we want elicit. To address this issue, we can also ask participants in acceptability testing to complete Eason’s User Cost-Benefit Questionnaire. A generic version of this questionnaire appears in Table 1.

Table 1—Eason’s User Cost-Benefit Questionnaire

Level	Proposed Change	Benefit	Cost	Comments
Task
Key task 1
Key task 2
Key task 3 …
Job Security
Security within organization
Employability
Job Content
Task variety
Effort necessary
New skills necessary
Old skills lost
Work pace and deadlines
Deadlines
Satisfaction
Organizational
Discretion and autonomy
Standardization and formality
Privacy
Communication
Personal Policies
Basic pay
Other rewards
Career prospects
	Total scores:

Using a User Cost-Benefit Questionnaire

There are three stages in the usage of a User Cost-Benefit Questionnaire.

Stage 1: Questionnaire Design

The generic questionnaire shown in Table 1 should be tailored to the specific IS initiative—possibly in relation to the different personas for which acceptability is being tested. In particular, we must carefully identify, then clearly define the key tasks, then add all of the key proposed changes to column 2.

Stage 2: Questionnaire Completion

Individuals complete the questionnaire, as follows:

• Participants score both the cost and the benefits that they perceive each change will bring. We explicitly ask participants to consider these costs and benefits as an individual rather than considering any wider organizational benefits.
• Scoring is on an integer scale of -5 to +5, where a 0 means no perceived cost or benefit. An example of a -5 cost might be that the participant would be concerned about becoming ill from work stress, would want to leave the organization, or would take legal or industrial action if the proposed change were realized. A +5 benefit would mean that the proposed change is eagerly anticipated and perceived as having the potential for career enhancement or an improved lifestyle.
• Encourage participants to make a comment against each proposed change, with the particular purpose of qualifying the change. For example: “I would be happier with this change if….”
• Give participants careful advice on how to complete the questionnaire.
• Participants complete the questionnaire anonymously. A robust mechanism must be in place to ensure their anonymity, and we must clearly and convincingly explain its robustness to participants.

Stage 3: Questionnaire Analysis

Analysis of the questionnaire has three components:

• Pay particular attention to high scores—4s and 5s for costs or benefits—because these indicate potentially show-stopping problems or significant business opportunities for the IS implementation. Of course, we should note any comments participants make regarding these scores with particular interest.
• Review all comments and cross-reference them with the qualitative feedback elicited from the discussions and interviews.
• Perform a statistical analysis on all of the questionnaire data to form an overall picture of the severity and the nature of the social-system backlash. In keeping with any statistical analysis, seek a reasonable sample size to maximize statistical validity. This requires a minimum of about 12 participants for each persona for which we are testing acceptability. For large-scale, global implementations, we should typically ensure some cultural or regional diversity within each persona group.

Acting on Conclusions from the Analysis

Hopefully, our conclusions from the analysis will enable us to adjust the IS development and implementation plan to minimize the negative aspects of the predicted social-system backlash and maximize the realization of any identified opportunities. In many instances, we’ll be able to avoid or largely mitigate serious problems arising from the backlash.

Of course, there will also be cases in which the new technical information system and business processes that an organization requires may unavoidably result in serious conflicts with the social system. In such cases, we may need to fight and win a battle with the social system! This reality prompts some to argue that acceptability testing may be redundant. However, even when a battle is inevitable, it is extremely useful to know that it is, as well as to have a prediction of the nature of the battle ahead, in advance of completing the implementation.

The Role of UX in Acceptability Testing

In many cases, experts in organizational change—who often have organizational psychology backgrounds—conduct acceptability engineering. This often works quite well, but there are two reasons why I would like to see more UX professionals involved in this area:

The first is that, as I mentioned earlier in this article, early prototypes can be very useful in helping to communicate to users the nature of the changes that a new information system will bring to an inward-facing information system and make them more tangible. Clearly, the production of such prototypes is the domain of the UX professional. However, it is important to note that the nature of prototypes for acceptability engineering may need to be quite different from those that we produce for usability testing, stakeholder demonstrations, or to communicate technical specifications.

As I also stated earlier, the acceptability of an information system must be part of the user experience of that information system. In my opinion, this means that acceptability engineering is intrinsically and inevitably the business of UX professionals. 

Reference

Eason, K. D. Information Technology and Organisational Change. London: Taylor and Francis, 1988.