In Part 1 of this three-part series, I discussed some key remaining problems with the software-development lifecycle (SDLC) as teams now typically realize it—in particular, those problems relating to the profession of User Experience. In that article, I pointed out how the analysis of problems generally leads to our learning the lessons on which we ultimately base our solutions.
Now, in Part 2, I’ll explore the key lessons that I’ve learned through the efforts of my company Ax-Stream to improve the SDLC, then set out the basis of the proposed solutions that I’ll present in Part 3.
In discussing how to improve the SDLC, I’ll employ the perennial comparison between large software-engineering projects and civil-engineering projects. People often draw this comparison because both types of projects are typically novel, complex, expensive, time consuming, critically important, and involve significant risk. However, unlike software-engineering projects, civil-engineering projects usually tend to proceed as expected and come in roughly on time and on budget.
Champion Advertisement
Continue Reading…
People who are familiar with the plethora of both commercial and academic material making this comparison are probably aware that one hypothetical reason why large software-engineering projects perform relatively poorly is because the discipline is relatively inexperienced, having existed for only decades. In contrast, people have been successfully delivering large civil-engineering projects for thousands of years. Others argue that software engineering is fundamentally a different discipline from civil engineering and, therefore, any such comparison is invalid.
It is certainly true that there are important differences between the software-engineering and civil-engineering disciplines. For example, technologies change far more frequently in software engineering and requirements are much more likely to change significantly during software-engineering projects. Similarly, it is easy to argue that modern software is much more complex than any physical structure.
While acknowledging these important differences, I remain convinced that the software-engineering discipline can still learn some key, high-level lessons from the domain of civil engineering. These lessons relate to the following:
developing a shared vision for a project
concept validation
functional-requirements definition
design detailing
communication across the different functions on a project
Shared Vision
Similar to many other engineers, when Peter Checkland made the transition from managing large civil-engineering projects to managing large software-engineering projects, he found that the software projects failed much more often, resulted in poor outcomes even more often, went over on time and budget more often, and were simply a nightmare to get right.
Checkland was clever enough to realize that these poor outcomes did not occur as a consequence of a management problem and set out to discover exactly what kinds of problem actually existed. His discoveries ultimately led him to develop to a whole new way of thinking about software-engineering methodology called Soft Systems Methodology. Its key point was Checkland’s shocking discovery regarding the lack of a shared vision for software projects. When researching the causes of failed software projects, he asked each of the key stakeholders what they thought they were building, why they were building it, and how they thought it would basically work. To his surprise, their visions varied greatly. It was like some thought they were building a suspension bridge for cars, others thought they were building a depot for trains, while still others thought they were building a small town! I don’t need to explain how this would inevitably result in the failure of a project.
Within Soft Systems Methodology, Checkland addressed this lack of shared vision through the use of root-definition documents. Root definitions pin down a shared vision for a project, making it the primary and critical objective at the beginning of any software project. This is why I use them on all of my projects, including when rescuing failing projects.
However, while root definitions are indispensable, they are abstract in nature because they are very high-level text documents. So we need to further pin down root definitions by building holistic conceptual prototypes that define the software’s scope and key features, in a concrete form, at an early stage of a project. Similar to Checkland’s root definition, building holistic conceptual prototypes is a highly iterative process whose facilitation requires specialized tools, techniques, and expertise.
Civil engineering has used such conceptual models extensively across its history. This approach also resonates well with modern, Lean thinking. However, in my experience, the creation of conceptual prototypes is a part of the SDLC that is massively underrepresented on current software-engineering projects, despite their being so fundamental to success.
Concept Validation
On large civil-engineering projects, an architect works closely with structural engineers and construction specialists to ensure that the concept is viable at an early stage. This avoids wasting time and money exploring designs that cannot be built. Further, these specialists—who lead the later phases of a project—can sometimes provide useful insights that enable a team to meet the requirements with designs that are easier to build and more robust.
This approach contrasts with my typical experience with software-engineering projects, on which business requirements and designs simply get passed to development teams, who must just get on with implementing them. This problem is exacerbated when business analysts (BAs) and UX designers lack any significant awareness of the implications of their requirements definitions and design decisions for the development team. I have even heard UX designers say: “I don’t care how difficult my design is to code. I want it like this.” This sometimes happens when an alternative design that would be much easier to code could also provide an equally good, or better, user experience, while meeting the requirements.
The key lesson we can learn from this problem is also in keeping with Lean UX and true user-centered design (UCD) thinking: Early concept design should be a multidisciplinary exercise, involving the leads responsible for all activities that occur later in a project’s SDLC. Such validation should take place early during a project. A benefit of this approach is that it also facilitates team buy-in and group cohesion for the project.
Functional-Requirements Definition
As I discussed earlier, many argue that requirements for software-engineering projects are more difficult to meet than those for civil-engineering projects because software requirements inevitably change more often throughout a project. This is true. I shall return to this issue in the final part of this series. However, in my opinion, failure to meet software requirements often occurs because the requirements are incorrectly or ambiguously defined in the first place.
Software-engineering projects have made much progress in the area of requirements definition. For example, I particularly like defining persona-driven requirements that are supported by good user research—an approach that both Lean UX and UCD advocate. But such requirements are often abstract text documents that no one follows up on or maps to concrete artifacts.
Ideally, iterative prototyping should model every requirement as an interactive feature or user journey. The prototype would then constitute both a requirements definition and a communication device that teams can share. This would ensure that all stakeholders know whether each user need and business requirement has been met, as well as the means by which they have been met.
Design Detailing
Civil-engineering practice includes a skilled professional function called design detailing, which translates the architect’s vinos and conceptual and fundamental designs into a highly unambiguous set of specifications drawings and documents that tell the construction engineers exactly what they need to build. This function necessarily includes leads from the construction team, who ensure that the detail design is viable and they can build it in a highly optimal way.
As I discussed in Part 1 of this series, the coming of the UX design profession has introduced more concrete design artifacts such as screen designs and wireframe models. However, I still see a lot of ambiguity in the information that designers pass to development teams, with the result that designs often get built incorrectly, necessitating costly rework.
This happens because the designs are not specified in sufficient detail. In particular, the way interactions should function is often missing altogether or is poorly modeled, providing inadequate detail. It is also common for there to be insufficient involvement of development leads during UX design.
In software engineering, there is simply no equivalent of the comprehensive detail-design documentation that exists in civil engineering. This is a key reason why software engineering often involves a lot of expensive and time-consuming rework, while civil engineering does not!
Communication
Some professionals working in information-technology (IT) possess skills that span two or more of the functions necessary for software-engineering projects, which aids effective communication. For example, UX designers who have a technical background can at least speak the basic language of the developers. Plus, the widespread use of agile scrums has greatly improved cross-functional communication. Some multidisciplinary design teams practice true UCD or Lean UX methods.
Despite all of this, cross-functional communication on many software-engineering projects is still lacking in comparison to civil-engineering projects. Perhaps because of the complexity of software engineering, too few IT professionals have a holistic view of the SDLC or are skilled in more than one function.
A paradox seems to exist: As I pointed out in Part 1, while roles are often ambiguous on software-engineering projects, there is still too little cross-functional collaboration or fluidity across roles. However, this is starting to change. For example, I recently trained a group of business analysts and UX designers in conceptual system design at a major financial corporation. They recognized that conceptual design necessarily involves both the requirements-engineering skills that business analysts typically have and the concept-modeling skills that UX designers typically have.
More importantly, many communication problems on software-engineering projects are the result of using too many different tools. This is particularly true for the UX functions. The more tools a team uses, the more communication problems result, and the more time a team spends managing data-translation and data-integrity issues.
I recently read a blog post in which a newbie UX designer was naively asking why he needed to do his design layout in Sketch, then convert it to Axure to add some interactions for usability testing. He said he could do the layout almost as quickly in Axure, so there was no need to do the conversions or to synchronize the files as he iterated his design. Of course, the answer to his question was that he didn’t need to use Sketch at all!
The key lesson: minimize the number of design tools you use throughout the SDLC rather than selecting different tools that are arguably optimal for each specific design task. Again, the civil-engineering discipline is ahead of UX design on this issue. Civil engineers often use a single powerful computer-aided design (CAD) system that can more seamlessly handle conceptual design, simulation—for example, building walkthroughs—and detail design, with the construction team accessing the design within the same CAD system at the build stage.
A further lesson is that different civil-engineering professionals who have different needs can use different features of the same CAD system, even though they might not be skilled in all aspects of the system. For example, the architect might use the 3D conceptual–modeling features, while the detail designers would be experts in using the drawing and specification features. This contrasts with the less efficient, riskier approach that seems to be prevalent in the software-engineering discipline, in which people in different professional roles use totally different software tools.
Overall Lessons
Looking at the SDLC as a whole, two key lessons have emerged from my company’s analysis of modern software-engineering methodology in comparison to the civil-engineering discipline.
First, following the root definition, projects seem to break down naturally into three key stages:
Conceptual design, which also incorporates user research and requirements engineering
Detailed design
Build
There seem to be clear demarcations between these stages. Therefore, they naturally become key sign-off or commitment points. While this thinking has a lot in common with waterfall approaches, that’s not a bad thing in my opinion!
Second, a single advanced modeling tool should support all three of these stages. Therefore, we need IT professionals and, in particular, UX professionals who are highly proficient in the use of these tools.
Ritch has worked in UX design since 1995. He has a BA with Honours in Creative Design, an MSc with Distinction in IT-Computing, and a PhD in Human-Computer Interaction (HCI) from Loughborough University’s HUSAT (HUman Sciences and Advanced Technology) Institute. Ritch has lectured at the Masters level in five countries—on user-centered design, UX design, usability engineering, IT strategy, business analysis, and IT development methodology. He also has numerous internationally recognized qualifications in IT-related training and education. He has published many HCI articles and was on the editorial team for the Encyclopedia of HCI (2007). Ritch has led major UX design and user-centered design projects at Bank of America, Vodafone, Thomson-Reuters, and Dell Computers. His international experience spans 15 countries. Ritch presently heads Ax-Stream, an approved Axure training partner. His work currently encompasses Axure training, advanced Axure prototyping, usability and acceptability testing of early conceptual prototypes, coaching and mentoring of senior UX designers, and strategic UX work—integrating UX design and user-centered design into agile development methods and moving organizations from second and third generation to fourth generation prototyping methods. Read More
