We Don't Need No Architects

by Joseph Hofstader

Contents

Introduction
Who Are Those Guys?
Problem Space
Solution Space
Don't Build "The Homer"
To Make a Short Story Long...
Resources

Introduction

The role of an architect in software development has come under attack of late. On software projects, the title Architect is often ambiguously defined and the value provided by architects is not easily quantifiable. The perception that architects live in an "ivory tower" disassociated from the realities of delivering a software solution contributes to some of the animosity toward those of us with the title.

This article presents a defense of the practice of architecture in software development. It defines the qualities of an effective architect and describes the skills required to succeed in this profession. The article examines widely held perceptions of architects and some of the mistakes that architects make which contribute to negative perceptions. Ultimately, the intent is to show the value good architects bring to a software development effort.

Who Are Those Guys?

In the field of information technology, no title conjures up more raw emotion than Architect. I have witnessed and been involved in many debates over the definition of the term. When I introduce myself at meetings, reactions range from "we're not worthy" to "we do not need an architect"—the former, although friendly, reflecting the lofty image of architects, and the latter implying that an architect's knowledge and skills are irrelevant. Both responses demonstrate a lack of understanding of what architects really do.

At the OOPSLA conference in 2005, I attended a "Birds of a Feather" (BOF) hosted by Grady Booch. The topic of the BOF was his then upcoming "Handbook of Software Architecture." One of the attendees related some negative experiences he had had with architects, both in IT and in construction. One story was about the architect who drew plans for his house expansion. The attendee said that he viewed the drawings with engineering software and the plans were off by a few feet, and that the actual construction could not and did not follow the architect's specification. He was making the point, which I have heard echoed by many qualified individuals, that architects are detached from the reality of delivering tangible results and that their responsibilities should be relegated to the engineers and builders who are fully engaged in product development.

That meeting, and many subsequent conversations with others, led me to wonder what exactly the role of an architect is on a software product and what the characteristics of good architects are. The most concise definition I have come up with is: The role of the IT architect is to solve a problem by defining a system that can be implemented using technology. Good architects define systems by applying abstract knowledge and proven methods to a set of technologies with the goal of creating an extendible and maintainable solution.

From this concise definition, we can extrapolate that good architects draw upon a foundation of knowledge to be successful in their role. To "solve a problem," the architect must have a good understanding of the problem domain. To "define a system using technology," implies that the architect has technical acumen. "Abstract knowledge" requires the architect to be able to conceptualize the technical solution. "Proven methods" assumes an understanding of the patterns used to solve similar problems. Figure 1 depicts the key skills of an architect.

Figure 1. The key skills of an architect

The key benefit an architect brings to a project is the ability to apply those skills in the definition and development of a robust software solution. An effective architect is part of the continuum of all phases of a software development project, with skills critical to defining the problem space, such as domain knowledge and the ability to conceptualize a software solution, as well as the ability to define the solution space using appropriate technologies and patterns. The risk of not having an architect actively involved in product development increases the likelihood that the product will take too long to develop or be developed incorrectly. Figure 2 illustrates the phases of a development project where the skills of an architect are applied.

Figure 2. Architect skills in phases of software development

Describing the architectural skills required for a successful project is not as straightforward as it may seem. Many areas, especially technical acumen and patterns, are often debated regarding the level of expertise necessary for an architect. The following sections, divided by problem space and solution space, offer an explanation of each of these skill sets and a rationalization of how these skills make an architect invaluable to a software development effort.

Problem Space

Defining the problem space and ultimately setting the scope of a software solution requires an understanding of what will be built, as well as domain knowledge and a conceptualization of how information will flow through the solution. As the following sections detail, these skills are essential to understanding the purpose of a software solution and communicating the proposed solution to all stakeholders.

Domain Knowledge

The problem domain for a software solution can be horizontal or vertical. A horizontal domain is applicable across industries, like workflow automation. Vertical domains are specific to a particular industry, like telecommunications. Problem domains can be further decomposed into subdomains, or aggregated into larger domains. An example of a subdomain within a vertical domain is network activation in telecommunications. An example of the aggregation of a subdomain into a larger horizontal domain is workflow in an enterprise application integration product.

There are many standards organizations and vertical industry groups that specify standards and protocols that need to be considered when defining a software solution. These organizations can be specific to a vertical industry domain or a horizontal industry domain. The TMForum is an example of a vertical organization that specifies management protocols for the telecommunications industry. The W3C specifies standards for the horizontal World Wide Web domain including technologies like Web services.

The value of domain knowledge is sometimes underestimated by IT managers. I once worked for a telecommunications company whose IT leadership wanted to change the organization from being structured around "centers of excellence" focused on a business domain to being structured with "pools" of resources based on technical skills without regard to domain knowledge. For some of the resources assigned to horizontal domains, like Web development, this model worked well. Many products require Web interfaces and the skills were applicable across verticals. Where the "resource pool" structure failed was in industry specific subverticals, like network activation. Understanding how to provision and activate services requires detailed knowledge of the provisioning process as well as the interfaces, protocols and standards of all network devices that comprise the telecommunications services.

Deep domain knowledge often involves a steep learning curve. If the staff on every project is required to learn the intricacies of the domain for every release of the project, productivity is significantly reduced. Assuming features are sufficiently decomposed so that the amount of time to deliver the feature is constant, then productivity suffers proportional to the amount of time spent acquiring domain knowledge. Conversely, maintaining a "center of excellence" around each vertical domain supported by a business can also be an expensive proposition. Development work is seldom evenly balanced throughout a given timeframe and keeping a fixed number of resources assigned to a project that is not actively engaged in delivering solution features can drain productivity on other development efforts.

A balance that is common in technology companies is having an architect be a domain expert and a pool of resources available to different projects. This strategy increases productivity by minimizing the amount of time obtaining domain knowledge within a vertical. It also allows some preliminary research to be done prior to engaging development staff, helping to ensure that the development is consistently productive. This approach provides the company the added benefit of a flexible staffing model, in that development staff can be augmented with contractors without having valuable domain knowledge walk out the door at the end of the contract.

Conceptual Thinking

One of the main responsibilities of an architect is to communicate the solution to technical and nontechnical stakeholders. The ability to conceptualize a software solution is essential to communicating a software solution to stakeholders who care about delivery of functional requirements, and may not know or care about technical details. Defining a conceptual architecture prior to commencing the development of a software solution helps facilitate the feedback loop needed to define the scope of a product and can help determine an initial level of effort and cost estimate for a product.

A conceptual model is the artifact most associated with software architecture. The conceptual model typically shows the components of a software system that will fulfill the functional requirements and where they apply in a software solution (user interface, domain layer, and so forth). The conceptual model is often accompanied by a number of diagrams showing the flow of information through the proposed solution. In the case where the software system consists of components from other products or solutions, the conceptual architecture often contains the protocols that will be used to access different parts of the solution.

Applying the correct level of granularity is the main challenge in defining the conceptual model. The conceptual architecture should not contain any references to a particular platform or technology, other than protocols used to access subsystems. I once took over as architect of a project that was stalled in "analysis paralysis" for over a year. As I was reviewing documents to get up to speed, I noticed that the conceptual architecture consisted of a number of boxes containing the names of technologies with no reference to system functionality. After reviewing that document I understood why the system could not be developed: There was no mention of the required features, making it hard to understand what needed to be developed.

Solution Space

It is in the area of defining the solution space that opposition to architecture is most obvious. Developers will usually accept the architect working in the problem space, but may be resistant to having the architect define the solution space. In many instances, developers have valid arguments about architects meddling in the solution space, especially if the architects have not kept their technical knowledge up-to-date.

A colleague of mine illustrates the attitudes developers have toward architects when he says "architects talk in bubbles and developers talk in code" (see Figure 3). The idea that code is the only artifact that matters in a software development project is so prevalent that it is one of the values listed in the Agile Manifesto: "We have come to value […] working software over comprehensive documentation." A good architect understands that code is undeniably the most critical part of a software solution, and many of the modern development environments now produce code from "bubbles," including tools that support Model Driven Architecture (MDA) and Domain Specific Languages (DSL).

Figure 3. An architect thinking in bubbles

That being said, a good architect also understands that a software solution consists of more than the functional requirements implemented in custom code (see Figure 4)—for example, development platforms, frameworks, and infrastructure technologies. Consideration also needs to be given to the nonfunctional requirements of a software solution, like: deployment, security, scalability, and performance. Neglecting the nonfunctional requirements increases the likelihood that the system will fail.

Figure 4. A developer seeing part of the picture

Another critical piece of solution space knowledge is the patterns used to implement a software solution. Patterns allow a software solution to be developed with thought toward extendibility and reuse, which are critical to reducing the total cost of ownership of a software solution—especially in later phases of development or product maintenance.

Technical Acumen

Technology has been rapidly evolving for as long as anybody can remember. I've implemented countless technologies over the last dozen years, including technologies for process distribution, user experience programming languages, enterprise application integration, object-relational mapping, virtualization, and data persistence.

Understanding how a technology works is not enough to develop a robust software solution—understanding where the technology is applicable within a solution is essential to the development of a quality product. On several occasions, I have reviewed architecture documentation consisting of little more than a number of boxes on a Visio diagram each representing a technology, with no mention of how the technology was intended to be used or any reference to the functional requirements to be fulfilled by the technology. Such practices give architects a bad name.

It is impossible for anybody to understand every detail of every technology. But an architect should understand, at a minimum, the intent and applicability of any technology prior to requiring its usage in the context of a software solution. The architect should also map the technology to the functional requirements, or to the applicable part of the conceptual architecture. Too often, I encounter the bad practice in which an architect makes a technical edict without explaining the intended application of the technology. To be honest, I made the same mistake on occasion earlier in my own career.

Architects sometimes allow their passion for technology to overshadow the problems that they need to solve. The art of choosing technology for a software solution is finding the minimum amount of technology required to meet the system requirements, both functional and nonfunctional. Delivering a software product that meets all quality standards, such as performance and scalability, will require a good amount of technical knowledge, and it is the job of the architect to define the platform for development and deployment.

With all of the advances in technology, keeping abreast of the latest trends can be daunting, but it is critical for an architect. One company that I worked with had a problem with the performance of a client/server reporting application. The application had been extended for years without a technology update. The architect responsible for the solution was adamant that building an object-layer over the relational model would solve the problems with his application. The proposed solution would have been status quo a decade ago, but database technologies have advanced greatly over the last decade and now contain optimized reporting services as part of the platform. The solution proposed by the architect would have increased the total cost of ownership of the solution (development, maintenance, licensing) and most likely would have adversely affected performance. Luckily, the architect was open to learning about the newer technologies and the product upgrade took advantage of the capabilities in the newer database platform.

Patterns

One critical skill possessed by all great architects is an understanding of how to leverage patterns in the definition of a software solution. Patterns have been a mainstay of software development for over two decades. Through the seminal Design Patterns by Gamma et al., the Pattern Oriented Software Architecture (POSA) series of books, various publications from industry luminaries, and the work of organizations like the Hillside Group, patterns have become a key measure of how software is judged. When reading about a new technology or a framework, I often try to list the different patterns that were used in the solution in order to assess the quality of the design.

The mainstream usage of patterns in software development has made it a critical skill for architects. Learning how to leverage patterns takes time and effort. Some architects, like me, have had the good fortune of working with experts in patterns who provide mentoring in the discipline of architecture and design; others are left to acquire these skills on their own. With or without mentors, developing a proficiency in patterns does not happen by accident and requires dedication. There is a large learning curve in acquiring the patterns vocabulary and an even larger learning curve in understanding where patterns can be applied in a software solution. The effort put into learning patterns is paid back tenfold, giving an architect the necessary skills to intelligently discuss design with their peers and to create extendable software systems.

One of the main motivations to leveraging patterns in a software solution is the ability to design frameworks that allow the solution to be easily extended. A software solution uses frameworks to define the solution's architecture in the context of the problem domain. Good software frameworks are usually defined using general-purpose patterns that are reusable over a number of domains. One of the main drivers behind domain-specific languages is to increase developer productivity by providing graphical tools to customize general frameworks. As mentioned previously, patterns are an essential component of defining frameworks.

I can provide many examples in which understanding patterns has increased productivity, but the best example was in the project I mentioned earlier which I inherited after it was stalled for over a year. Having designed similar solutions in the past, I understood the patterns that were necessary to build an extendable domain model. While equipment was being ordered for the lab and development staff was being interviewed, I was able to define the domain model and frameworks with patterns that I had used on similar efforts in the past, accelerating the development phase. The initial frameworks I developed were a key factor in being able to deliver the product in a short time frame.

Don't Build "The Homer"

With all of the skills possessed by good architects, it is often challenging to narrow the numerous options to be used in a software solution. Between the standards defined for specific domains, the alternatives for conceptualizing a software system, the plethora of technological options, and the numerous patterns to promote extendibility and reuse, it is easy to over-architect a solution that bears a greater resemblance to alphabet soup than to a robust software system (see Figure 5).

Figure 5. An architect making alphabet soup

The first step in simplifying a software system is to understand that no matter how fun it is to try out new techniques and technologies, they must be applied in the context of a system requirement. It was no coincidence that the first system I designed after reading Design Patterns contained almost all of the patterns defined in that book.

Many years later, after a few painful experiences, I have learned that a minimalist approach, using the least amount of technology to fulfill the requirements, always yields the best results.

There is an episode of The Simpsons I often think about when defining a software solution. In the episode, Homer, the dim-witted father, was asked to design a car for an auto manufacturer. The design for Homer's car was full of nonessentials like multiple horns, cup holders, and shag carpeting—without thought to the overall cost or customer appeal of the car. The prototype, appropriately called "The Homer," was so costly and unappealing that the product bankrupted the company. Whenever I design a solution, or review the design of a solution, I frequently stop and ask myself if the resulting product is beginning to resemble "The Homer." If the answer is yes, I sharply refocus on the essential functionality and technology required to deliver the software system.

To Make a Short Story Long...

A couple of months ago, while waiting to make a presentation at a customer event, I was introduced to a developer attending the event as an architect from Microsoft. Expecting one of the usual greetings like "pleased to meet you," I was surprised that the first thing he said was "I think that architects are obsolete." Not wanting to be roped into an argument before my presentation, I responded with a grin and said, "I hope not."

Upon reflection, I have wondered why he felt he needed to make that statement from the outset: Was he having a bad day? Did he have a bad experience with an architect? Perhaps he is unaware of the contributions of architects in software development. Maybe his solution was so well architected that the development staff only needed to consider coding and testing functional requirements without needing to understand the considerations necessary to create a robust software solution.

A poorly architected project languishes without results, or is delivered with poor results. On the other hand, a successful software solution is well architected, by someone with domain knowledge, the ability to think conceptually with technical acumen, and the ability to leverage patterns to solve problems. Industry titles for this role have come to include architect, technical lead, and a number of others, but the importance of the role is clear. We should stop wasting time arguing the virtues of the role and continue solving problems with technology.

Resources

The Agile Manifesto

https://agilemanifesto.org/

Design Patterns: Elements of Reusable Object-Oriented Software, Erich Gamma et al. (Addison-Wesley Professional, November 10, 1994, ISBN: 978-0201633610)

Handbook of Software Architecture

https://www.booch.com/architecture/index.jsp.

The Hillside Group

https://hillside.net/

The Homer Wikipedia Entry

https://en.wikipedia.org/wiki/Oh_Brother,_Where_Art_Thou

"Using Patterns to Define a Software Solution," Joseph Hofstader, November, 2006.

https://msdn2.microsoft.com/en-us/library/bb190165.aspx

"Using Patterns for Rapid Development," Joseph Hofstader, February, 2007.

https://www.iasahome.org/c/portal/layout?p_l_id=PUB.1.366

About the author

Joseph Hofstader is an Architect for Microsoft's Developer and Platform Evangelism organization. He is also an adjunct professor in the Department of Information Technology and Electronic Commerce at the University of Denver. Prior to joining Microsoft, he was a Distinguished Member of Technical Staff at Avaya. Joe has spent his career architecting, designing, and developing solutions in the telecommunications industry. He is happy to receive feedback at joe.architect@live.com.

This article was published in the Architecture Journal, a print and online publication produced by Microsoft. For more articles from this publication, please visit the Architecture Journal Web site.