Author: Frederick P. Brooks
The Mythical Man-Month: Essays on Software Engineering Frederick P. Brooks 1975
“The Mythical Man-Month” was written at a time when humanity did not even dream of a personal computer in every home. The book brought that possibility closer. iPhones and wireless communication are indistinguishable from the 1960s and 1970s, but the principles of managing complex projects that were formulated in those years by the young scientist Frederick Brooks, then working in the advanced field of cybernetics – creating OS / 360 at IBM – can be useful to this day. . We have prepared a summary of this wonderful book with comments from Vyacheslav Nikulin, head of Smart Reading development.
In the 1960s and 1970s, there were no diverse ready-made tools for remote communication, joint work on layouts and documentation, systems for maintaining and controlling tasks, and resources used. Although all these technologies are now actively used, the process of developing and designing software is only getting more complicated. Software is developed for many applied areas of life, different programs automatically interact with each other, errors in the assessment of labor costs for the development of such systems have the same roots as 40 years ago.
A program and a software product are not the same things. A software product is different from a program:
- the most generalized range and type of input data;
- thorough testing, which is a separate complex task;
- detailed documentation.
Creating a software product takes three times as long as a program. At the same time, software (SW) is one of the most complex creations of man, and this complexity is not easy to cope with. The widespread use of high-level languages 1 (researchers attribute a five-fold increase in software productivity to this factor) was a powerful driver for software productivity.
Unified programming environments also played their role. And yet, the software can only be simplified up to a certain limit, which is set by four conditions:
1) complexity (a huge number of software configurations makes it difficult to describe and test them);
2) subordination to form (the interfaces of the created software must obey the diversity of human institutions and systems for which they are created);
3) variability (software is functional, and functionality is what is most subject to change due to a rapidly changing reality);
4) invisibility (the reality of software is not built into space, it is difficult to imagine it in the sense in which, say, the Earth is reflected on geographical maps, it is difficult to visualize software without a fundamental simplification).
Objective difficulties, however, can be minimized if the creation of software is organized correctly. It’s not about how computers work, it’s about people.
How to organize a team
Revealing the man-hour
The main reason most software projects fail is lack of time.
- Programmers are ruined by optimism: since their product is not created from clay or paint, but from pure mental abstractions, they are often confident that there will be no great difficulties in the implementation of the project. Meanwhile, the ideas themselves are erroneous, and this leads to errors in programs.
- We confuse effort and progress. The progress of the work does not depend on man-hours, people and time are not interchangeable.
People and months are interchangeable only when there is no need to build communication between employees. Picking cotton in the fields? The man-hour idea works. Software development? The idea of man-hours fails because the different stages of this work are interdependent and programmers have to spend time interacting with each other. It always takes more time to set up communication than it does to reduce the time it takes to complete individual work steps. Therefore, the introduction of new forces into the project at the later stages of development only pushes back the deadline for the project (Brooks’ law).
Brooks developed a rule for scheduling software builds.
Here, the proportion of planning is increased and the time for debugging the code is significantly increased – for the reason that it is in this area that programmers do not expect great difficulties, but they happen everywhere.
20 years later, in the 1995 revision, Brooks pointed out a weakness in this scheme: if it is implemented linearly, the accumulation of errors is inevitable and with increased code debugging time. In the 1975 edition, Brooks insisted on the practice of a pilot system that programmers must create before developing the final model. The pilot will identify errors in the design, and then will be completely edited. For 20 years, the approach to creating programs has changed, and in 1995 Brooks recognizes: that the principle “Plan to throw it out!” does not work. He points to the advantages of an incremental model – a phased strategy when different parts of the system are developed at different times and at different rates, and if one part is ready, it is immediately integrated into the system. But his basic principle has remained unchanged: add labor to a project that is lagging behind schedule – finally slow down the process.
Conceptual integrity is a key condition of the project. Let us recall the medieval cathedrals of Europe, which were built over decades, but retained their stylistic unity. This is what software developers should strive for. An excellent example of conceptual integrity is the WIMP (windows, icons, menus, pointers) interface, now known to every user.
Conceptual integrity requires that a project come from a single developer (or several people acting in unison). This developer is called the lead architect for the project. Acting solely in the interests of the user, he decides what should be included in the software product and what is not (this vision is embodied in the project reference guide, which is described below).
Traditionally, the structure of the organization is represented as a tree, reflecting the general principle of hierarchical subordination. In fact, it is more like a network with many organizational groups, and committees.
Each subproject of such a network contains two leadership roles: the role of producer 2 (he assembles the team, distributes work, determines the schedule, seeks resources) and the role of technical director 3, or architect (he thinks through the design and is responsible for the concept). Sometimes, especially on smaller teams, the producer and director can be the same person. However, in a large project, each of the roles requires a full-time job, in which case it is better to separate the roles: the producer can be the head of the group, and the director is his right hand, and vice versa.
Architects and developers
To ensure the conceptual integrity of the system, it is important to separate the architecture from the implementation. The architecture of the project determines what is done, the implementation determines how it is done.
A simple example is a watch. Their architecture consists of a dial and hands. Those who have mastered this architecture will never forget how to tell time. The implementation is responsible for the operation of the mechanisms in the body. The ratio of cost to product performance depends on the implementers.The mythical man-month
Architects are the aristocracy of projects, but that doesn’t mean they limit the creativity of developers. Those can get started as soon as they have a more or less clear understanding of the reference manual and cost and performance criteria. Developers can design data flows, packaging concepts, and so on. Often the processes of architecture, software, and hardware implementation run in parallel, as do hardware and software design.
In this regard, one of the most promising technological solutions is the development of approaches and tools for rapid system prototyping as part of the specification development process. Prototypes simulate the intended software product, allowing you to review the given conceptual structure, check it for convenience and consistency.The mythical man-month
The project architect is an aristocrat, but not a dictator. He must be ready not only to propose a way to implement his ideas but also to accept any other equally good way.
It would be tempting to involve a small team of top-notch geniuses in the creation of the project, rather than leading a hundred or two people. Seductive and unrealistic. But Brooks suggests a way out: each segment of a major job should be performed by a team organized like a surgical team. Instead of everyone wielding a scalpel, you need to put the key tool in the hands of one person, and the rest will assist him. 4 The roles are distributed as follows:
- “surgeon” – the main programmer, who sets the specifications for the functionality and performance of the project, the key developer of the design, who directly works with the most critical parts; “surgeons” – programmers from the best, such work 5-10 times faster than others;
- “assisting surgeon” – less experienced, but able to perform any part of the work, representing the team in project discussions with other teams, knowing all the code thoroughly (he can even write the code, but he is not fully responsible for it);
- administrator, dealing with staff, workspace, etc.;
- editor – processes and corrects all documentation prepared by the “surgeon”;
- two secretaries 5 for administrator and editor;
- clerk – responsible for registering all the technical data of the team in the library of the software product;
- toolmaker, who creates specialized utilities, cataloged procedures, macro libraries, responsible for access to basic interactive services;
- tester responsible for the bank of test cases, testing work in the process and at its final stage, planning the testing sequence;
- programming language consultant. 6
These 10 people perform strictly specialized roles, which facilitates communication between them and ultimately contributes to the conceptual integrity of the work. Even if 200 people are involved in the project, management needs to coordinate only 20 “surgeons”.
Ideally, managers and technicians should be as interchangeable as their talents and knowledge allow, and the boss should be able to easily move employees between managerial and technical roles (without loss of prestige and salary).The mythical man-month
Work on the project should not repeat the fate of the Tower of Babel. As you know, the reason for its collapse is the lack of communication 7 and, as a result, the lack of coordination. Likewise, any large project fails because the left-hand does not know about the actions of the right, and employees increasingly disagree with each other in assumptions. Teams should communicate with each other in as many ways as possible: informally, in meetings, and through the project workbook.
Reference guide. At the heart of any project is a reference manual that consistently and clearly describes the product specification—every detail of what the user is dealing with. The preparation of such a guide is being phased in as feedback from users and developers reveals design flaws. This requires both a formal design definition (for accuracy) and an informal one written in plain English (Russian, etc.) (for clarity). One of these descriptions must be standard, the other – derived (for example, the Algol 68 programming language had a formal definition as a standard and a textual definition as a derivative).
Current updates are fixed at a weekly conference of all architects and official representatives of hardware and software developers. This meeting is led by the chief architect. Proposals for discussion are circulated among the participants before the meeting, at which the assembled invent solutions. Each proposal, before turning into a solution, is considered by the developers, and its pros and cons are clearly outlined. If consensus cannot be reached, the decision is made by the chief architect. Decisions are submitted to one or more architects for refinement into well-defined proposals for changing the reference manual.
Over time, some decisions do not justify themselves, and with others, unforeseen difficulties arise. Accumulated petty appeals are proposed to be considered at the annual “trial hearings” 8, which usually last two weeks (they can also be held twice a year). They are held just before the decisive dates of freezing the reference manual, they are attended not only by architects and developers but also by programming and marketing managers. The project manager chairs. During the preparation of the System/360 project, there were usually 200 items on the agenda. All parties were heard and decisions were made. The next morning, each participant found a revised manual at their workplace, in which the decisions made the day before was made.
In the course of work, developers have many questions about the architecture, regardless of how precise the specification is. At such a moment, each employee should be able (and willing) to immediately contact the responsible architect and not put the problem on the back burner. The architect must keep a special journal in which each question and answer is recorded. Each week, magazines from several architects are pooled and distributed to users and developers.
Another guarantee of the reliable operation of the project is a product testing team and a mandatory external audit. It checks machines and programs according to specifications and identifies any possible inconsistencies. Every team needs such a “devil’s advocate”.
Project workbook. The project workbook is not a document, but a structure that is superimposed on all project documentation: target criteria, external specifications, technical standards, and administrative memorandums.
- This structure is planned carefully and well in advance.
- Timely updating of the workbook is of fundamental importance. nine
- It is important for an employee who gets acquainted with the contents of a workbook that the changes made to the document since the last time he read it are clearly highlighted and their significance is noted.
- A quick summary of changes in the “last changes seen first” mode is important.
- Parts of the workbook should be organized so that each employee can only see their part of the work.
Brooks strongly convinces managers not to shy away from working with documents, no matter how boring, and sometimes redundant it may seem. In fact, a small number of documents concentrate a critical part of the work. The regular fixation on ideas generates a clear logic of decisions, making them obvious to everyone, and eliminating possible contradictions. The necessary set of formal documents for each department should be completed at the very beginning of the work – this information will guide the process, crystallize the discussion of the project, provide a mechanism for monitoring the state of affairs and preventing failures, and serve as both a checklist and a database.
A software project requires target criteria, user reference manual, internal documentation, schedule, budget, organizational chart, space allocation. A computer development project may differ, but one way or another, all documentation boils down to basic questions: what? when? how many? where? and who?The mythical man-month
How is it that the project is delayed by as much as six months? It all started with the fact that once the project was only a day late and no one noticed it.
To avoid this problem, you should plan a schedule consisting of milestone dates – specific, easily measurable events. The clearer they are, the harder it will be to miss them.
The authorities controlling the work schedule will need two types of data:
1) information about deadlines that require direct intervention;
2) a general picture of affairs – for awareness.
It is important for the boss not to confuse one type of information with another and not to take active action on problems that could well be solved by his managers in the current order. At the same time, managers should not be intimidated by the constant mode of readiness for emergency work: the boss should often emphasize that all meetings, reports, and conferences are primarily aimed at studying current affairs, and not looking for implicit shortcomings, which will be followed by inevitable punishment.
At the same time, the work schedule should be consulted regularly, referring to the control points. Work with the schedule should be entrusted to a small (up to three people) group of employees, who will become the authorized representative of the management in the team. Its main authority is to interrogate managers about the compliance of work with accepted control points and their possible transfer.
How to optimize a product
Where are the limits of optimization?
How long does it take to program the system? The above was a general ratio (1/3 time for process planning, 1/6 time for writing code, 1/2 for testing individual components and the entire system), but you cannot accurately schedule a project by simply multiplying the time spent writing code, to coefficients for other parts of the work. It is worth considering, for example, that when using the right high-level programming language, development productivity can be increased five times.
Assessing the productivity of programmers, the researchers were faced with the fact that programming took only half of the working week – the rest of the time was eaten up by meetings, work with documentation, small corporate affairs, and illnesses. In large projects, there is another danger: each employee optimizes his work to achieve current goals, as a result, programmers turn into disunited competitors who do not work for one goal but earn personal points. The integrity of the project is lost, and the total effect on the client falls. Maintaining a shared vision and user focus should be a top priority for program managers.
Here it is especially important to remember the conceptual integrity of the project. Everyone strives to improve, without thinking about where it will lead. As a result, elegant and simple solutions are replaced by clearly redundant ones, and bloated software with a lot of additional functions that consume a disproportionate amount of system resources (the so-called second system effect 10 ) appears. Brooks admits that the legendary OS/360 is also an example of the second system effect (and Windows NT already in the 1990s).
How to avoid the effect of the second system?
- Pre-assign strict a priori limits on various system functions in bytes and microseconds.
- Be willing to reject even a great idea if it doesn’t fit into the established system design. In the first place – simplicity: it is better to embody only part of the system’s capabilities than to overcomplicate it, leaving some of the capabilities unused by anyone. That is, the designed architecture should be expandable in logical directions, but should not include additional functionality that may not be required.
Another subtle point is software maintenance. It consists of correcting bugs that were identified by users and taking into account their wishes that arose as a result of working with the still unfinished system. The client’s desire is the law, and yet it must be taken into account up to a certain point, otherwise, the work on the system simply will not be completed.
In addition, fixing a bug has a significant (up to 50%!) chance of introducing a new bug. The system designers must negotiate a point after which the specifications are frozen and all subsequent customer requests are deferred until the next version is worked on.
Defining functions as carefully as possible and eliminating functional redundancies will reduce the number of system errors that will show up when testing the system.
Optimization of working tools
Working on a large project should be as optimized as possible: all the necessary tools should always be at hand.
- Every project needs a database of 11 full of standard components: good routines or macros for queuing, searching, hashing, and sorting.
- Each development team should have one programmer responsible for writing 12 utilities for their team. There should also be a group that creates utilities for everyone working on a particular system.
- To reduce the cost of development, project managers can buy a piece of software from other developers without wasting time and money on “reinventing the wheel”.
There were other optimization principles half a century ago when Brooks and his colleagues developed the IBM System/360.
- An important role in optimizing the IBM System/360 was played by the logical emulator, IBM’s method of running programs designed for older devices on new models (today, for example, emulators allow you to run Windows on a Mac computer and vice versa). Instead of developing completely new applications for new computers, this compatibility gave developers more flexibility in solving problems.
- An important innovation was the library of master programs, divided into several systems with different access rules. All tested (or still being tested) code was stored in this library. Each working group had its own area where employees kept copies of their programs and test cases. Here they had complete freedom of action. When a well-oiled component appeared, ready to be integrated into a larger part, it was handed over to the manager of the larger department, who placed this component in the system integration sub-library, and the author could no longer change it. As the system grew, it became ready for wider use and moved into a sub-library of the current version. Formal separation and control were two key principles that successfully coordinated all work. They are relevant today.
- The most important tools for system programming turned out to be tools that were hardly used in the development of OS / 360. These are high-level languages that allow you to describe data much shorter than with machine code, and interactive programming (by allowing you to make changes to the code while the program is already running, it at least doubles the productivity of system programming).
How to avoid defects
Long before the code is released, the specification must be submitted to an external testing team for scrutiny for clarity and completeness.
Brooks considers Wirth’s top-down design based on step-by-step refinement of planned work to be the best formalization of programming.: a rough outline is created that reflects the principal result, then each item of the plan is broken down into specific steps and each refinement in the definition of the problem is introduced into the solution algorithm and the way the data is presented. The process identifies data modules that can be further worked on, regardless of what contributes to the flexibility of the whole process. Having noticed a mistake, developers can always go back – top-down design eliminates the need to hastily “sew-on patches” in problem areas of the project. The top-down design reflects the general principles of structured programming. It greatly simplifies the process of creating software, reducing the work on the program to elementary structures (linear, cyclic, alternative). Programs become less complex and more visual.
This, however, does not allow neglecting the thoroughness of debugging the system. It, unlike component debugging, always takes longer than expected.
- Debugging a system should only be started after people have made sure that all parts of the system work (as opposed to the “screw it all together and see if it works” approach).
- When debugging a system, you should add one component at a time 13 – this time-consuming method is often neglected by programmers, but the more consistently the process is, the fewer errors will escape the attention of specialists.
- Debugging “scaffolding” (testing code) can take up to 50% of the debugged product, but it’s worth it.
- All current changes are carefully documented. It is important to have complete test cases when testing systems after each new part is added.
The documentation for the user is as important as the code for the machine. It reflects the whole vision of the product, which was defined by the chief architect. To keep the documentation up to date, it is important that it is built into the source program and not stored as a separate document.
The highest level structural graph can be safely stored as a separate document (because it is not subject to frequent changes), but can also be embedded in the source program as a comment.The mythical man-month
Often, user documentation suffers from excessive detail and a lack of a general overview. When compiling it, one should go from the general to the specific: from the purpose of the program (what is its main function) and the environment of its application (on what machines, with what hardware configurations and operating systems it will work) to the implemented functions (what exactly the program can do) and operating instructions (including behavior in emergency situations).
The program must come with several test cases: some for valid inputs, some for boundary inputs, and some for obviously invalid inputs. In documentation intended for program modifiers, it is important to describe not only how the program works, but also why it works the way it does.
This documentation contains five items:
1. Block diagram or graph of the structure of the subroutine.
2. Full descriptions of the algorithms used or links to such descriptions.
3. Explanation of the general scheme of all used files.
4. An overview of the organization of the data flow – the sequence in which the data is loaded, and a description of what happens at each stage.
5. Discuss modifications to the original design and what modifications may be desirable.
Few programs need more than a one-page flowchart. Flowcharts of algorithms are too overrated and irrelevant due to the development of high-level languages.The mythical man-month
You can reduce the effort of documenting your program by:
- more using the required parts of the program for documentation (symbolic names and declaration statements);
- making the most of space and format to show subordination and nesting (this makes documentation easier to read).
Guidance through the decades
This book began with the thesis that the main thing in programming is not machines, but people 14. Brooks writes that programming is creativity, like creating paintings or symphonies. There are geniuses in this area, and the closest attention should be paid to their search :
- systematically identifying the best developers as early as possible (remembering that the best are not always experienced);
- appointing mentors who will be responsible for the systematic training of promising employees (it includes carefully selected internships, courses, etc.);
- providing opportunities for maximum interaction between prospective developers.
Fortunately, the environment itself, which has fundamentally changed over the past decades, contributes to the emergence of geniuses. Developers in the 1970s expected software tools from scientists that would:
- so cheap that almost anyone could access them;
- according to our creative needs.
And what? This is what we see today when the smartphone in your pocket is more powerful than all the computing centers that sent the first people to the moon. Programming is still one of the most difficult intellectual pursuits, but human genius does not standstill. Everyone can be convinced of this simply by paying attention to the gadget from which this text is being read.
Top 10 Thoughts
1. The time spent working on a project is not inversely proportional to the number of programmers: time and effort are spent on coordinating work and interacting with employees.
2. If the project does not meet the deadline, then the addition of labor will delay it even more (Brooks’ law).
3. The development team should have one best programmer in charge of major decisions, and several others to help him. This is how surgical teams operate.
4. Conceptual integrity is the goal of any project. To do this, the architecture of the project must be separated from its implementation. The final word belongs to the chief architect.
5. Simplicity is as important as conceptual integrity. Trying to add every conceivable feature to the system should be avoided.
6. Each project manager should create a set of formal documents that describe the goals, deadlines, cost and executors of the project. These documents become part of the workbook, the structure that defines the entire project workflow. Because of this, each development team has a clear set of requirements for their part of the system.
7. A project late by months starts with a delay of one day. To prevent this, a schedule consisting of dates of control points – specific, easily measurable events will help. Usually, such a schedule is drawn up in detail for the next 3-5 weeks.
8. Development teams should interact with each other in all available ways. You should never neglect the opportunity to ask again.
9. Work with tools should be optimized as much as possible. Buy some software from other developers. Assign a programmer responsible for writing utilities for your group. Designate a group that creates utilities for everyone working on this system.
10. The wishes of the user should be taken into account only up to a certain point, otherwise the work on the system will never be completed.
1. The producer role is now called team lead.
2. The role of the technical director on a small project is performed by the tech lead.
3. The approach is somewhat outdated. The most modern approach is a group of so-called T-specialists, the most interchangeable employees, in which everyone knows some part of the functionality very well, but can do everything else.
4. Modern office calendars, mail, and instant messengers usually cover all needs.
5. Today this problem is usually solved with the help of Google search.
6. Team communication is still a big problem today, especially in large organizations, where the same task is often solved synchronously in different projects. And, as a rule, each division finds its own solution, which causes further difficulties with unification.
7. Now development has accelerated significantly compared to the realities described in the book.
8. Today, according to the agile methodology, it is generally accepted that good communication is more important than documentation (although documentation is important).
9. The second system effect – today this problem has even worsened, as development is accelerating all the time and experienced developers are trying to provide too many moments for the future, unconsciously ignoring the TOR.
10. Now all this is stored in common repositories on the Web.
11. For large projects, this is true, but for most of these tasks, there are now ready-made services, the rent of which costs $5–10 per month.
12. Today there are very functional tools for stepping through the program. This makes it easier to find errors.
13. Debugging “scaffolding” is now called autotests.
14. In the time that has passed since the book was written, the era of good, standard specialists has already begun. Geniuses are very difficult to manage, they often suddenly leave the project. Although, with regard to the search and development of specialists, Brooks’ advice is still working quite well.
For Next Post