Process debt and Workflow

What’s going on?

How often have you completed a piece of work but were not sure of the next step, or who needs to work on it next? Or you are yourself not clear what needs to be done next? How much time gets wasted because of this lack of clarity? And what can we do to reduce this waste? This waste is known as “process debt”.

If a person has one job, always does the same thing, and always passes on the work to the same next actor (person or department), such as working on a production line, then organizing their work is fairly straightforward, at least in the sense of always knowing what needs to be done now and next. But for most of us, this is not the case. We do many different things, working with many different people. Even if you are doing the same type of work, you may be receiving it from many people and passing on the results to many different people from different groups. It is far too easy for things to fall through the cracks – you don‘t notice an email asking you to do some work, you have to hunt to find out who to send work on to, you can’t track down all the material you need to do your job and don’t know who to ask, you forget a step,…

Given that most of our working lives are following processes, it is natural to ask if there is anything we can do to make that more efficient or effective. A quote from Abraham Lincoln is appropriate here: “If I had eight hours to chop down a tree, I’d spend the first six of them sharpening my axe.”. In other words, before just jumping in and starting a task, think about the task and how it can be made more effective. We can apply the same thinking at a higher level to processes and projects, not just individual tasks.

In practice in a process, what typically happens is that some person, as an actor in the process, carries out some action to create some output. Assuming the next action is carried out by some other person (though the same applies if the same actor continues), then the original actor has to:

• know what the next action is
• know what they need to supply for the next action to be carried out
• know who has to carry it out
• inform the next actor that there is work waiting for them and where they can find the necessary material

Ideally our systems make all these things easily visible.

So what’s the problem?

Our working lives would be much simpler if we didn’t have to spend so much time looking for the answers to the questions implied at the end of the previous section. In many organizations, far too much time is spent working around problems in the way we work within our processes where there is lack of clarity about the process and specific tasks within it. The two main issues are that we don’t have a clear definition of who needs to do what and when, and much information is not easily accessible.

In my consultancy career much of my work involved helping client organizations to define their processes and implement them in software tools. It was a very common experience to be in a workshop to understand a specific process and there would be disagreement among the client staff about how the process should work, even if it was notionally documented and they were all supposed to be following it. This highlights the need to define processes clearly and, perhaps just as important, make it easy for staff to follow them.

Is there a better way?

So what is the answer? Of course, there is no single solution, and people find lots of ways around it. For example, I have personally set up Outlook so that I can turn a message into a task in a couple of clicks, and have rules so that some messages automatically get turned into tasks. I have been in several meetings where people are doing amazing things with Excel spreadsheets to track work. But these are somewhat lacking. In the first case, the gains in productivity are limited to just one person – nobody else has access to my emails and l don‘t have access to theirs, although other people could, of course, use the same approach. In the second case, the spreadsheets are manually maintained and record status and, to a limited extent, next actions. In at least one case there were three different spreadsheets in use to manage essentially the same information for slightly different purposes so the potential for conflict and confusion was massive.

Some companies are fortunate in having knowledge bases of processes and materials. But these are often limited in a number of ways:

• They document processes but do not provide any support to ensure that they are followed.
• It is often difficult to find the appropriate material within them. Searching generally throws up a mass of material that matches whatever search term you have chosen but this often finds irrelevant material where the term occurs incidentally.
• There is generally no real structure and any tagging and categorization mechanisms are not used consistently.
• They are not always maintained consistently and hence become out of date.

What we really need is a tool to manage the work of a team, where people automatically get notified of their tasks, where status reporting and passing on work to the next step is as simple as checking a box and where the information needed to do a job is maintained consistently. Fortunately, there is a multitude of such workflow management tools available, ranging from the fairly simple to full-featured.

Process definition

The start point of all such tools is process definition. A process can essentially be defined by its inputs, its outputs, and the process itself - how it transforms the inputs to the outputs. This definition is itself recursive, as each process can be decomposed into other processes until you reach something atomic such as “add one to the count of widgets produced”. We also need to define who does the work. Each of these processes is the responsibility of some actor, which may be a person or, more likely, a job role or a team. In general, the overall process involves several actors.

Unfortunately, many of our processes are not well documented, if they are documented at all, so this first step can be quite difficult. You can almost guarantee that if more than one person is following a process they will actually be doing it in different ways. I have experienced this in customer meetings when I ask the customer engineers to explain their current process. Despite all believing they follow the same process, there are almost always people saying “that’s not how we do it”.

When you start to define a process, you will probably be surprised at how many distinct steps are involved. Some people’s reaction to this is that they cannot possibly define the process as there is no way that they can do that many things. The fact, of course, is that they do those things whether they are defined or not, and not defining them is the reason that essential steps get missed. So being honest is important. Some workflow tools only use a single level of definition, others allow you to define sub-processes. Defining sub-processes is useful for two reasons. First, it allows you to create a single definition for a common sub-process that is used in many places. Second, it allows you to defer the definition of detail, adding it when it becomes useful or necessary.

Having defined a process, you might decide to optimize it, using techniques such as Value-Stream Mapping. That is outside the scope of this post, but well worth looking into.

Using the tools

Workflow tools work in many different ways, the features generally provided include:

• Individuals can quickly identify their own work
• Individuals can quickly identify work products for which they are responsible
• Automatic notification of individuals when there is work waiting for them
• Ability to see upcoming work and its current state, including who is currently working on it

Additional features often found include:

• Ability to find the status of a work product
• Ability to identify unallocated tasks that are otherwise ready to be performed
• Dashboards providing management information on status, burndown, progress, etc.
• Prioritization of activities, generically and for specific instances
• Different views including, for example, Gantt charts, Kanban
• Time tracking. This is particularly useful when companies are still using timesheets, and hence the time spent, as a metric for work done, inaccurate, misleading and damaging though this is (perhaps a topic for another post)

Benefits

Most of the benefits have been discussed above, here is a quick summary:

• Individuals and teams know what they are supposed to be working on
• Individuals and teams know what work is upcoming, so can plan ahead
• Knowledge is available to all, not hidden in spreadsheets on personal computers
• Individuals and teams are notified when work is ready
• Managers and others can see the progress of tasks and the overall status of work

In short, work becomes much more efficient and effective.

A commenter on an earlier version of this post made the point that standardisation and digital processes can support lean, but it can also make processes rigid, less agile and can stifle creativity and continuous improvement. The need is to find the middle ground somewhere, where standardisation of key processes simplifies and reduces administrative burden, while leaving some things up to qualified, competent and experienced employees giving them the freedom to change things quickly should it be needed. You need a stable foundation to start from and the flexibility to tailor processes to a team’s or client’s needs.

Choosing the right tool

It would be ideal if there was a simple way to decide which tool to use. A useful reference is https://www,makeuseof.com/online-task-management-tips-choosing-right-app/. In the end, it comes down to which tool provides the best mix of features that you need for the price you can afford. In a small organization, almost any tool will provide benefits, at the risk that it may not allow for growth. A larger organization should probably consider running pilot projects with different tools.

Origin of tools

Workflow tools come from five main disciplines, project management, software development, business process management, document control, and task management. These disciplines overlap to a greater or lesser extent, so you find tools generally support more than one.

Project management tools, such as MS Project, arose from the need for large projects to be able to manage and report on their work and progress. They generally take a very hierarchical approach and so are suitable for work that is mostly planned in advance, such as most projects, but less so for more responsive tasks such as customer service, sales, etc.

Many tools originate in the need for software development teams to control change, which essentially restricts who can do what work on what content. This has led to the need to define the flow of work and then control it. There are some very simple, but still effective, tools in this area but they are not generally useful for the larger task of managing business processes.

More sophisticated tools, such as IBM’s Engineering Workflow Manager (formerly Rational Team Concert), have grown to include project planning, Kanban, workflow, issue management, and other facilities. These provide all that is needed in a workflow tool but can be difficult to configure and use and are best suited to organizations that can clearly define their processes and have the need to coordinate many people on many different activities.

Tools originating in business process management generally start from the foundation of drawing a business process as a flow diagram. Tools that start from this basis generally develop by allowing the definition of who can do each activity and what work products are involved. These are often the tools that are best suited to an organization that is starting to understand the need for some sophistication in how it manages its processes.

Tools originating in document control are similar to those originating in business process management, with more of an original emphasis on how documents can be changed and formally issued. These often have good electronic signature capabilities.

The tools coming from task management often start with the need to manage the tasks that an individual has to carry out. They gradually add sophistication, such as task dependencies and allocating work to different people. These are often suitable for organizations getting started in process management and can often grow to be sufficient even up to fairly complex situations.

Terms

Those of you in a technical field, especially software-related, will have heard the term “technical debt” being bandied about. According to Wikipedia: “Technical debt is a concept in software development that reflects the implied cost of additional rework caused by choosing an easy solution now instead of using a better approach that would take longer.” (https://en.wikipedia.org/wiki/Technical_debt).

Reflecting this Wikipedia definition I have used the term “Process Debt”, which I define as: “Process debt reflects the implied cost of additional work caused by choosing an easy solution to a process problem now instead of using a better approach that would take longer”. To understand this, we need to take a step back. Let‘s start with the Wikipedia definition of a process: “A process is a series or set of activities that interact to produce a result: it may occur once-only or be recurrent or periodic.“ (https://en.wikipedia.org/wiki/Process). So pretty much everything we do is part of a process. What l am considering here are especially those processes that we repeat and form part of our business as usual, though much the same thinking can be applied to one-off processes.

Written with StackEdit.