The promise of automation is doing more with less, freeing people from repetitive tasks allowing focus on more interesting activities. This claim makes for a great tagline but can fall short in implementation.
Automation doesn’t have to include complicated machine learning or deep learning. It could be a simple script. Automation is far from a panacea and can create hard to rectify issues. In this post, I’ll provide some perspective and a quick zero friction gut check on the impacts of automation.
Contents
Real Issues Stemming from Automation
Issues from automation aren’t theoretical. They’ve happened to me, and they’ve happened to you. Before we dive in, let me describe two very recent events.
Outlier events
A mobile provider charged me for a phone I didn’t have. Since I had autopay, it immediately hit my credit card. I called them countless times.
At first, everyone I talked to treated me with skepticism. After all, how could my situation happen? I was transferred around to multiple departments while I could almost hear people’s eyes rolling. After a while, everyone I talked to in every department saw the issue and was very apologetic.
Once identified as an issue, nobody from the various departments (including billing) could fix it. This was over the course of weeks. I was in a weird limbo, an outlier. I’d been a customer for 20 years, and it literally took them taking money from me to leave.
Changes occurring without human intervention
In another instance, I made an order from an online retailer. My package went out for delivery in my neighborhood and then back to the regional distribution center every day for two weeks before I received a notification my package was being returned to the sender for having an “Insufficient Address.”
Four days later, I received the package with the notification that it was delivered to the original sender (I was the recipient.) When I looked at the label, my address was clearly printed and visible, but above my name there was a single question mark. I used the same ordering system that everyone else used, but somehow, without my interaction, something got messed up.
Negative Impacts of Automation
Automation is inevitable, and I’m not suggesting we don’t automate, but we need to understand the negative impacts and implement mitigations. Two things about the future of automation are certain: 1) error rates will increase, and 2) it will become harder to correct errors. Automation removes people from a process. But if you remove the people entirely, unanticipated small issues can become big fast.
Two things about the future are certain: error rates will increase, and it will become harder to correct errors.
An algorithm or process that is 99% accurate seems great. But think about this: if you have a million instances, then 1% is 10,000. That is not an insignificant amount. Imagine each instance is a person, so potentially 10,000 people are affected by an issue.
Companies are often quick to implement automation, but they don’t consider the adverse effects or how they’d handle these effects. The negative impact to humans is often thought of as job loss, and any technical issues that crop up are considered addressable with a future tweak.
A human system without automation may very well have a higher error rate, but there is a human in the loop. People are more likely to believe that a human made a mistake than an automated system. Being this is the case, human systems can have more robust resolution processes.
People are more likely to believe that a human made a mistake than an automated system.
Our confidence in technology will lead to a lack of trust in other humans. The system can’t be wrong, so the human must be wrong. This isn’t a perspective we should encourage.
What Is an Automation Impact Audit?
We have to realize that technology is not infallible, and mistakes will happen. Given this fallibility, you need to have an appropriate mechanism for people to correct errors and inaccuracies.
To start with, you can perform a simple process that I call an automation impact audit. An automation impact audit will help you understand the process being automated and help identify potential issues. This process looks at six fundamental elements.
1. Process
Evaluate the process you are automating.This can range from automating small tasks of a larger human process to the complete removal of humans from the loop. Although any inaccuracies can be bad, in a process that has humans completely removed, they can be harder to detect and resolve.
Question(s) to Ask:
- What components make up the system today, and how will it look after implementing automation?
- What percentage of the process will be automated?
2. Inputs
Higher complexity systems can lead to a higher number of unexpected issues. You should also evaluate the data you are using and implement data quality standards. Poor quality data can lead to poor decisions, and sometimes you don’t realize it until after the system has launched.
Question(s) to Ask:
- How complex is the process we are trying to automate?
3. Impact
Understanding the impact of automation gone wrong is one of the most important aspects of the audit. The higher the impact of inaccuracy, the more controls you need to put in place. Something that may appear as a minor issue or cost may be a point of frustration for a customer, causing them to discontinue using your service.
Question(s) to Ask:
- What is the impact of a wrong or inaccurate decision?
- Will people or businesses incur harm, or would it be relatively inconsequential?
4. Detection
Think of ways or areas that you can measure and determine if a problem arises. You should also implement this detection periodically so that you can look for issues over time. This can be an indicator that something is changing in your data or process that needs to be adjusted. Customers may not report issues and just choose to discontinue using your business.
Question(s) to Ask:
- Do we have a way of detecting issues that could result from our implemented automation?
5. Rollback
If you have reassigned people who were previously performing tasks that automation handles now, you may not be able to go back to your previous state. This is why automation implemented in phases is more robust to the overall process.
Question(s) to Ask:
- How do we get out of automation?
- Do we have the ability to go back to a previous version that worked better or had fewer issues?
6. Resolution
Just having a resolution process isn’t enough. That process needs to be clearly communicated so that people know what to do when an issue arises.
Question(s) to Ask:
- Is there a way for impacted parties to correct issues and inaccuracies?
Learn more about Kudelski Security’s AI Security & Strategy advisory services
Automation for Automation’s Sake Isn’t a Winning Strategy
Human-level performance doesn’t equal human-level resolution. When implementing automation, robust resolution processes need to be implemented, allowing for proper resolution when issues present themselves. Preparation and care are required before implementing automation, including considerations of the impacts as well as clearly communicated resolution processes. Automation thrown at a problem for the sake of automation isn’t a winning strategy.
