The Great Resignation: 10 Lessons for Data Teams
How to improve your technology and team architecture to protect your enterprise from the next resignation boom, as told by someone who resigned.
- By Stan Pugsley
- November 23, 2021
For a variety of reasons, 2021 has been a year of high staff turnover, earning the name "The Great Resignation." Whatever the economic reasons might be, I would like to focus on the key lessons for data teams as we look towards the future. These lessons are drawn from dozens of interviews with partner companies and graduate Information Systems students over the last few months.
Lower Your Technical Risk
The first set of five lessons relate to protecting your organization from the effects of high turnover. The common goal for these lessons is to create an architecture that supports a much smaller "minimum viable team" that could still meet your service-level agreements. If your department was, instead, an airplane normally staffed with a flight crew of 10 people, what changes would allow you to keep the plane flying with just five (or even two) people?
Lesson #1: If you have done something three times, automate it
Lack of automation is a key reason personnel costs escalate and a key source of discontent from analysts and engineers. It is also a key risk factor because high staff turnover can make manual processes unsustainable.
Nobody wants to spend their career copying and pasting data into reports. Nor do they want to create the same charts for PowerPoint slides every month. Those types of tasks drain productivity, energy, and enthusiasm from your staff. A good rule of thumb is to watch for any processes that are done more than three times in a row in the same way -- then automate those processes.
One caution: automation projects are often not the first priority for business stakeholders because the manual effort is often invisible to them. IT staff often need to champion these projects through the process. As an added bonus: staff analysts and developers often find these projects are a source of interesting work and a chance to innovate.
Lesson #2: Leverage vendor connectors
If you are having trouble filling positions, you need to focus your people on essential tasks that are unique to your business. Don't waste your limited resources building something that vendors have already created.
When you do custom development to build out data pipelines, those integrations will require maintenance for years to come. Outsourcing those integrations with APIs will free up your people to focus on more important projects.
Lesson #3: Document and fix tech debt
Whenever a staff member leaves your organization, they walk away with your institutional knowledge. They know how to fix errors that occur, and even more important, they have ideas about how to prevent errors in the future. However, a 30-minute exit interview isn't even going to start pulling that knowledge into the light where it can be usable for future staff.
As discussed in my previous article on tech debt, your goal should be to continually bring all of that knowledge of incomplete and weak code into your user story backlog for future work. Don't wait until it is too late to ask where loose ends from past projects exist.
Lesson #4: Ruthlessly simplify your tech stack
This is not the time to add random technologies to your environment just because a developer was bored and wanted to try out something new. (Was there ever a good time for that?) Such pet projects are difficult to support and train new staff to handle. In fact, when you are working with a lean team size, pet projects may be impossible to support. For the good of your department -- and possibly its survival -- simplify!
Lesson #5: Migrate to the cloud ASAP
Moving data and systems to the cloud lowers your staffing levels by shifting the staffing burden for admin tasks to your provider and off your department. Even if you pay more in overall licensing and storage costs, you be able to run a leaner, more nimble department for years to come. When adding up the costs for cloud versus on premises, keep in mind the increasingly high costs of turnover, recruiting, and training staff to support on-premises systems.
Raise Up Your People
Let's shift our focus to ideas to retain and build your current staff.
Lesson #6: Harness those idle minutes and brain cycles
From discussions with developers and my students, I'm convinced that more of them leave a company because they are bored than because they are stressed out. They want to be busy doing interesting things; they get frustrated sitting idle during long meetings or doing repetitive manual data tasks.
How do we more fully utilize developers' time without adding stress and engage their minds in solving company problems rather than surfing sports scores? Ideas include:
- Create proof-of-concept working groups; always have a project in the works for each group
- Determine what excites a developer's interest and steer more of their work in that direction
- Always have a backlog of ready-to-code tasks that can be picked up when a sprint finishes early
- Create competitions to allow high-potential, younger developers to shine
- Sponsor learning groups with defined deliverables; buy them books or rewards
- Ask developers to help blog or create marketing content
Whatever your method, the goal is to create a never-ending collection of interesting challenges for your staff and make sure those challenges align with business goals.
Lesson #7: Commute for a reason
Developers know when their time is being wasted and when it is being well used. When you ask them to travel to the office, help them see the value. Make on-site workdays more effective by doing things on those days that can only be done in person. Collect everyone for lunch-and-learn sessions. Conduct brainstorming sessions. Create comfortable, natural, team-building experiences. Even the most introverted staff members won't mind traveling if they understand the reason and view it as worth their time. They just don't want to do the same things in the office that they would do more easily and comfortably at home.
Lesson #8: Don't guess if employees are satisfied -- ask them
Tools such as Qualtrics are making employee surveys easy and painless. Employees are becoming more accustomed to providing feedback and reviews in every part of their life. Zoom meetings are taking away the awkwardness of sitting face to face in a potential power imbalance with a manager. There are no excuses for not taking the pulse of employees' satisfaction and needs on a regular basis (ideally every month). The more often you ask for feedback, the more quickly you can catch issues before they boil over into resignations. With the tight labor market, it is more important than ever to listen and respond quickly to employees.
Lesson #9: Prioritize training and upskilling
Does every employee in your organization have a career development plan and a list of next steps? Do employees feel empowered to set aside time to get the training they want and need? Improving employee skills doesn't have to involve formal training classes. It may involve special assignments on new projects or mentoring relationships with senior staff. Training programs need to have time allocated and need to have accountability for completion to be effective. If you haven't prioritized training, employees may take advantage of the hot labor market to steer their career development into a job at a new company.
Lesson #10: Compensation must match value
There is a compensation gap that seems to repeat at every company: you hire an employee and develop their skills and experience that increase their value by 20 percent, then only increase their pay by 5 percent. HR departments use logical explanations to explain the gap, from pay equity to financial constraints, but the gap persists and often causes turnover within 1-2 years. For the highest-performing employees, the pay-value gap is the biggest and is especially endemic to the big data and data science fields, where rare, highly sought-after skills can be learned in a matter of months.
The free market labor pool in the United States doesn't care about pay bands, pay equity or profitability constraints. Your high-performing employee especially doesn't care. Closing the gap is never easy because it disrupts the orderly systems created by HR departments, but it is important to recognize the gap and constantly work to shrink it.
Conclusion
The great resignation of 2021 is not going to be a one-time event. Trends and shifts that began this year will continue into the coming decade as skills become more specialized, tools such as LinkedIn become better at helping recruiters target your best employees, and transparency tools such as Glass Door make it easier for employees to compare companies and evaluate new jobs. By lowering your technical risk you can protect yourself from some of the costs and disruptions of turnover. By raising your employees' skills and satisfaction, you can create a place where people want to stay and invest their time and energy.