Building a Predictive Analytics Platform to Measure Company Health
We used predictive analytics to create a platform to reliably predict burnout and turnover based on internal communication patterns.
Technology services company
Software Development, Product Design, Machine Learning
Elixir, AWS, React
Burnout and lack of engagement are two hugely expensive problems in today’s work environment. Most organizations experience double-digit turnover rates, but struggle to understand why key team members leave. Instead of needing to retroactively correct organizational problems, what if you could prevent them from becoming problems in the first place?
In most technology companies, team members have hundreds of interactions with each other throughout the day. As collaboration tools have evolved and more companies are supporting remote working environments, those interactions have moved from the physical water cooler to digital channels.
Slack, one of the largest platforms to help teams collaborate, has over 8 million active users every day. It was also the tool used by our client, who asked us to help them implement a predictive analytics solution to reduce churn within their company after several core team members left without warning.
As we considered the problem at hand and our own experience using Slack as a remote-first organization, we realized that technology can be a barrier to workplace empathy. When face-to-face interactions are replaced by Slack and email messages, small signs of unhappiness are easier for people to hide and harder for organizations to detect. Luckily, people leave subtle clues about their emotional state, even in short, digital communications.
There are robust, real-time measurement and reporting tools available for the health of systems infrastructure. Why couldn’t there be something similar for the health of your team and your overall organization?
This was an extraordinarily complicated problem to tackle, so our first order of business was to truly understand the root cause of the problem. We asked, “What causes burn out?” Christina Maslach, a social psychologist at the University of California (Berkley) has been researching occupational burnout for decades and is widely recognized as the predominant subject matter expert.
According to Maslach, people ultimately burn out because of a mismatch between the job and the individual. While many managers still think that burnout is an individual employee problem, that’s an incorrect assumption. Burnout is a response to chronic stressors in the workplace, which can affect many people. To prevent it, you have to fix the work environment.
This led us to our next challenge — how can you fix something if you don’t know when it’s broken? Survey tools are woefully inaccurate as employees often aren’t honest about real issues. So we started thinking about how to leverage the primary communication tool everyone is already using every day — Slack — as a data source.
We had a hypothesis that subtle variations in punctuation, message length, word choice, time of day, grammatical correctness, and other variables are markers for a person’s emotional state. Everybody’s markers are unique and virtually undetectable by other people. However, we believed machine learning services and statistics-based algorithms could find patterns and anomalies in these communications, and even predict the likelihood of a team member, a team, or even an entire company experiencing burnout.
To kick-off the development process, we designed some small-scale models to validate the general direction we were headed. This yielded promising results, and we moved on to developing a robust data pipeline for ingesting real-time events from Slack. The data pipeline extracted features from each message, including grammatical errors, sentiment, emoji usage, and readability (scored as a Flesch reading level) on a per-message basis.
As we worked to arrive at an MVP, we began developing more complex analysis tools using natural language processing and statistics to visualize the flow of information within the company, identify communication silos and bottlenecks, and measure the level of collaboration among team members and across teams.
While these measurements were invaluable, they didn’t accomplish the goal of predicting burnout before it happened. We stepped back and applied a common pattern found in infrastructure monitoring — anomaly detection. Just like everyone has a unique fingerprint, the way an individual communicates also has a unique pattern. By measuring when and how person diverges from their normal communication pattern, we were able to identify several different clusters of anomalies. Given these data points, correlated certain kinds of anomalies to negative events like voluntary turnover.
Our predictive analytics platform has been in use for a little over a year now, and the results are astounding — even to us. Utilizing machine learning services, we were able to identify individuals who are likely to turnover due to burnout in the next 90 days with a 79% accuracy. With some more data, we expect to achieve greater than 90% accuracy and a longer future-looking time window.
By providing real-time sentiment analysis, collaboration graphs, and topic clustering on a team and company level, leaders have benefited from understanding how their actions impact their direct reports and whether their team’s communication health is within normal ranges compared to the rest of the company.
After launch, a few things happened...
Our client’s forward-thinking leadership — armed with advanced analytics — is empowered with the right information to be able to take early corrective actions to fix workplace environmental issues. The result? A 950% reduction in voluntary turnover, which translates to approximately $10 million in savings annually.