The emergence of big data has given rise to “dataism” — a mindset or philosophy that believes relying on data could reduce cognitive biases and illuminate patterns of behavior we haven’t yet noticed. But what are the consequences of dataism in the legal industry, and how do we determine the applicability of AI solutions to augment service delivery?
Introduction
Welcome back to the Intapp Legal AI Explained series. In our last chapter, we unraveled the hype surrounding AI, how it fits within the legal services business model, and how it reflects the leading edge of many law firms’ innovation agendas. In chapter two, we’ll dive deeper into the reality of AI to identify relevant and credible applications of this technology. In other words, we want to explore how AI can be a meaningful benefit to law firms, as opposed to an ancillary experiment merely used to market the firm as driving innovation.
Trending Towards Dataism in the Business World
In 2013, David Brooks of the New York Times coined the term “dataism” — a mindset or philosophy created by the emerging significance of big data. He argued that in a world of increasing complexity, relying on data could reduce cognitive biases and illuminate patterns of behavior we haven’t yet noticed.
Although this may have been mere opinion in 2013, dataism has been the bedrock of many of the world’s most successful businesses in the last decade. Consider any household brand today — Netflix, Amazon, Spotify, Facebook, Google — all these businesses use data as a competitive advantage, which has paid dividends.
But how are these approaches to data fundamentally different to those of yesteryear? The answer lies in the increasing ubiquity of data in today’s society enabling these large companies to apply unique solutions to business problems. Take content streaming platforms, for example — the personalization of follow-on recommendations using the history of other users with similar tastes to suggest what you may be most interested in next — keeps you engaged in content continuing your subscription. In short, the hallmark of success is the ability to examine and organize operational data that comes with greater velocity, in higher volumes, and with more variety than ever before.
Translating Dataism to the Legal Industry
What are the consequences of dataism within the legal industry? Although businesses with millions of customers can benefit from technology and data-driven platforms, how does that translate to law firms?
The skeptics among us might argue that dataism is much less effective in professional services, especially when you consider that much of the context and data in legal work isn’t inherently captured in a consistent and structured manner. In our knowledge and relationships-driven industry, determining which data points can be leveraged can be a daunting challenge. This is especially true when considering the operational data gold mine of law firms: drafted legal documents, copious communications, and nebulously defined case strategies — the culmination of years of expertise and knowledge.
Using the Intapp Framework for AI Enablement
How can law firms navigate these challenges? At Intapp, we deconstruct the problems within a legal AI applicability framework; AI is just another business tool. However, AI can be the preferred business tool when we’re able to confidently articulate the accuracy of predictions paired with the significance of the client pains we’re solving. These two aspects are typically covered in the questions described below:
| Accuracy of Predictions | Significance of the Pain Points |
| Can we train an AI model to meet a minimum threshold of competence? What’s the size of the historical learning data set available to us? | What’s the current technological approach to addressing this business pain? Does applying AI significantly augment the outcome? |
| How available and accurate is newly obtained data? Do we know — or do we have consistent availability of — the inputs that inform our predictions? | How many manual interventions currently exist within the business process? Will AI significantly reduce the time invested to deliver the desired outcome? |
| Have we determined causation, and is this corroborated by industry norms? Can a client organization compare an AI model’s predicted answer to their own calculated answer? | Is the business process something you’d personally do if you weren’t paid to do it? How illogical is the business process, and how much do people feel it’s a waste of their time? |
| Are time-dependent data shifts well understood? What magnitude of impact is expected from data drift (a shift in patterns from past data to new data)? How does that influence the aging of the AI model? | What’s the financial cost of doing things as-is? What’s the cost of fee-earner time, business services, and other financial factors? |
| What needs to be done to augment AI models beyond minimum thresholds of competence? What’s the cost of manual supervision measures on top of a basic AI model? | If your firm had to pick between accuracy of the process versus the cost to deliver the process, which would be prioritized? Is the business problem sufficiently complex and core to the way the firm works that human intervention is necessary? |
Our research and development in recent years has focused on this approach, which we explored in our article, What Can True AI Do for Timekeeping?
Putting AI to work in a way that drives value for a firm requires a combination of consistently available and structured data, a willingness to leverage data-driven insights, and a method for generating sound analysis. Anecdotally, many of us have witnessed this firsthand when clients (external or internal) say they don’t care how a black box works until they disagree with its outcome.
With this knowledge, the question then becomes: Which aspects of legal service delivery are best suited to an AI-led approach? We believe that firms can achieve AI-advancing business goals at the more commoditized end of legal services work or non-fee-earning activities with large pools of structured data. Consider examples like pricing matters or timekeeping; The role of AI in these areas does not erode the status quo but helps augment it through enhanced insights and a more seamless user experience.
AI Powered Pricing
Driving Innovation with Caution
How can you carefully discern the credibility and competence of purported AI solutions? It goes without saying that not all AI solutions are equal; there are technical measures you can apply. The measure of an AI model’s predictive accuracy is known as its F-score — shorthand for its level of competence.
When evaluating the competence of an AI model, one useful analogy is determining whether its predictions are akin to what a parrot, teenager, or first-year lawyer might conclude in a similar setting. Our experiences have shown that the AI ecosystem still requires augmentation with human intelligence. Any claim of an AI solution with a high F-score without ongoing human input should be met with skepticism. This accuracy-to-effort balance, which ultimately distills into an accuracy-to-cost balance, will help determine whether the AI model itself passes muster in a cost-benefit analysis for a given business use case.
Taming the Beast
AI continues to offer significant benefits to many industries, but it’s essential for innovators and leaders to understand how to tame the beast. This statement holds particularly true in the legal industry.
Building upon this article, our next chapter will dive deeper into questions you should ask of AI solution providers in order to determine good versus poor implementations of AI models. We’ll also explore ways in which firms are overcoming the challenges of enabling AI, such as the foundations of clean and structured data.
To learn more about how Intapp Strategic Consulting works with clients to put legal AI to work across the client matter lifecycle, please get in touch.