Sunday, April 12, 2015
Back in February, I wrote a post on The The Early Days of Legal Analytics. It discussed some of the innovations at Lex Machina, a legal start-up that uses Big Data to value contested patents and develop a litigation strategy designed to maximize value / minimize risk. I recently came across another company, Premonition, that claims to use artificial intelligence to select lawyers with the best odds of achieving a favorable result. See Premonition Infographic at bottom of this post.
I spend a lot of time on the road talking to law firm lawyers and legal innovators, including legal start-ups. Many large firm lawyers tend to dismiss new innovations without stopping to listen to, much less gather, relevant facts. Likewise, there is a lot of puffery among legal start-ups as they try to land their first few customers. Thus, I tend to apply a windage factor to accurately interpret what I am being told.
The image below contains Premonition”s simple, one-sentence pitch. I don”t know about the product, the concept is pretty clear.
The core benefit Premonition appears to offer is a list of lawyers with winning track records in front of specific judges. We don”t need artificial intelligence (AI) to make that calculation. A win-rate is a simple descriptive statistic, even if it has been filtered for a variety of matching criteria.
That said, AI could come in handy in building the requisite data sets. As explained on Premonition”s website, courts don”t construct their case management systems so they can be vacuumed out by data mining companies. Indeed, local court officials would likely to be hostile to such requests because any resulting statistical model is unlikely to make them look good. Indeed, the purpose of the model is, at least in part, to identify and exploit imperfections in the judicial process.
Because the courts have no incentive to make life easy for Big Data vendors, Premonition”s Chief Innovation Officer, Toby Unwin, claims to have tackled the data assembly problem by building a technology that scrapes and buckets the necessary data from the jumbled chaos of online local courthouse. Such a task, in theory, can be performed by fairly standard machine learning, which qualifies as AI, at least in some circles.
Assuming Premonition has built a machine that can calculate win-rate of lawyers, is that information valuable to clients trying to maximize the likelihood of a favorable result?
I don”t know, but its plausible enough to test with data. Some data skeptics will argue that win-rates, whether high or low, are just artifacts of any normally distributed outcome. The reasoning runs, “Two, three, and four sigma events occur in the ordinary course of life, but regression to the mean is pulling them back to the center. Thus, they are poor predictors of the future.” This reasoning is why many people buy indexed funds rather than shares in actively managed mutual funds. Cutting the other way, the hedge fund industry is premised on the belief that some money managers are a lot better than others. Five-year return rates are aggregated and published in the industry trade press. Some of the returns may be due to random luck, but some could be attributed to superior skill. It is absurdly unlikely, for example, that Warren Buffet”s success in buying and selling stocks is just a 60-year winning streak.
In the case of win-rates in court, I can think of at least two plausible non-random factors that could affect outcomes:
I can”t vouch for Premontion”s technology beyond two statements: (1) it sounds plausible, and (2) it is a waste of time to debate its usefulness because it”s an empirical question that the market will answer in the relatively near term. Below is one of Premonition”s infographics.