Moneyball for Litigation? A Conversation with Premonition’s Toby Unwin

July 31, 2015

Moneyball for Litigation? A Conversation with Premonition’s Toby Unwin

Jul 30, 2015 at 5:31 PM

Even if you’re not a baseball fan, there is a good chance you’ve heard of Moneyball. The term originally referred to the book written by Michael Lewis about the Oakland A’s, but it’s taken on its own meaning, especially after Brad Pitt played the part of Billy Beane. The simple takeaway from Moneyball is that (in my best business school voice) by focusing on certain metrics and not others, you can expose inefficiencies and get ahead of the market before the market corrects itself.

Is there a Moneyball for litigation? Several companies think so. Last month, I had an opportunity to catch up with Owen Byrd from Lex Machina, a well established venture backed company in Silicon Valley that provides its customers with critical information about what happens in the courtroom. They have even referred to themselves as “Moneyball law.” But they’re not the only ones trying to provide lawyers with the metrics that matter. Case in point: Premonition. Premonition is building an artificial intelligence, predictive analytics and data mining to try to figure out which lawyers win the most before which judge. Seems like valuable information, so I wanted to know more. Toby Unwin is the Chief Innovation Officer for Premonition and he joins me here on Above the Law to discuss.

Zach Abramowitz

Toby, thanks for chatting with me. Tell us more about premonition.ai — give me your elevator pitch.

Toby Unwin

Premonition is an Artificial Intelligence system that mines Big Data to find out which Attorneys usually win before which Judges. It is a very, very unfair advantage in Litigation.

Zach Abramowitz

Before I ask you how the system works, tell me how you came up with the idea. You”re not a lawyer, why were you thinking about the issue of litigation in the first place?

Toby Unwin

America has 5% of the World”s population and 95% of the World”s litigation (because we”re practically the only democracy without a “loser pays” rule). Since moving here full time in 2000 (immigration is the sincerest form of flattery…), I”ve had all kinds of reasons to need to use lawyers – immigration, real estate, general business, a messy divorce, etc.

Probably my “favorite” frivolous lawsuit was getting sued for not having an elevator in a 1 story shopping center that I owned.

So, to answer your question, I”ve thought about litigation a lot.

During my divorce and the associated litigation, I came across 2 problems:

1. I couldn”t find ordinary civil cases unless I knew exactly where they”d been filed(i.e. there is no proper system that links court data).

2. I felt that some attorneys I”d hired in the past had been truly awful and knew instinctively that there must be some really good ones out there. But I had no way of knowing who they were…

Zach Abramowitz

Do you remember what your legal fees were for the elevator suit?

Toby Unwin

Yes, $10,000 to have it dismissed.
I didn”t get a penny back…

They call them American Disabilities Act “Drive by” lawsuits. In this case the guy hadn”t even bothered driving by…

Zach Abramowitz

So how does Premonition solve these problems?

Toby Unwin

Big data and transparency.

Big Data

We”ve assembled the World”s largest database of lawsuits, bigger, we believe, than every major legal database combined. Because all the cases sit in just one, big, database. You can search for a case, person, company, lawyer, etc, across all courts and even internationally. We”re starting to get quite a bit of Government interest on this, as it”s an ability they lack. It”s amazing that, with the $Bns spent on homeland security the Government still lacks the ability to search across its own courts.

Transparency

We”ve found a few attorneys(thankfully not too many) that quite simply take advantage of their clients. Take a look at this graph:

It shows divorce case durations and identifies one lawyer in particular who clearly runs the clock on all his litigation. Bar Associations will never discipline him, because they don”t have the data and know what he”s doing. If people could look up his times and compare them to the better attorneys they would simply stop using him. The market would fix itself through transparency.

Performance

We have found no correlation between price and performance.

There are many high priced litigators who are quite simply terrible, along with some cheap lawyers who are actually phenomenal.

These are the barristers in the UK court of appeals:

The most popular counsel is a bit above average. There are many, far better performers who are practically ignored. More worryingly, when the popular advocate is busy, the big law firms hire counsel that usually perform worse than Pro Se litigants(42%). The 2nd, 3rd and 5th most popular barristers have win rates in the 20″s.

The market here is really, really broken. The best performers are actually less likely to be re-hired. The same holds true for General Counsel selecting law firms – they actually do 18% worse than random…

Many lawyers are uncomfortable with their performance being measured, but there is a lot of upside here. Top litigators struggle to break the $1,100 an hour ceiling, yet top Queens Counsel in the UK can charge nearly 8 times that. The very best litigators deserve to be paid more and win rates are the way to show that. Many lawyers struggle with marketing themselves and billing at a rate commensurate with their ability. With performance transparency, the right cases will find them and the best advocates can command premium rates(which clients will happily pay because they know they”re getting proven value).

Zach Abramowitz

Okay, but doesn”t this kind of information beg the question, how valuable are these stats? That”s not to say that people won”t pay for this information, clearly you”ve shown they will.

My question is how much does this kind of information really teach us? Are wins the best way to evaluate lawyers? Some lawyers take on easier cases or have clients with greater means than the people who have sued them.

Can”t wins be misleading?

Toby Unwin

Absolutely, a win out of context is hugely misleading.

The “best” lawyer here in Orlando is a chap called James D. Barron III. He has thousands of cases a year with an 85% win rate. Our AI system is besotted with him, like a teenager with a crush.

None of our clients hire him though.

Why?

Because he”s an eviction attorney, which goes a long way towards explaining the high volume and win rate. We simply don”t get asked for that much.

Context is everything.

Venue

Outside of the Federal system we rarely compare attorneys from different courts. Aside from differing State licensing, knowledge of the local court, procedures and Judges is vital. Geography matters.

Case type

It would be total nonsense to compare evictions or traffic tickets with appellate work, for example. So we don”t. We have 108 different case types in the system and can even drill down further by adding criteria like winning party=”insurance company”.

Judges

We”ve found that the Attorney/Judge pairing is worth 30.7% of the verdict on average. This may not seem fair, but it is fact. A former Judge that helped us refine the system said “Facts and law have lost too many times.” People matter. Judges are human.
They have a lot of opportunity to gauge the veracity of counsel appearing before them.
How picky are they with the cases they take on?
Are they well prepared?
Do they present coherent arguments?
Some attorneys are simply more persuasive to some Judges.
Big data picks this up.

Let”s look at some criticism for a moment:

Easy vs hard cases

It is certainly true that a lawyer can get a string of hard cases that may affect their win rates temporarily. However, over the 41,000 cases filed every day in America, the lumpiness of easy/tough sets of facts even out.

Our system is designed to spot the outliers.

People like Alvin Benton, who had 32 straight wins before Judge Schreiber here locally. If he lost a few cases due to tough facts, he”d still be number 1 by a huge margin before her.

I”m not a big believer that there are many attorneys who only take “easy” cases. Attorneys who were that selective would soon be broke. Given that, until the last year or so, absolutely no one in America kept track of win rates. There was little disincentive for litigators when they lost, if they are paid by the hour, because no one really knew.

Lawyers who only took easy cases would also quickly de-select themselves from our short lists. The United Kingdom has a “cab rank rule” that forbids barristers from cherry picking cases. We have found fairly similar results for litigators across all our data sets. Cherry picking case difficulty is not much of a factor.

It”s the start of a selection process, not the end

Having 20 lawyers to choose from that usually get the result you”re wanting for that case type and judge is a very, very good place to start your selection. It is hard to go far wrong when you start from such a well defined pool.

But that”s when human decision making and experience take over…

I used the system last year to pick a litigator for myself.
I chose not to go with the #1 pick even though his numbers were excellent.
Why?
Quite frankly, he was an asshole.
I didn”t want to spend 2-3 years of my life dealing with him.

Barrister”s clerks often state that the number 1 reason why a barrister is not re-hired is failing to get along with the instructing solicitor. This leads to the “hire your mates” system that”s been in place for centuries and the negative win rate/re-hire rate correlation in the UK courts.

I”ve had 2 solicitors in the last year tell me
“winning isn”t important,”
to which I respond,
“clients like it.”

Picking lawyers by performance is not perfect.

But what method is better?

Zach Abramowitz

Just looking at your replies (long even by ReplyAll standards), your passion for Premonition is unmistakable. You are screaming premonition from the rooftops.

Are you this passionate about everything or is this Premonition specific?

Toby Unwin

We”re all very passionate about changing law here. My CEO and CTO have both had more than their fair share of frivolous litigation. We believe that transparency is the first step to changing anything. As Peter Diamandis said “Most evil is done in the dark.”

Other than that, I think I get obsessive and focussed on things. When I was younger it was rowing. That was all I thought about and I wound up winning the National Championships.

Later it was flying, which led to a World Airspeed record.

When I was 23 I started reading obsessively, whenever I had spare time. People come into my library, see thousands of books and say “It”s great that you”ve had all this time to read.”
“Not really”, I say, “This is a monument to frustration. It represents all the time I”ve had to spend on hold, waiting in line or for people to do stuff.”

Basically, when I do something I do it with maximum effort and take a multi-disciplinary approach that enables me to find solutions that elude experts who know only one field.

Every day of my life “experts” tell me something is “impossible”.
I”ve learned to ignore them

Zach Abramowitz

Ok, I”m glad I asked.

I mentioned Lex Machina in my intro as another company doing Moneyball for lawyers.

How is Premonition different and is the market big enough for both?

Toby Unwin

We like Lex Machina a lot.

Any people who bring transparency to law deserve huge kudos.

Lex Machina is a great product that in many ways is a lot more detailed than ours and their interface design is much prettier than ours.

The main difference between the 2 companies is scale:

They have a team of 15 people analyzing 6,000 patent cases a year.

We have an Artificial Intelligence system analyzing 41,000 cases a day.

In many ways we”ve been really lucky to have had Lex Machina do such a great job of proving the market for litigation analytics. They”re certainly the leader in the patent space right now with a level of analysis we currently don”t have.

Premonition is more focussed on general litigation and what we refer to as “Perception-Reality Arbitrage.”

Zach Abramowitz

Toby, this has been fun — thanks for dropping by. If you want to learn more, head over to Premonition.ai.

Toby Unwin

Thank you so much, Zach.

Great format.

Really enjoyed it.

Zach Abramowitz is a former Biglaw associate and currently CEO and co-founder of ReplyAll. You can follow Zach on Twitter (@zachabramowitz) or reach him by email at [email protected].