CLSP Student Seminar – Andrew Blair-Stanek “Shelter Check and GPT-4’s Bad Legal Performance”

Abstract
Our goal is to use AI to automatically find tax minimization strategies, an approach which we call “Shelter Check.” It would come in two variants. Existing-Authority Shelter Check would aim to find whether existing tax law authorities can be combined to create tax minimization strategies, so the IRS or Congress can shut them down. New-Authority Shelter Check would automate checking whether a new tax law authority – like proposed legislation or a draft court decision – would combine with existing authorities to create a new tax minimization strategy. We had initially had high hopes for GPT-* large language models for implementing Shelter Check, but our tests have showed that they do very poorly at basic legal reasoning and handling legal text. So we are now creating a benchmark and training data for LLM’s handling legal text, hoping to spur improvements.

Date: 
Friday, October 6, 2023 - 12:00 to 13:15
Location: 

Hackerman Hall B17 @ 3400 N. Charles Street, Baltimore, MD 21218