JURIX 2022: 35TH INTERNATIONAL CONFERENCE ON LEGAL KNOWLEDGE AND INFORMATION SYSTEMS
PROGRAM FOR THURSDAY, DECEMBER 15TH
Days:
previous day
next day
all days

View: session overviewtalk overview

09:00-10:30 Session 4: Argumentation and Legal Reasoning
Chair:
09:00
Implementing a Theory of a Legal Domain

ABSTRACT. We describe a system for constructing, evaluating and visualising arguments based on a theory of a legal domain, developed using the Angelic methodology and the Carneades argumentation system. The visualisation can be used to ex- plain particular cases and to refine and maintain the theory. A full implementation of the well known US Trade Secrets Domain is used to illustrate the process.

09:20
A Hybrid Model of Argument Concerning Preferences Between Statutory Interpretation Canons

ABSTRACT. This paper extends the existing account of statutory interpretation based on argument schemes theory. It points out that the preference relations among statutory canons are not always determined by some predefined rules, but in certain systems of law or legal domains, it is necessary to argue these preference relations on the basis of case law. A set of factors favouring linguistic arguments and teleological arguments is presented, and a case-based argument scheme for the assignment of preference relations is reconstructed.

09:40
Modelling and Explaining Legal Case-based Reasoners through Classifiers

ABSTRACT. This paper brings together two lines of research: factor-based models of case-based reasoning (CBR) and the logical specification of classifiers. Logical approaches to classifiers capture the connection between features and outcomes in classifier systems. Factor-based reasoning is a popular approach to reasoning by precedent in AI & law. Horty (2011) has developed the factor-based models of precedent into a theory of precedential constraint. In this paper we combine the modal logic approach (binary-input classifier logic, BCL) to classifiers and their explanations given by Liu & Lorini (2021) with Horty’s account of factor-based CBR, since both a classifier and CBR map sets of features to decisions or classifications. We reformulate case bases of Horty in the language of BCL, and give several representation results. Furthermore, we show how notions of CBR, e.g. reason, preference between reasons, can be analyzed by notions of classifier explanation.

10:00
An Argumentation and Ontology based Legal Support System for AI Vehicle Design

ABSTRACT. As AI products continue to evolve, more and more legal problems are emerging for the engineers that design them. For example, if the aim is to build an AV that adheres to current laws, should we give it the ability to ignore a red traffic light in an emergency, or is this merely an excuse we permit humans to male? The paper argues that some of the changes brought by AVs are best understood as necessitating a revision of law’s ontology. As the current law is often ambiguous, inconsistent or undefined as to how new situations may be brought about by AI, engineers need an aid system that can provide feedback on whether a product design complies with different possible laws and what options do they have. Thus this research aims at exploring a new representation of legal ontology by importing argumentation theory and finally constructing a trustworthy legal decision system for AV designers.

10:15
Unpacking arguments

ABSTRACT. The contemporary study of argumentation in AI and Law, and in AI generally, often uses the graph of attacks between arguments as a starting point, abstracting from the structure of arguments (following Dung's influential work on abstract argumentation frameworks). Such an approach works well if the structured arguments can be generated from a given knowledge base, after which the attack relations are determined. Actual debate however shows a dynamic interaction between argument structure and attack. For instance, in a discussion, unstated premises or intermediate premises can be made explicit, or critical questions can be asked that point to issues not yet considered. Inspired by Loui and Norman's work on the rationale of arguments, we study the relation between argument structure and attack in terms of the unpacking of arguments. The paper provides an analysis of two kinds of rationales discussed by Loui and Norman. We show how work on argument structure, argumentation schemes and argumentation semantics can provide further insight into the unpacking of arguments, as a model of attack identification. Example dialogues inspired by Dutch tort law are used for illustration.

10:30-11:00Coffee Break
11:00-12:15 Session 5: Legal Knowledge Extraction I
11:00
A Multi-step Approach in Translating Natural Language into Logical Formula

ABSTRACT. Translating often has the meaning of converting from one human language to another. However, in a broader sense, it means transforming a message from one form of communication to another form. Logic is an important form of communication and the ability to translate natural language into logic is important in many different fields, in which logical reasoning and logical arguments are used. In the legal field, for example, judges must often reason from facts and arguments presented in natural language to logical conclusions. In this paper, toward the goal of support for this kind of reasoning with machines, we propose a method for translating natural language into logical representations using a combination of deep learning methods. Our approach contributes methodologies and insights to the development of computational methods for converting natural language into logical representations.

11:20
Investigating Strategies for Clause Recommendation

ABSTRACT. Clause recommendation is the problem of recommending a clause to a legal contract, given the context of the contract in question and the clause type to which the clause should belong. With not much prior work being done towards the generation of legal contracts, this problem was proposed as a first step towards the bigger problem of contract generation. As an open-ended text generation problem, the distinguishing characteristics of this problem lie in the nature of legal language as a sublanguage and the considerable similarity of textual content within the clauses of a specific type. This similarity aspect in legal clauses drives us to investigate the importance of similar contracts' representation for recommending clauses. In our work, we experiment with generating clauses for 15 commonly occurring clause types in contracts expanding upon the previous work on this problem and analyzing clause recommendations in varying settings using information derived from similar contracts. We would open-source our code to ease future work on this problem.

11:40
Toward an Intelligent Tutoring System for Argument Mining in Legal Texts

ABSTRACT. We propose an adaptive environment (CABINET) to support caselaw analysis (identifying key argument elements) based on a novel cognitive computing framework that carefully matches various machine learning (ML) capabilities to the proficiency of a user. CABINET supports law students in their learning as well as professionals in their work. The results of our experiments focused on the feasibility of the proposed framework are promising. We show that the system is capable of identifying a potential error in the analysis with very low false positives rate (2.0-3.5%), as well as of predicting the key argument element type (e.g., an issue or a holding) with a reasonably high F1-score (0.74).

12:00
Recognising legal characteristics of the judgments of the European Court of Justice: Difficult but not impossible

ABSTRACT. Machine learning has improved significantly during the past decades. Computers perform remarkably in formerly difficult tasks.This article reports the preliminary results on the prediction of two characteristics of judgments of the European Court of Justice, which require the knowledge of concepts and doctrines of European Union law and judicial decision-making: The legal importance (doctrinal outcome) and leeway to the national courts and legislators (deference).The analysis relies on 1704 manually labelled judgments and trains a set of classifiers based on word embedding, LSTM, and convolutional neural networks. While all classifiers exceed simple baselines, the overall performance is weak. This suggests first, that the models learn meaningful representations of the judgments. Second, machine learning encounters significant challenges in the legal domain. These arise doe to the small training data, significant class imbalance, and the characteristics of the variables requiring external knowledge. The article also outlines directions for future research.

12:15-13:00 Session 6: Keynote Speech
12:15
Is there a future for AI in the Judiciary and under what conditions?
13:00-14:30Lunch Break
14:30-16:00 Session 7: Legal Knowledge Classification
14:30
Linking Appellate Judgments to Tribunal Judgments - Benchmarking different ML techniques

ABSTRACT. The typical judicial pathway is made of a judgment by a tribunal followed by a decision of an appellate court. However, the link between both documents is sometimes difficult to establish because of missing, incorrect or badly formatted references, pseudonymization, or poor drafting specific to each jurisdiction. This paper first shows that it is possible to link court decisions related to the same case although they are from different jurisdictions using manual rules. The use of deep learning afterwards significantly reduces the error rate in this task. The experiments are conducted between the Commercial Court of Paris and Appellate Courts.

14:50
Transfer Learning for Deontic Rule Classification: the Case Study of the GDPR

ABSTRACT. This work focuses on the automatic classification of deontic sentences. It presents a novel Machine Learning approach which combines the power of Transfer Learning with the information provided by two famous LegalXML formats. In particular, different BERT-like neural architectures have been fine-tuned on the downstream task of classifying rules from the European General Data Protection Regulation (GDPR) encoded in Akoma Ntoso and LegalRuleML. This work shows that fine-tuned language models can leverage the information provided in LegalXML documents to achieve automatic classification of deontic sentences and rules.

15:05
Functional Classification of Statements of Chinese Judgment Documents of Civil Cases

ABSTRACT. The format of judgment documents in Taiwan follows some basic principles, but the principles often provide only guidelines for including the meta data about the judgments, e.g., the name of the court, the judgment time, the case number, and the names of the plaintiffs, the defendants, and the judge, etc. The statements in judgment documents do not always include hints about their legal functions. As a consequence, when using such documents to train a software for assisting decisions via machine learning mechanisms, we often face the problems of explainability (or justifiability) for the recommendations that the resulting software would offer.

Enabling inference systems to identify the legal functions of statements in judgment documents can enhance the quality of the training data, and thereby improving the justifiability of the algorithmic recommendations of the systems that aim to offer legal assistance to human professionals. There are on-going research activities based on this line of needs. Researchers attempt to do named-entity recognition (NER), and extend to build knowledge graphs (KGs) for legal judgments based on the NER results, for instance.

Our current work aims at algorithmically annotating the functions of statements in judgment documents, and complements the previous work that focuses on the lexical or syntactic annotations for the judgment documents, e.g., NER and KGs. For civil cases, we wish to identify statements that are for the applicants, the statements that are for the opposite parties, the statements that reflect the viewpoints of the court, and statements about the judgments.

Our focusing on civil cases can be considered as a merit because the conflicts in civil cases are relatively unclear and typically harder to be identified. In contrast, specific keywords or their variants must be included in the judgment documents for criminal cases explicitly. Perhaps for this reason, many of the current judgment prediction systems are designed for criminal cases.

We obtain the judgment documents that were publicized by the court systems of Taiwan, whose open data site now hosts more than 17 million judgment documents. In this manuscript, we shall report our experience in handling and annotating the documents of the cases that are about the problems of family support. More specifically, the applicants are usually the elderly in a family, and the opposite parties are those who are supposedly obliged to support the elderly.

We approach our goal from a few different perspectives. Thousands of downloaded documents were annotated by people with legal expertise in the fist place. We train and test the classifiers with these annotated data, which is the normal procedure for a machine-learning based approach. The current results are not extremely perfect, but are encouraging. We are on our way to annotate the statements by more fine-grained functions next.

We also tried to annotate the documents with some heuristic principles, and used these documents for training. These heuristic principles might be observed by reading a large amount of judgment documents by ordinary people, and these ordinary people might believe that they can annotate the data at a professional standard. We expected that the classifiers that were trained with heuristically annotated data would not perform as well as the classifiers that were trained with professionally annotated data. We conduct realistic comparisons among our classifiers, and will report the observed differences in the manuscript.

The current work will contribute to the legal reasoning community in some ways. The resulting classifiers can annotate the statements with their functions in the judgment documents, which complement the annotations at the word and phrase levels. We work on a relatively unexplored genre of cases, namely the civil cases. The annotations for the cases about the family support problems have achieved reasonable quality and are usable. We compare the effects of using professionally annotated data and of using heuristically annotated data to train the classifiers, and will report some interesting realistic observations, though this line of exploration is perhaps only theoretically interesting since we have already annotated the data professionally.

15:20
The Illinois Intentional Tort Qualitative Dataset

ABSTRACT. Research in AI & Law modeling precedential legal reasoning and judgment re-quire cases about which to reason. Several datasets of such cases exist, in a variety of representational formalisms and across many legal doctrines and jurisdictions. We introduce the Illinois Intentional Tort Qualitative Dataset, a collection of Illinois Common Law cases in Assault, Battery, Trespass, and Self-Defense, machine-translated into qualitative predicate representations. After dis-cussing existing case sets and gaps that exist therein, we describe the cases underlying our dataset, the natural language understanding system used to translate those cases into a machine-interpretable format, and validation measures that serve as performance baselines for future AI research using the dataset.

15:35
Multi-granularity Argument Mining in Legal Texts

ABSTRACT. In this paper, we explore legal argument mining using multiple levels of granularity. Argument mining has usually been conceptualized as a sentence classification problem. In this work, we conceptualize argument mining as a token-level (i.e., word-level) classification problem. We use a Longformer model to classify the tokens. Results show that token-level text classification identifies certain legal argument elements more accurately than sentence-level text classification. Token-level classification also provides greater flexibility to analyze legal texts and to gain more insights on what the model focuses on when processing a large amount of input data.

15:50
An Interactive Natural Language Interface for PROLEG

ABSTRACT. PROLEG is a famous computer program supporting attorneys in the legal inference process. However, the input of this system is expressed in a language called Prolog, which most lawyers are not familiar with. This technical barrier is a serious problem for using PROLEG in the real legal context. A natural language interface is one of the solutions to this problem. We have developed a prototype of such an interface. This paper describes the prototype and its current performance. The prototype translates input facts into Prolog expressions following PROLEG syntax. The system consists of three main modules, (1) natural language perceiver, (2) PROLEG reasoner, and (3) inference explainer. In addition, we analyze the performance of the prototype and identify existing issues and discuss possible solutions.

16:00-16:30Coffee Break
16:30-18:20 Session 8: Legal Information Management
16:30
The Effectiveness of Bidirectional Generative Patent Language Models

ABSTRACT. Generative patent language models can assist humans to write patent text more effectively. The question is how to measure effectiveness from a human-centric perspective and how to improve effectiveness. In this manuscript, a simplified design of the autocomplete function is proposed to increase effectiveness by more than 10%. With the new design, the effectiveness of autocomplete can reach more than 60%, which means that more than 60% of keystrokes can be saved by autocomplete. Since writing patent text does not necessarily start from the beginning to the end, a question is whether the generative model can assist a user no matter where to start writing. To answer the question, the generative models in this manuscript are pre-trained with training data in both directions. The generative models become bidirectional. Since text generation is bidirectional, the calculation of autocomplete effectiveness can be bidirectional and starts from anywhere in the text. After thorough experiments, a key finding is that the autocomplete effectiveness of a model for the same text remains similar no matter where the calculation starts. The finding indicates that such bidirectional models can assist a user at a similar level, no matter where the user starts to write.

16:45
Automating the response to GDPR's Right of Access

ABSTRACT. With the enforcement of the European Union’s General Data Protection Regulation, users of Web services -- the `data subjects’ --, which are powered by the intensive usage of personal data, have seen their rights be incremented, and the same can be said about the obligations imposed on the `data controllers’ responsible for these services. In particular, the `Right of Access', which gives users the option to obtain a copy of their personal data as well as relevant details such as the categories of personal data being processed or the purposes and duration of said processing, is putting increasing pressure on controllers as their execution often requires a manual response effort, and the wait time is negatively affecting the data subjects. In this context, the main goal of this work is the development of an API, which builds on the previously mentioned structured information, to assist controllers in the automation of replies to right of access requests. The implemented API method is then used in the implementation of a Solid application whose main goal is to assist users in exercising their right of access to data stored in Solid Pods.

17:00
Consumer Dispute Resolution System based on PROLEG

ABSTRACT. It is challenging for lay consumers to predict a legal conclusion by applying appropriate law in a consumer dispute. We developed a system that assists consumers to predict a possible legal conclusion. The system enables consumers to identify the type of consumer disputes by using a tree structure, and to apply appropriate legal rules implemented as PROLEG programs. We arranged the system to avoid possible inconsistency between the tree structure and the PROLEG program.

17:10
The LegAi Editor: A Tool for the Construction of Legal Knowledge Bases

ABSTRACT. A key challenge in legal knowledge representation is the construction of formal knowledge bases. Such knowledge bases then allow for various applications such as searching and reasoning. In this paper we describe a web application for the annotation of legal texts. The result of the annotation is a formal representation of the legal texts in a form which might be used for searching and reasoning. The novelty of the tool is threefold: it provides a wizard which guides the user in the annotation process; it uses a high-level annotation language which is close to the language of the original text; and, it allows validation of the formal structure by allowing a simple comparison of the original and the annotated versions of the legal text.

17:20
Measuring the Complexity of Dutch Legislation

ABSTRACT. For legislation to be effective, it should not be too complex; otherwise, it cannot be sufficiently understood by those who have to apply the law or comply with it. This paper adds to the research in AI & law on developing precise mathematical complexity measures for legislation and applying these measures by computational means. The framework of Katz & Bommarito (2014) is applied to measure the complexity of Dutch legislation. The aim is twofold: first, to investigate whether this framework is more widely applicable by applying it to a different jurisdiction and a corpus of larger size; and second, to identify possible improvements to the framework. It was found that Katz & Bommarito's framework can be applied to Dutch legislation. However, it is argued that complexity measures that strongly correlate with the structural size of legislation are less useful since they may be beyond the legislator's control. To this end, additional correlation results to those of Katz & Bommarito are reported and it is recommended that their knowledge-acquisition-cost approach to measuring the complexity of legislation is refined by taking the possibility of the legislator's control into account.

17:35
An End-to-End Pipeline from Law Text to Logical Formulas

ABSTRACT. This paper develops a pipeline for converting natural English law texts into logical formulas via a series of structural representations. The goal is to study how law-to-logic translation can be achieved with a sequence of well-defined steps. The texts are first parsed using a formal grammar derived from light-weight text annotations designed to minimise the need for manual grammar construction. An intermediate representation, called assembly logic, is then used for logical interpretation and supports translations to different back-end logics and visualisations. The approach, while rule-based and explainable, is also robust: it can deliver useful results from day one, but allows subsequent refinements and variations. While some work is needed to extend the method to new laws, the software presented here reduces the marginal effort necessary. As a case study, we demonstrate our approach on one part of Singapore’s Personal Data Protection Act. Our code is available open-source.