Step up your game: A research on two/multi-step summarisation of long, regulatory documents

Abstract

Automatic Text Summarization (ATS) is the process of automatically summarizing a text. Advancements in neural models in Natural Language Processing (NLP) have significantly enhanced summarization capabilities, making it a crucial tool for processing extensive regulatory documents. Long regulatory texts are challenging to summarize due to their length and complexity. To address this, a two- and multi-step extractive-abstractive architecture is proposed to handle lengthy regulatory documents more effectively. This research shows that the effectiveness of a two-step architecture for summarizing long regulatory texts varies significantly depending on the model used. Specifically, the two-step architecture improves the performance of decoder-only models. For abstractive encoder-decoder models with short context lengths, the effectiveness of an extractive step varies, whereas for long context encoder-decoder models, the extractive step worsens their performance. This research also highlights the challenges of evaluating generated texts, as evidenced by the differing results from human and automated evaluations. Most notably, human evaluations favoured legal language models, while automated metrics preferred general language models. The results underscore the importance of selecting the appropriate summarization strategy based on model architecture and context length. Broadly, this research contributes to the development of more efficient and accurate tools for summarizing complex regulatory documents, enhancing accessibility, and aiding in compliance and decision-making processes in dynamic industries such as the green molecule sector.