Artificial Intelligence (AI) for Integrated Safety Assessment Model (ISAM)
Abstract
This project aims to advance risk-informed decision-making at the FAA by developing machine learning techniques for extracting safety-relevant information from incidents and accident reports and enhancing large language model (LLM) capabilities for navigating aviation regulatory text. The work addresses two primary challenges: the labor-intensive process of manually extracting safety data from Aviation Safety Reporting System (ASRS) and National Transportation Safety Board (NTSB) narratives for use in the Integrated Safety Assessment Model (ISAM), and the inability of general-purpose language models to accurately interpret the hierarchical structure and cross-references within Title 14 of the Code of Federal Regulations (CFR). To improve safety data extraction, the project developed two datasets. The first is a question answering (QA) dataset designed to enable precise supervision for fine-tuning LLMs to extract events, contributing factors, and operational conditions from narrative text. The second consists of synthetic safety reports generated using a fine-tuned LLM informed by a domain-specific knowledge graph (KG) built from FAA aircraft data to eliminate aircraft specification errors and improve multilabel classification performance. To enhance regulatory reasoning, the project built a regulation-aware knowledge graph and developed a retrieval-augmented generation (RAG) system with improved retrieval precision for semi-structured regulatory content. An evaluation…
Read the paper
Related studies
- Knowledge Management Perspective of Generative Artificial Intelligence
- Navigating Generative Artificial Intelligence Promises and Perils for Knowledge and Creative Work
- Characterizing the evolving research on enterprise content management
- Large language models and multimodal foundation models for precision oncology
- From Generative AI to Extended Reality: Multidisciplinary Perspectives on the Challenges, Opportunities, and Future of Educational Computing
- Generative Artificial Intelligence: A New Engine for Advancing Environmental Science and Engineering
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.