MULTISEISMO: A Multimodal Seismic Dataset and Model for Cross-Modal Seismic Understanding
Abstract
The application of generalist multimodal models (GMMs) to specialized scientific domains remains limited due to the scarcity of comprehensive domain-specific datasets that integrate multiple data modalities beyond text and images. In seismology, understanding earthquake phenomena requires the synthesis of timeseries waveform data, geographical imagery, and contextual metadata, a multimodal integration absent in existing seismic datasets. We present MultiSeismo, a large scale structured multimodal seismic dataset, comprising over 16K seismic events spanning 13 years (2010 to 2023) across diverse geographical regions. Each event data integrates waveform recordings from global station networks, intensity maps, population exposure visualizations, and a comprehensive textual description within a standardized JSON format. We additionally develop MISCE, a multimodal instruction set on top of raw data to enable supervised training and evaluation of GMMs on seismic reasoning tasks ranging from basic information retrieval to complex cross modal analysis. We leverage MISCE to finetune an existing multimodal model (Unified IO 2) enhanced with a specialized timeseries encoder, which yields SeisModal, the first domain specific multimodal model for comprehensive seismic analysis. Evaluation of state of the art multimodal models on MultiSeismo reveals significant challenges, particularly with time-series data processing for general purpose models, while demonstrating SeisModal's superior…
Read the paper
Related studies
- Current state and future directions for deep learning based automatic seismic fault interpretation: A systematic review
- Physics of Partially Saturated Porous Media: Residual Saturation and Seismic-Wave Propagation
- Modern Imaging Using Seismic Reflection Data
- Effects of marine seismic surveys on free-ranging fauna: a systematic literature review
- A review of the recent seismic activity in the southern part of egypt (upper egypt)
- A review of research on mechanism of seismic attenuation and energy compensation
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.