ROUGE is constructed to improve data on disaster impacts in the Global South, where conventional databases often underreport and emphasize monetary losses. The source data are IFRC operational reports from DREF and Emergency Appeal Fund, issued by National Societies for crises ranging from small-scale to large-scale disasters. Reports from 2016 onward are used for structural consistency: 2,610 reports over 1,221 unique appeals. One report per appeal is retained, usually the latest, and reports are filtered by natural hazard type, leaving 776 unique appeals after removing 471 reports and excluding five corrupted files. Hazards include drought, flood, glacial lake outburst, cyclone, hurricane, typhoon, storm, tornado, heatwave, coldwave, mass movement, earthquake, volcano, tidal wave, and wildfire, with epidemics and conflicts considered only as compounding factors. The typology maps to EM-DAT and UNDRR Sendai categories, with 20 impact subtypes under Human, Infrastructure and Service Access, and Economy and Culture. Qualitative mentions are retained when no numerical value is reported.
The pipeline has four main steps. First, reports are scraped as PDFs and converted to text using pdfplumber. Second, preprocessing uses spaCy for language detection and cleaning; non-English reports, URLs, ligatures, and irrelevant sections are removed, and sections containing hazard and impact information are selected by headers and keywords. Third, LLM extraction uses meta-llama/Llama-4-Scout-17B-16E-Instruct through GROQ. Text is split into sentence chunks of about 1,000 characters for impact identification, while full reports provide context for locations, hazards, and dates. Prompts extract impact subtype, value, unit, location, hazard, and start/end dates. Pydantic validates JSON fields and allowed values; failed outputs are reprompted once. Control subtypes, such as money raised or assisted people, are extracted to prevent misclassification and later removed. Fourth, post-processing harmonizes and standardizes values, units, and subtypes; converts currencies to EUR and metric units to SI; normalizes units such as households to people; and geocodes textual locations to polygons using OpenStreetMap Nominatim, GAUL, and Natural Earth at ADMIN0–ADMIN2.
Quality flags track extraction errors, post-processing steps and errors, warnings, controlled removals, and error removals. Final filtering removes duplicates, controlled entries, and critical errors. The final database has 15,147 records for 776 events. Africa and Asia contain the most records, 5,416 and 4,903, covering 269 and 270 events. Floods and storms are most frequent. Geocoding reaches country level for 56% of records, ADMIN1 for 31%, and ADMIN2 for 13%. Validation against 39 manually labelled reports, covering 1,433 sentences, 464 quantitative and 227 qualitative impacts, yields precision 0.71, recall 0.75, and F1 0.73; quantitative precision is 0.68 and qualitative 0.79. Comparisons with EM-DAT and IFRC GO show ROUGE adds quantitative information, particularly for displaced, missing, and homeless people. Limitations include duplicate impacts, missing end dates, geocoding fallbacks, and category fuzziness. Users are advised to check sourceExcerpts, use maximum rather than summed values where appropriate, and apply quality flags. Data and code are available under CC BY 4.0.
Sources: Scientific Data https://www.nature.com/articles/s41597-026-08159-2 .
Comment list ( 0 )
