Перейти в статусOPMET compareVAA graphics

Service Monitoring 2026-06-05 09:13:30 UTC - 6h58m40s

Now: 2026-06-05 16:12:11 UTCAction trace:enabledTrace file:/app/tmp/service_actions.log
PostgreSQL
Status:connected
Config:DB_URL / DATABASE_URL
Value:postgres://weather:***@new_weather_db/weather
Connection:postgres://new_weather_db/weather
Check: OK (0 ms)
METAR decoder
Status:connected
Config:METAR_TAF_DECODE_URL / METS_DECODER_URL
Value:http://metar_taf_parser:7000
Health URL:http://metar_taf_parser:7000/healthcheck
HTTP: 200 (7 ms)
OPMET parser (primary RB)
Status:connected
Config:ASG_MET_PARSER_PRIMARY / *_URL
Value:http://opmet_parser:7000
Health URL:http://opmet_parser:7000/healthcheck
HTTP: 200 (11 ms)
OPMET parser (shadow CS)
Status:connected
Config:ASG_MET_PARSER_SHADOW / *_URL
Value:http://old_meteo_text_parser:5001
Health URL:http://old_meteo_text_parser:5001/
HTTP: 200 (5 ms)
TAF decoder
Status:connected
Config:METAR_TAF_DECODE_URL / METS_DECODER_URL
Value:http://metar_taf_parser:7000
Health URL:http://metar_taf_parser:7000/healthcheck
HTTP: 200 (7 ms)

Потоки данных: что собираем и куда пишем

PipelineSourceCollected dataDestination tables
NOAA METARNOAA metars.cache.csv.gzMETAR/SPECI: ICAO, observed_at, raw_text, weather, wind, pressure, cloudsnoaa_metars noaa_cache_runs noaa_ingest_state aerodromes
NOAA TAFNOAA tafs.cache.xml.gzTAF: ICAO, issued/valid range, raw_text, decoded forecast groupsnoaa_tafs noaa_cache_runs noaa_ingest_state
ArcGIS Tropical CyclonesArcGIS Active_Hurricanes_v1 FeatureServerObserved/forecast positions, tracks, cones, warnings, wind swaths and radii polygonsactive_tropical_cyclones
SADIS Tropical Cyclone AdvisoriesSADIS FTP /TROPICAL_CYCLONE_ADVISORIESWMO/TAC tropical cyclone outlooks and advisories as raw bulletinstropical_cyclone_raw_payloads
ArcGIS Recent Tropical CyclonesArcGIS Recent_Hurricanes_v1 FeatureServerObserved positions, observed tracks, observed wind swaths for recent systemsrecent_tropical_cyclones
Roshydro ingestFTP *.b files (ROSHYDRO_FTP_*)METAR/TAF/OPMET batches + parser outputs + file processing statsroshydro_metars roshydro_tafs roshydro_opmet_processed_files aerodromes opmetdata sigmet_geo airmet_geo gamet_geo
Roshydro OPMET parity (code present, runtime switch pending)Same FTP *.b, ETX/=//END PART splittingAIRMET/SIGMET/GAMET normalized records, parse/build errors, geo projectionsopmetdata opmeterror sigmet_geo airmet_geo gamet_geo
Volcanic Ash AdvisorySADIS FTP + configured VAAC web sourcesVAA raw text, VAAC, volcano, issue/valid times, ash cloud altitude, movement and geometryvolcanic_ash_sources volcanic_ash_advisories volcanic_ash_clouds volcanic_ash_graphics

Active Tropical Cyclones

Existsyes
Rows60
Distinct storms3
Latest fetch2026-06-05 15:58:32 UTC
Ago13m39s

Sources

Source serviceRowsStormsLatest fetchAgo
ARCGIS_ACTIVE_HURRICANES_V15922026-06-05 15:58:32 UTC13m39s
JTWC_SIGNIFICANT_TROPICAL_WEATHER_ADVISORY112026-05-26 03:03:29 UTC10d13h8m42s

Source layers

Source serviceLayer idLayerRowsLatest fetchAgo
ARCGIS_ACTIVE_HURRICANES_V10Forecast Position92026-06-05 15:58:32 UTC13m39s
ARCGIS_ACTIVE_HURRICANES_V11Observed Position212026-06-05 15:58:32 UTC13m39s
ARCGIS_ACTIVE_HURRICANES_V12Forecast Track12026-06-05 15:58:32 UTC13m39s
ARCGIS_ACTIVE_HURRICANES_V13Observed Track32026-06-05 15:58:32 UTC13m39s
ARCGIS_ACTIVE_HURRICANES_V14Forecast Error Cone and Danger Area12026-06-05 15:58:32 UTC13m39s
ARCGIS_ACTIVE_HURRICANES_V17Tropical Storm Force (34kts)102026-06-05 15:58:32 UTC13m39s
ARCGIS_ACTIVE_HURRICANES_V110Raw 1/10th Degree Data (All)132026-06-05 15:58:32 UTC13m39s
ARCGIS_ACTIVE_HURRICANES_V111Observed Wind Swath12026-06-05 15:58:32 UTC13m39s
JTWC_SIGNIFICANT_TROPICAL_WEATHER_ADVISORY3001Tropical Disturbance Summary12026-05-26 03:03:29 UTC10d13h8m42s

Recent Tropical Cyclones

Existsyes
Rows2908
Distinct storms44
Latest fetch2026-06-05 09:23:32 UTC
Ago6h48m39s

Sources

Source serviceRowsStormsLatest fetchAgo
ARCGIS_RECENT_HURRICANES_V11125282026-06-05 03:00:00 UTC13h12m11s
JTWC_RECENT_TROPICAL_CYCLONES_BDECK_UNOFFICIAL_MIRROR599342026-06-05 09:23:32 UTC6h48m39s
NOAA_IBTRACS_RECENT_CSV1184312026-06-05 03:16:02 UTC12h56m9s

Source layers

Source serviceLayer idLayerRowsLatest fetchAgo
ARCGIS_RECENT_HURRICANES_V10Observed Position9412026-06-05 03:00:00 UTC13h12m11s
ARCGIS_RECENT_HURRICANES_V11Observed Track1842026-06-05 03:00:00 UTC13h12m11s
JTWC_RECENT_TROPICAL_CYCLONES_BDECK_UNOFFICIAL_MIRROR2100Observed Position5572026-06-05 09:23:32 UTC6h48m39s
JTWC_RECENT_TROPICAL_CYCLONES_BDECK_UNOFFICIAL_MIRROR2101Observed Track422026-06-03 03:20:01 UTC2d12h52m10s
NOAA_IBTRACS_RECENT_CSV2000Observed Position11552026-06-05 03:16:02 UTC12h56m9s
NOAA_IBTRACS_RECENT_CSV2001Observed Track292026-05-27 06:51:40 UTC9d9h20m30s

Tropical Cyclone Raw Payloads

Existsyes
Rows7338
Latest fetch2026-06-05 16:09:47 UTC
Ago2m24s

Sources

Source serviceRowsLatest fetchAgo
JTWC_SIGNIFICANT_TROPICAL_WEATHER_ADVISORY22026-05-26 03:03:29 UTC10d13h8m42s
NOAA_NHC_GIS_RSS72902026-06-05 16:02:21 UTC9m50s
SADIS_TROPICAL_CYCLONE_ADVISORIES462026-06-05 16:09:47 UTC2m24s
Products:nhc_gis_rss=7290,significant_tropical_weather_advisory=2,tropical_cyclone_advisory=46Statuses:no_active_cyclones=7113,raw=225

Recent payloads

SourceProductStatusBasinStormIssuedFetchedAgo
SADIS_TROPICAL_CYCLONE_ADVISORIESFKPZ21_KNHC_051443raw———2026-06-05 16:09:47 UTC2m24s
SADIS_TROPICAL_CYCLONE_ADVISORIESFKPZ21_KNHC_050833raw———2026-06-05 16:09:47 UTC2m24s
SADIS_TROPICAL_CYCLONE_ADVISORIESFKPZ21_KNHC_050232raw———2026-06-05 16:09:47 UTC2m24s
SADIS_TROPICAL_CYCLONE_ADVISORIESFKPZ21_KNHC_042032raw———2026-06-05 16:09:47 UTC2m24s
SADIS_TROPICAL_CYCLONE_ADVISORIESFKPZ21_KNHC_041439raw———2026-06-05 16:09:47 UTC2m24s
SADIS_TROPICAL_CYCLONE_ADVISORIESFKPZ21_KNHC_040834raw———2026-06-05 16:09:47 UTC2m24s
NOAA_NHC_GIS_RSScentral_pacificno_active_cyclonescentral_pacific——2026-06-05 16:02:21 UTC9m50s
NOAA_NHC_GIS_RSSeast_pacificraweast_pacific——2026-06-05 16:02:21 UTC9m50s
NOAA_NHC_GIS_RSSatlanticno_active_cyclonesatlantic——2026-06-05 16:02:21 UTC9m50s
NOAA_NHC_GIS_RSScentral_pacificno_active_cyclonescentral_pacific——2026-06-05 15:47:20 UTC24m51s
NOAA_NHC_GIS_RSSeast_pacificraweast_pacific——2026-06-05 15:47:20 UTC24m51s
NOAA_NHC_GIS_RSSatlanticno_active_cyclonesatlantic——2026-06-05 15:47:20 UTC24m51s

Volcanic Ash Advisory

Pipeline:enabledLast run: 2026-06-05 16:08:40 UTCRun status: okAgo: 3m31sFreshness: warn 120 min / critical 360 min
Sources: 6Advisories: 1688Clouds: 2076Graphics: 134 (georeferenced=61,no_geometry=11,parsed=62)State: fresh=3 warning=1 critical=2 missing=0
SourceVAACStatusLatest issueAgoLatest advisory24h48hTotalLast error
DARWIN_BOM_AGGREGATEDARWINcritical——— / —22 adv / 83 clouds42 adv / 151 clouds191 adv / 666 clouds—
MONTREAL_ECCCMONTREALfresh——— / —0 adv / 0 clouds0 adv / 0 clouds940 adv / 0 clouds—
SADIS_FTPMULTIfresh2026-06-05 15:18:00 UTC54m11sLEWOTOBI / 2026/8733 adv / 94 clouds68 adv / 177 clouds384 adv / 1027 clouds—
TOKYO_JMATOKYOwarning2026-06-05 12:00:00 UTC4h12m11sSHEVELUCH / 2026/17012 adv / 28 clouds23 adv / 48 clouds90 adv / 121 clouds—
TOULOUSE_METEO_FRTOULOUSEcritical2026-06-03 14:58:00 UTC2d1h14m11sTERCEIRA / 2026/70 adv / 0 clouds0 adv / 0 clouds10 adv / 29 clouds—
WASHINGTON_OSPOWASHINGTONfresh2026-06-05 14:52:00 UTC1h20m11sFUEGO / 2026/6544 adv / 16 clouds10 adv / 40 clouds73 adv / 233 clouds—

Таблицы и последние изменения данных

TableExistsRowsLatestAgoError
aerodromesyes56482026-06-04 11:35:31 UTC1d4h36m40s—
noaa_metarsyes1960842026-06-05 16:09:00 UTC3m11s—
noaa_tafsyes290842026-06-05 16:08:00 UTC4m11s—
roshydro_metarsyes1078262026-06-05 16:09:00 UTC3m11s—
roshydro_tafsyes275092026-06-06 16:00:00 UTCnow—
active_tropical_cyclonesyes602026-06-05 15:58:32 UTC13m39s—
recent_tropical_cyclonesyes29082026-06-05 09:23:32 UTC6h48m39s—
tropical_cyclone_raw_payloadsyes73382026-06-05 16:09:47 UTC2m24s—
roshydro_opmet_processed_filesyes3357482026-06-05 16:11:52 UTC18s—
noaa_cache_runsyes1263472026-06-05 16:11:52 UTC19s—
noaa_ingest_stateyes2———
opmetdatayes226142026-06-05 16:11:52 UTC19s—
opmeterroryes0———
sigmet_geoyes115272026-06-05 16:11:52 UTC19s—
airmet_geoyes26412026-06-05 16:02:31 UTC9m40s—
gamet_geoyes84462026-06-05 16:06:45 UTC5m26s—
volcanic_ash_sourcesyes62026-06-05 16:11:03 UTC1m8s—
volcanic_ash_advisoriesyes16882026-06-05 15:18:00 UTC54m11s—
volcanic_ash_cloudsyes20762026-06-06 09:00:00 UTCnow—
volcanic_ash_graphicsyes1342026-06-05 14:52:00 UTC1h20m11s—

Runtime env snapshot

VariableValue
RECENT_TROPICAL_CYCLONES_SOURCEALL
ACTIVE_TROPICAL_CYCLONES_SOURCEALL
TROPICAL_CYCLONES_JOB_INTERVAL_MIN15
RECENT_TROPICAL_CYCLONES_JOB_INTERVAL_MIN15
RECENT_TROPICAL_CYCLONES_LOOKBACK_DAYS130
ROSHYDRO_JOB_INTERVAL_MIN—
NOAA_METAR_JOB_INTERVAL_MIN—
NOAA_TAF_JOB_INTERVAL_MIN—
NOAA_METAR_CACHE_URL—
NOAA_TAF_CACHE_URL—
ARCGIS_TROPICAL_CYCLONES_URLhttps://services9.arcgis.com/RHVPKKiFTONKtxq3/ArcGIS/rest/services/Active_Hurricanes_v1/FeatureServer
ARCGIS_RECENT_TROPICAL_CYCLONES_URLhttps://services9.arcgis.com/RHVPKKiFTONKtxq3/ArcGIS/rest/services/Recent_Hurricanes_v1/FeatureServer
NOAA_IBTRACS_RECENT_CSV_URLhttps://www.ncei.noaa.gov/data/international-best-track-archive-for-climate-stewardship-ibtracs/v04r01/access/csv/ibtracs.last3years.list.v04r01.csv
JTWC_TROPICAL_CYCLONES_PRODUCTS_BASE_URLhttps://www.metoc.navy.mil/jtwc/products/
JTWC_RECENT_TROPICAL_CYCLONES_BDECK_UNOFFICIAL_MIRROR_BASE_URLhttps://hurricanes.ral.ucar.edu/repository/data/bdecks_open/
SADIS_FTP_TC_ADVISORIES_DIR/TROPICAL_CYCLONE_ADVISORIES
SADIS_TROPICAL_CYCLONES_FETCH_LIMIT50
VOLCANIC_ASH_PIPELINE_ENABLEDtrue
VOLCANIC_ASH_SOURCESSADIS,WASHINGTON,TOKYO,MONTREAL,TOULOUSE,DARWIN_AGGREGATE
VOLCANIC_ASH_FETCH_LIMIT50
VOLCANIC_ASH_JOB_INTERVAL_MIN15
VOLCANIC_ASH_FRESHNESS_WARN_MIN120
VOLCANIC_ASH_FRESHNESS_CRITICAL_MIN360
SADIS_FTP_HOSTsadisftp.metoffice.gov.uk
SADIS_FTP_VA_ADVISORIES_DIR/VOLCANIC_ASH_ADVISORY_STATEMENTS
ROSHYDRO_FTP_HOST193.7.166.1
ROSHYDRO_FTP_REMOTE_GLOB—
OPMET_COMPARE_ENABLEDfalse
OPMET_COMPARE_OUTPUT_DIRcompare_opmet
OPMET_COMPARE_LIMIT—
OPMET_ERROR_RETENTION_HOURS—
OPMET_ERROR_CLEANUP_INTERVAL_MIN—
METS_DECODER_URLhttp://metar_taf_parser:7000
ASG_MET_PARSER—
ASG_MET_PARSER_MODE—
ASG_MET_PARSER_PRIMARYRB
ASG_MET_PARSER_SHADOWCS
ASG_MET_PARSER_CS_URLhttp://old_meteo_text_parser:5001
ASG_MET_PARSER_RB_URLhttp://opmet_parser:7000
ACTION_TRACE_ENABLEDtrue

OPMET errors

Top error types:
created_atopmet_typeerror_typemessageopmetdata_idoriginal_file

Temporary action trace (TEST)

Это временный функционал для тестирования. После завершения тестов удалить:src/app/services/support/action_recorder.rb, маршрут/monitor/actions и вызовыActionRecorder.record(...)

[
  {
    "at": "2026-06-05T16:12:11Z",
    "source": "api.monitor",
    "status": "info",
    "action": "monitor_requested",
    "details": null
  },
  {
    "at": "2026-06-05T16:11:52Z",
    "source": "orchestrator.roshydro",
    "status": "ok",
    "action": "ingest_cycle_finished",
    "details": {
      "files_count": 3
    }
  },
  {
    "at": "2026-06-05T16:11:52Z",
    "source": "orchestrator.roshydro",
    "status": "info",
    "action": "opmet_shadow_processed",
    "details": {
      "filename": "UUUJ03081477.b",
      "opmet_primary_mode": "RB",
      "opmet_shadow_mode": "CS"
    }
  },
  {
    "at": "2026-06-05T16:11:52Z",
    "source": "orchestrator.roshydro",
    "status": "info",
    "action": "processing_file",
    "details": {
      "filename": "UUUJ03081477.b"
    }
  },
  {
    "at": "2026-06-05T16:11:52Z",
    "source": "orchestrator.roshydro",
    "status": "info",
    "action": "opmet_shadow_processed",
    "details": {
      "filename": "UUUJ03081476.b",
      "opmet_primary_mode": "RB",
      "opmet_shadow_mode": "CS"
    }
  },
  {
    "at": "2026-06-05T16:11:52Z",
    "source": "orchestrator.noaa.metar",
    "status": "ok",
    "action": "ingest_cycle_finished",
    "details": {
      "messages_count": 145,
      "skipped": 5096
    }
  },
  {
    "at": "2026-06-05T16:11:51Z",
    "source": "orchestrator.roshydro",
    "status": "info",
    "action": "processing_file",
    "details": {
      "filename": "UUUJ03081476.b"
    }
  },
  {
    "at": "2026-06-05T16:11:51Z",
    "source": "orchestrator.roshydro",
    "status": "info",
    "action": "opmet_shadow_processed",
    "details": {
      "filename": "UUUJ03081475.b",
      "opmet_primary_mode": "RB",
      "opmet_shadow_mode": "CS"
    }
  },
  {
    "at": "2026-06-05T16:11:51Z",
    "source": "orchestrator.noaa.taf",
    "status": "ok",
    "action": "ingest_cycle_finished",
    "details": {
      "messages_count": 69,
      "skipped": 3006
    }
  },
  {
    "at": "2026-06-05T16:11:51Z",
    "source": "metar_ingest",
    "status": "skipped",
    "action": "skip_future_metar",
    "details": {
      "reason": "future_observed_at",
      "source": "noaa",
      "station": "SGCO",
      "observed_at": "2026-06-05T18:00:00Z",
      "now_utc": "2026-06-05T16:11:51Z"
    }
  },
  {
    "at": "2026-06-05T16:11:51Z",
    "source": "metar_ingest",
    "status": "skipped",
    "action": "skip_future_metar",
    "details": {
      "reason": "future_observed_at",
      "source": "noaa",
      "station": "SGGR",
      "observed_at": "2026-06-05T18:00:00Z",
      "now_utc": "2026-06-05T16:11:51Z"
    }
  },
  {
    "at": "2026-06-05T16:11:51Z",
    "source": "metar_ingest",
    "status": "skipped",
    "action": "skip_future_metar",
    "details": {
      "reason": "future_observed_at",
      "source": "noaa",
      "station": "SGSP",
      "observed_at": "2026-06-05T18:00:00Z",
      "now_utc": "2026-06-05T16:11:51Z"
    }
  },
  {
    "at": "2026-06-05T16:11:51Z",
    "source": "metar_ingest",
    "status": "skipped",
    "action": "skip_future_metar",
    "details": {
      "reason": "future_observed_at",
      "source": "noaa",
      "station": "SGPI",
      "observed_at": "2026-06-05T18:00:00Z",
      "now_utc": "2026-06-05T16:11:51Z"
    }
  },
  {
    "at": "2026-06-05T16:11:51Z",
    "source": "metar_ingest",
    "status": "skipped",
    "action": "skip_future_metar",
    "details": {
      "reason": "future_observed_at",
      "source": "noaa",
      "station": "SGPJ",
      "observed_at": "2026-06-05T18:00:00Z",
      "now_utc": "2026-06-05T16:11:51Z"
    }
  },
  {
    "at": "2026-06-05T16:11:51Z",
    "source": "metar_ingest",
    "status": "skipped",
    "action": "skip_future_metar",
    "details": {
      "reason": "future_observed_at",
      "source": "noaa",
      "station": "SGME",
      "observed_at": "2026-06-05T18:00:00Z",
      "now_utc": "2026-06-05T16:11:51Z"
    }
  },
  {
    "at": "2026-06-05T16:11:32Z",
    "source": "orchestrator.noaa.metar",
    "status": "info",
    "action": "ingest_cycle_started",
    "details": null
  },
  {
    "at": "2026-06-05T16:11:02Z",
    "source": "orchestrator.volcanic_ash.advisories",
    "status": "info",
    "action": "ingest_cycle_started",
    "details": {
      "source_names": [
        "DARWIN_AGGREGATE"
      ]
    }
  },
  {
    "at": "2026-06-05T16:10:41Z",
    "source": "orchestrator.roshydro",
    "status": "info",
    "action": "processing_file",
    "details": {
      "filename": "UUUJ03081475.b"
    }
  },
  {
    "at": "2026-06-05T16:10:41Z",
    "source": "orchestrator.roshydro",
    "status": "info",
    "action": "files_detected",
    "details": {
      "count": 3
    }
  },
  {
    "at": "2026-06-05T16:10:32Z",
    "source": "orchestrator.roshydro",
    "status": "info",
    "action": "dependencies_ok",
    "details": {
      "opmet_primary_mode": "RB",
      "opmet_primary_url": "http://opmet_parser:7000",
      "opmet_shadow_mode": "CS",
      "opmet_shadow_url": "http://old_meteo_text_parser:5001"
    }
  },
  {
    "at": "2026-06-05T16:10:32Z",
    "source": "orchestrator.noaa.taf",
    "status": "info",
    "action": "ingest_cycle_started",
    "details": null
  },
  {
    "at": "2026-06-05T16:10:32Z",
    "source": "orchestrator.roshydro",
    "status": "info",
    "action": "ingest_cycle_started",
    "details": null
  },
  {
    "at": "2026-06-05T16:09:50Z",
    "source": "orchestrator.noaa.metar",
    "status": "ok",
    "action": "ingest_cycle_finished",
    "details": {
      "messages_count": 522,
      "skipped": 4720
    }
  },
  {
    "at": "2026-06-05T16:09:50Z",
    "source": "orchestrator.sadis.tropical_cyclones",
    "status": "ok",
    "action": "ingest_cycle_finished",
    "details": {
      "remote_dir": "/TROPICAL_CYCLONE_ADVISORIES",
      "files_count": 6,
      "raw_payloads_saved": 6
    }
  },
  {
    "at": "2026-06-05T16:09:48Z",
    "source": "orchestrator.roshydro",
    "status": "ok",
    "action": "ingest_cycle_finished",
    "details": {
      "files_count": 2
    }
  },
  {
    "at": "2026-06-05T16:09:48Z",
    "source": "orchestrator.roshydro",
    "status": "info",
    "action": "opmet_shadow_processed",
    "details": {
      "filename": "UUUJ03081474.b",
      "opmet_primary_mode": "RB",
      "opmet_shadow_mode": "CS"
    }
  },
  {
    "at": "2026-06-05T16:09:47Z",
    "source": "orchestrator.sadis.tropical_cyclones",
    "status": "info",
    "action": "ingest_cycle_started",
    "details": null
  },
  {
    "at": "2026-06-05T16:09:47Z",
    "source": "orchestrator.roshydro",
    "status": "info",
    "action": "processing_file",
    "details": {
      "filename": "UUUJ03081474.b"
    }
  },
  {
    "at": "2026-06-05T16:09:47Z",
    "source": "orchestrator.roshydro",
    "status": "info",
    "action": "opmet_shadow_processed",
    "details": {
      "filename": "UUUJ03081473.b",
      "opmet_primary_mode": "RB",
      "opmet_shadow_mode": "CS"
    }
  },
  {
    "at": "2026-06-05T16:09:47Z",
    "source": "metar_ingest",
    "status": "skipped",
    "action": "skip_future_metar",
    "details": {
      "reason": "future_observed_at",
      "source": "noaa",
      "station": "SGCO",
      "observed_at": "2026-06-05T18:00:00Z",
      "now_utc": "2026-06-05T16:09:47Z"
    }
  },
  {
    "at": "2026-06-05T16:09:47Z",
    "source": "metar_ingest",
    "status": "skipped",
    "action": "skip_future_metar",
    "details": {
      "reason": "future_observed_at",
      "source": "noaa",
      "station": "SGGR",
      "observed_at": "2026-06-05T18:00:00Z",
      "now_utc": "2026-06-05T16:09:47Z"
    }
  },
  {
    "at": "2026-06-05T16:09:47Z",
    "source": "metar_ingest",
    "status": "skipped",
    "action": "skip_future_metar",
    "details": {
      "reason": "future_observed_at",
      "source": "noaa",
      "station": "SGSP",
      "observed_at": "2026-06-05T18:00:00Z",
      "now_utc": "2026-06-05T16:09:47Z"
    }
  },
  {
    "at": "2026-06-05T16:09:47Z",
    "source": "metar_ingest",
    "status": "skipped",
    "action": "skip_future_metar",
    "details": {
      "reason": "future_observed_at",
      "source": "noaa",
      "station": "SGPI",
      "observed_at": "2026-06-05T18:00:00Z",
      "now_utc": "2026-06-05T16:09:47Z"
    }
  },
  {
    "at": "2026-06-05T16:09:47Z",
    "source": "metar_ingest",
    "status": "skipped",
    "action": "skip_future_metar",
    "details": {
      "reason": "future_observed_at",
      "source": "noaa",
      "station": "SGME",
      "observed_at": "2026-06-05T18:00:00Z",
      "now_utc": "2026-06-05T16:09:47Z"
    }
  },
  {
    "at": "2026-06-05T16:09:47Z",
    "source": "metar_ingest",
    "status": "skipped",
    "action": "skip_future_metar",
    "details": {
      "reason": "future_observed_at",
      "source": "noaa",
      "station": "SGPJ",
      "observed_at": "2026-06-05T18:00:00Z",
      "now_utc": "2026-06-05T16:09:47Z"
    }
  },
  {
    "at": "2026-06-05T16:09:45Z",
    "source": "metar_ingest",
    "status": "skipped",
    "action": "skip_future_metar",
    "details": {
      "reason": "future_observed_at",
      "source": "roshydro",
      "station": "KBCB",
      "observed_at": "2026-06-05T16:15:00Z",
      "now_utc": "2026-06-05T16:09:45Z"
    }
  },
  {
    "at": "2026-06-05T16:09:41Z",
    "source": "orchestrator.roshydro",
    "status": "info",
    "action": "processing_file",
    "details": {
      "filename": "UUUJ03081473.b"
    }
  },
  {
    "at": "2026-06-05T16:09:41Z",
    "source": "orchestrator.roshydro",
    "status": "info",
    "action": "files_detected",
    "details": {
      "count": 2
    }
  },
  {
    "at": "2026-06-05T16:09:32Z",
    "source": "orchestrator.roshydro",
    "status": "info",
    "action": "dependencies_ok",
    "details": {
      "opmet_primary_mode": "RB",
      "opmet_primary_url": "http://opmet_parser:7000",
      "opmet_shadow_mode": "CS",
      "opmet_shadow_url": "http://old_meteo_text_parser:5001"
    }
  },
  {
    "at": "2026-06-05T16:09:32Z",
    "source": "orchestrator.roshydro",
    "status": "info",
    "action": "ingest_cycle_started",
    "details": null
  },
  {
    "at": "2026-06-05T16:09:32Z",
    "source": "orchestrator.noaa.metar",
    "status": "info",
    "action": "ingest_cycle_started",
    "details": null
  },
  {
    "at": "2026-06-05T16:08:52Z",
    "source": "orchestrator.noaa.metar",
    "status": "ok",
    "action": "ingest_cycle_finished",
    "details": {
      "messages_count": 425,
      "skipped": 4821
    }
  },
  {
    "at": "2026-06-05T16:08:51Z",
    "source": "orchestrator.roshydro",
    "status": "ok",
    "action": "ingest_cycle_finished",
    "details": {
      "files_count": 1
    }
  },
  {
    "at": "2026-06-05T16:08:51Z",
    "source": "orchestrator.roshydro",
    "status": "info",
    "action": "opmet_shadow_processed",
    "details": {
      "filename": "UUUJ03081472.b",
      "opmet_primary_mode": "RB",
      "opmet_shadow_mode": "CS"
    }
  },
  {
    "at": "2026-06-05T16:08:49Z",
    "source": "orchestrator.noaa.taf",
    "status": "ok",
    "action": "ingest_cycle_finished",
    "details": {
      "messages_count": 22,
      "skipped": 3015
    }
  },
  {
    "at": "2026-06-05T16:08:40Z",
    "source": "orchestrator.volcanic_ash.advisories",
    "status": "ok",
    "action": "ingest_cycle_finished",
    "details": {
      "sources": {
        "toulouse_meteo_fr": {
          "fetched": 1,
          "saved": 1,
          "errors": []
        }
      },
      "fetched": 1,
      "saved": 1,
      "errors": []
    }
  },
  {
    "at": "2026-06-05T16:08:32Z",
    "source": "orchestrator.volcanic_ash.advisories",
    "status": "info",
    "action": "ingest_cycle_started",
    "details": {
      "source_names": [
        "TOULOUSE"
      ]
    }
  },
  {
    "at": "2026-06-05T16:08:09Z",
    "source": "metar_ingest",
    "status": "skipped",
    "action": "skip_future_metar",
    "details": {
      "reason": "future_observed_at",
      "source": "noaa",
      "station": "SGCO",
      "observed_at": "2026-06-05T18:00:00Z",
      "now_utc": "2026-06-05T16:08:07Z"
    }
  },
  {
    "at": "2026-06-05T16:08:09Z",
    "source": "metar_ingest",
    "status": "skipped",
    "action": "skip_future_metar",
    "details": {
      "reason": "future_observed_at",
      "source": "noaa",
      "station": "SGSP",
      "observed_at": "2026-06-05T18:00:00Z",
      "now_utc": "2026-06-05T16:08:07Z"
    }
  },
  {
    "at": "2026-06-05T16:08:08Z",
    "source": "metar_ingest",
    "status": "skipped",
    "action": "skip_future_metar",
    "details": {
      "reason": "future_observed_at",
      "source": "noaa",
      "station": "SGGR",
      "observed_at": "2026-06-05T18:00:00Z",
      "now_utc": "2026-06-05T16:08:07Z"
    }
  },
  {
    "at": "2026-06-05T16:08:08Z",
    "source": "metar_ingest",
    "status": "skipped",
    "action": "skip_future_metar",
    "details": {
      "reason": "future_observed_at",
      "source": "noaa",
      "station": "SGPI",
      "observed_at": "2026-06-05T18:00:00Z",
      "now_utc": "2026-06-05T16:08:07Z"
    }
  },
  {
    "at": "2026-06-05T16:08:08Z",
    "source": "metar_ingest",
    "status": "skipped",
    "action": "skip_future_metar",
    "details": {
      "reason": "future_observed_at",
      "source": "noaa",
      "station": "SGPJ",
      "observed_at": "2026-06-05T18:00:00Z",
      "now_utc": "2026-06-05T16:08:07Z"
    }
  },
  {
    "at": "2026-06-05T16:08:07Z",
    "source": "metar_ingest",
    "status": "skipped",
    "action": "skip_future_metar",
    "details": {
      "reason": "future_observed_at",
      "source": "noaa",
      "station": "SGME",
      "observed_at": "2026-06-05T18:00:00Z",
      "now_utc": "2026-06-05T16:08:07Z"
    }
  },
  {
    "at": "2026-06-05T16:07:42Z",
    "source": "orchestrator.roshydro",
    "status": "info",
    "action": "processing_file",
    "details": {
      "filename": "UUUJ03081472.b"
    }
  },
  {
    "at": "2026-06-05T16:07:42Z",
    "source": "orchestrator.roshydro",
    "status": "info",
    "action": "files_detected",
    "details": {
      "count": 1
    }
  },
  {
    "at": "2026-06-05T16:07:32Z",
    "source": "orchestrator.roshydro",
    "status": "info",
    "action": "dependencies_ok",
    "details": {
      "opmet_primary_mode": "RB",
      "opmet_primary_url": "http://opmet_parser:7000",
      "opmet_shadow_mode": "CS",
      "opmet_shadow_url": "http://old_meteo_text_parser:5001"
    }
  },
  {
    "at": "2026-06-05T16:07:32Z",
    "source": "orchestrator.noaa.taf",
    "status": "info",
    "action": "ingest_cycle_started",
    "details": null
  },
  {
    "at": "2026-06-05T16:07:32Z",
    "source": "orchestrator.noaa.metar",
    "status": "info",
    "action": "ingest_cycle_started",
    "details": null
  },
  {
    "at": "2026-06-05T16:07:32Z",
    "source": "orchestrator.roshydro",
    "status": "info",
    "action": "ingest_cycle_started",
    "details": null
  },
  {
    "at": "2026-06-05T16:06:48Z",
    "source": "orchestrator.roshydro",
    "status": "ok",
    "action": "ingest_cycle_finished",
    "details": {
      "files_count": 3
    }
  },
  {
    "at": "2026-06-05T16:06:48Z",
    "source": "orchestrator.roshydro",
    "status": "info",
    "action": "opmet_shadow_processed",
    "details": {
      "filename": "UUUJ03081471.b",
      "opmet_primary_mode": "RB",
      "opmet_shadow_mode": "CS"
    }
  },
  {
    "at": "2026-06-05T16:06:45Z",
    "source": "orchestrator.roshydro",
    "status": "info",
    "action": "processing_file",
    "details": {
      "filename": "UUUJ03081471.b"
    }
  },
  {
    "at": "2026-06-05T16:06:45Z",
    "source": "orchestrator.roshydro",
    "status": "info",
    "action": "opmet_shadow_processed",
    "details": {
      "filename": "UUUJ03081470.b",
      "opmet_primary_mode": "RB",
      "opmet_shadow_mode": "CS"
    }
  },
  {
    "at": "2026-06-05T16:06:44Z",
    "source": "orchestrator.roshydro",
    "status": "info",
    "action": "processing_file",
    "details": {
      "filename": "UUUJ03081470.b"
    }
  },
  {
    "at": "2026-06-05T16:06:44Z",
    "source": "orchestrator.roshydro",
    "status": "info",
    "action": "opmet_shadow_processed",
    "details": {
      "filename": "UUUJ03081469.b",
      "opmet_primary_mode": "RB",
      "opmet_shadow_mode": "CS"
    }
  },
  {
    "at": "2026-06-05T16:06:42Z",
    "source": "orchestrator.roshydro",
    "status": "info",
    "action": "processing_file",
    "details": {
      "filename": "UUUJ03081469.b"
    }
  },
  {
    "at": "2026-06-05T16:06:42Z",
    "source": "orchestrator.roshydro",
    "status": "info",
    "action": "files_detected",
    "details": {
      "count": 3
    }
  },
  {
    "at": "2026-06-05T16:06:32Z",
    "source": "orchestrator.roshydro",
    "status": "info",
    "action": "dependencies_ok",
    "details": {
      "opmet_primary_mode": "RB",
      "opmet_primary_url": "http://opmet_parser:7000",
      "opmet_shadow_mode": "CS",
      "opmet_shadow_url": "http://old_meteo_text_parser:5001"
    }
  },
  {
    "at": "2026-06-05T16:06:32Z",
    "source": "orchestrator.roshydro",
    "status": "info",
    "action": "ingest_cycle_started",
    "details": null
  },
  {
    "at": "2026-06-05T16:06:06Z",
    "source": "orchestrator.volcanic_ash.advisories",
    "status": "ok",
    "action": "ingest_cycle_finished",
    "details": {
      "sources": {
        "montreal_eccc": {
          "fetched": 4,
          "saved": 4,
          "errors": []
        }
      },
      "fetched": 4,
      "saved": 4,
      "errors": []
    }
  },
  {
    "at": "2026-06-05T16:06:04Z",
    "source": "orchestrator.jtwc.tropical_cyclones",
    "status": "partial",
    "action": "ingest_cycle_finished",
    "details": {
      "advisories_count": 0,
      "records_count": 0,
      "saved_count": 0,
      "fetch_errors": [
        {
          "error_class": "Timeout::Error",
          "error_message": "execution expired",
          "product": "abpwweb.txt"
        },
        {
          "error_class": "SocketError",
          "error_message": "DNS resolution is temporarily unavailable for www.metoc.navy.mil",
          "product": "abioweb.txt"
        }
      ]
    }
  },
  {
    "at": "2026-06-05T16:06:02Z",
    "source": "orchestrator.volcanic_ash.advisories",
    "status": "info",
    "action": "ingest_cycle_started",
    "details": {
      "source_names": [
        "MONTREAL"
      ]
    }
  },
  {
    "at": "2026-06-05T16:06:02Z",
    "source": "orchestrator.jtwc.tropical_cyclones",
    "status": "info",
    "action": "ingest_cycle_started",
    "details": null
  },
  {
    "at": "2026-06-05T16:06:00Z",
    "source": "orchestrator.roshydro",
    "status": "ok",
    "action": "ingest_cycle_finished",
    "details": {
      "files_count": 4
    }
  },
  {
    "at": "2026-06-05T16:06:00Z",
    "source": "orchestrator.roshydro",
    "status": "info",
    "action": "opmet_shadow_processed",
    "details": {
      "filename": "UUUJ03081468.b",
      "opmet_primary_mode": "RB",
      "opmet_shadow_mode": "CS"
    }
  },
  {
    "at": "2026-06-05T16:06:00Z",
    "source": "orchestrator.roshydro",
    "status": "info",
    "action": "processing_file",
    "details": {
      "filename": "UUUJ03081468.b"
    }
  },
  {
    "at": "2026-06-05T16:06:00Z",
    "source": "orchestrator.roshydro",
    "status": "info",
    "action": "opmet_shadow_processed",
    "details": {
      "filename": "UUUJ03081467.b",
      "opmet_primary_mode": "RB",
      "opmet_shadow_mode": "CS"
    }
  },
  {
    "at": "2026-06-05T16:05:58Z",
    "source": "orchestrator.roshydro",
    "status": "info",
    "action": "processing_file",
    "details": {
      "filename": "UUUJ03081467.b"
    }
  },
  {
    "at": "2026-06-05T16:05:58Z",
    "source": "orchestrator.roshydro",
    "status": "info",
    "action": "opmet_shadow_processed",
    "details": {
      "filename": "UUUJ03081466.b",
      "opmet_primary_mode": "RB",
      "opmet_shadow_mode": "CS"
    }
  },
  {
    "at": "2026-06-05T16:05:58Z",
    "source": "orchestrator.noaa.metar",
    "status": "ok",
    "action": "ingest_cycle_finished",
    "details": {
      "messages_count": 402,
      "skipped": 4845
    }
  }
]