Feature: World / Geo map panel for custom dashboards (GeoIP)

[SemoTech]

Problem

Self-hosted / WAN reverse-proxy users (Caddy, nginx, Traefik) often want a geography view of inbound clients: where in the world traffic to exposed hostnames is coming from.

LogTide already has:

• GeoIP enrichment via parsing pipelines (embedded GeoLite2 → country, city, coordinates, ISP)
• Custom dashboards with time series, Top-N, live stream, stats, etc.

What is missing is a first-class map panel on custom dashboards so GeoIP coordinates can be plotted (e.g. flat/equirectangular world map with dots or heat for hit volume).

Today, maps appear mainly in SIEM incident detail views, not as a composable dashboard widget. Teams fall back to Top-N by country or export to Grafana GeoMap.

Proposal

Add a dashboard panel type, e.g. Geo map / World map:

1. Data source: logs (project-scoped), using GeoIP lat/lon (or country centroids as fallback)
2. Visual: flattened / equirectangular world map (dark/light friendly)
3. Encoding: point size or heat by hit count; optional color by http_status / severity
4. Filters: same as other panels (time range, service, host, query string)
5. Tooltip / side panel: source IP, city, country, target host, status, hit count, last seen
6. Companion: works well next to existing Top-N tables (top countries, top hosts, top IPs)

Why it fits LogTide

• Complements pipelines GeoIP (enrich once → visualize many places)
• High value for edge / self-hosted reverse-proxy and SIEM-ish WAN monitoring
• Panel registry is already designed for new types (shared types + fetcher + Svelte panel)

Alternatives considered

• Top-N by country only (useful, but not spatial)
• Grafana GeoMap beside LogTide (works, but splits the UX)
• Incident-only maps (too late / too narrow for routine WAN ops dashboards)

Additional context

Happy to help refine UX (flat map vs globe, clustering at zoom levels, etc.). Related: custom dashboards + GeoIP pipelines from 0.9.x.

[Polliog]

Sorry for the very slow reply, work has been keeping me busy.

I really like this idea and I want to build it. Here is where things stand after looking at the code, so we can shape it properly.

What is already in place

• The geoip pipeline step uses an embedded GeoLite2 City database (auto refreshed daily by the worker) and produces country, countryCode, city, latitude, longitude, timezone, subdivision, postalCode. There is no ISP/ASN today, so that part of the proposal would need a separate data source.
• Leaflet is already a frontend dependency: it powers the GeoIP map in the SIEM incident detail view. A dashboard panel would not add a new mapping library.
• The custom dashboard panel registry is genuinely pluggable (shared type + Zod schema + backend data fetcher + Svelte panel + config form, and nothing else needs touching), so the panel itself is the easy half.

The real blocker, and it is not the map

The geoip step writes a nested object under metadata.<target> (for example metadata.geo.country). The group-by primitive that Top-N uses treats everything after metadata. as a single JSON key, so on TimescaleDB and ClickHouse metadata.geo.country resolves to a literal key named geo.country and returns nothing. MongoDB does resolve the path. LogTide is multi engine by design, so the aggregation layer has to behave identically on all three before any map panel is meaningful.

So the work splits into:

1. Aggregation: either have the geoip step also write flat keys, or add nested path support to the query translators for all three engines (plus a server side aggregation that returns buckets rather than raw rows).
2. The panel itself: schema, fetcher, Svelte component, config form.

Design questions where your input would help

1. Flat/equirectangular map only, or is a globe worth it? My instinct is flat only, it is easier to read at dashboard panel size.
2. Aggregate by country (centroid, choropleth-ish) versus raw lat/lon points with clustering. I am leaning towards two explicit modes on the same panel: an aggregate mode that is cheap and always safe, and a points mode capped to a top N, since a tooltip with individual source IPs is close to raw data and expensive at dashboard refresh rates.
3. Color by http_status / severity: fixed dimensions, or a configurable metadata field?
4. Privacy: plotting source IPs on a shared dashboard is a different exposure level than an incident detail view, and PII masking rules may already redact the IP field before storage. Any preference on how visible individual IPs should be by default?

I will not promise a date, but this is on my list and I will open a tracking issue with the design once the aggregation part is settled.

[Polliog]

Update: this is implemented and merged to develop, it will ship with the next release.

What landed:

A new geo_map panel type for custom dashboards, with two modes:

• By country: aggregate bubbles placed on country centroids, sized by hit volume (square-root scale, so bubble area tracks volume).
• By location: bubbles on actual GeoIP coordinates, pre-rounded to ~1 km, with city-level tooltips ("Rome, Italy - 1,240"). Top-N capped and configurable.

Filters: time window (1h/24h/7d), log levels, service, hostname, project scope. The geo field prefix is configurable too, so if you run two GeoIP steps on the same pipeline (say client IP and upstream IP) you can point a panel at either.

How it gets its data: the GeoIP pipeline step now writes flat metadata keys next to its existing nested object (geo_country_code, geo_place, etc.). That is what makes the map a cheap single group-by on all three storage engines instead of a raw-row scan. Two consequences worth knowing:

• Existing pipelines keep working unchanged, but logs ingested before this change do not have the flat keys, so they will not appear on the map. New traffic shows up immediately.
• Records whose geo values are missing or malformed are dropped defensively and surfaced as a "N values without valid coordinates were skipped" note on the panel (you will see this for anycast IPs that GeoLite2 cannot place in a country).

Rendering: Leaflet over a world outline bundled with the app. No tile servers, no CDN assets, zero network requests: it works fully air-gapped and follows the app theme in both light and dark mode. Given the self-hosted audience I did not want dashboards phoning OpenStreetMap on every view.

Deferred to follow-ups, as discussed:

• Color by http_status / severity: needs metadata filters in the aggregation layer across all three engines, so it gets its own cycle.
• ISP/ASN: GeoLite2 City has no ASN data, needs a separate database.
• Per-IP tooltips / drill-down on dashboards: privacy and cost tradeoff, an aggregate map on a shared dashboard is a different exposure level than the incident detail view.

Thanks again for the well-scoped proposal, it made the design work noticeably easier. If you get a chance to try it on your Caddy/WAN setup once released, feedback on the two modes is very welcome.

[SemoTech]

Thanks for the detailed write-up — and especially for shipping geo_map on develop. That was faster than we expected, and the design choices look excellent for self-hosted / air-gapped use (bundled Leaflet outline, no OSM tile calls, flat keys for multi-engine group-by).

For the record, answers to your earlier design questions from our Caddy/WAN ops use case (we are recreating a Graylog-style “Caddy” dashboard: hit table + geo map + volume charts):

1. Flat / equirectangular only — yes. A globe is unnecessary at dashboard panel size. Your flat Leaflet world-outline approach matches what we want (same idea as Graylog’s Leaflet “Geo IP Locations” widget).

2. Both modes are useful.

   • By country for the always-on overview (cheap, safe).
   • By location (city / ~1 km) for WAN drill-down when we care about “who is hitting rmm.* from where.” Top-N cap is the right tradeoff. We do not need per-IP markers on a shared dashboard by default.

3. Color by status/severity: nice-to-have later. Fixed dimensions (http_status / level) would cover 90% of our edge monitoring; a free-form metadata field can wait. Companion Top-N / tables already cover status breakdown.

4. Privacy / IPs on dashboards: prefer aggregate by default (country / city place + hit counts). Individual source IPs belong in Explore / Details / incident views, not on a shared ops dashboard. Skipping per-IP tooltips on the map panel is the right call for us.

What we are aiming for once this hits a release: one project dashboard for reverse-proxy WAN traffic (similar to the Graylog layout) with:

• a log / hit table (host, client IP, status, geo columns — related to issue Feature: Dashboard panel — log hit table with configurable metadata columns #289),
• the new geo_map panel (country mode + location mode),
• time-series volume and Top-N by host / country.

We will try it on our self-hosted Caddy → Vector → LogTide pipeline as soon as it is in a tagged release (GeoIP pipeline already writing geo / flat keys on new traffic). Happy to report back on the two modes, skipped anycast/malformed rows, and how it feels next to Top-N for WAN monitoring.

Thanks again — the well-scoped proposal note is appreciated from this side too.

[Polliog]

Thanks for the detailed answers, they line up almost exactly with what shipped, which is reassuring: flat map only, both modes with aggregate as the default posture, no per-IP markers on shared dashboards, and color-by-status deferred with fixed dimensions as the likely shape when it comes.

On #289: it is tracked and it will also land in the next release. The log hit table is a good fit as the companion piece, and as you noted the metadata columns can reuse the same path resolution Log Search already has, since a table reads rows rather than aggregating them, so it does not hit the nested-path limitation the map had to work around. That means your Graylog-style Caddy dashboard (hit table + geo map + volume/Top-N) should be assemblable in one go once the release is out.

Looking forward to the feedback from the Caddy -> Vector -> LogTide setup, especially on how the skipped anycast rows read in practice.

[Polliog]

Update: 1.3.2 is out, and it closes the loop on the two follow-ups you filed after running the geo map for real (#304 and #305), plus there is now a proper docs page for dashboards.

Docs: https://logtide.dev/docs/dashboards/

It covers the full panel catalog, the time range presets and the board-wide picker, YAML export/import, the batch data API, and a Geo Map section that spells out the GeoIP pipeline prerequisite and the flat keys the step writes, which is the part that was previously only discoverable from this thread.

What shipped in 1.3.2

• Panel time ranges (Feature: Per-panel time range / timescale options on dashboards #305): every windowed panel now shares one list, 15m 1h 6h 12h 24h 48h 3d 7d 14d 30d. Your 48h case works everywhere, and the old per-panel enums are a subset of the new list, so existing dashboards and YAML exports keep working with no migration. Config forms label the control "Time range" and spell values out in full.
• Dashboard-wide time picker (Feature: Per-panel time range / timescale options on dashboards #305): retimes every windowed panel at once, which is what an ops review usually wants. It is view-level only, so per-panel settings and the saved board are untouched and "Panel defaults" restores each panel's own window.
• Geo map viewport (Bug: Geo map panel re-fits viewport (snaps/jumps) on every refresh #304): the map is no longer rebuilt on every poll, so pan and zoom survive background refreshes. Auto-fit now runs on first paint and when you change what the panel queries, with a Fit data control to re-frame on demand. Two things I found while verifying this and fixed as well: the mouse wheel over the panel used to scroll the page and zoom the map at the same time, which quietly destroyed your framing when scrolling past it, and the map could paint over open dialogs.

Two bugs worth knowing about, since they affected boards built before this release

While implementing the time ranges I found two panels that were silently ignoring their configured window:

• time_series always rendered the last 24 hours whatever interval you picked, and also dropped its service filter.
• top_n_table on the service dimension always aggregated the last 7 days.

Both are fixed. If your WAN board has a "Top services (24h)" panel or a multi-day volume chart, the numbers will change with this upgrade, because they are now actually the window you asked for. Time series also picks its bucket size from the window (one-minute buckets on a 1 hour view up to daily on 30 days) and zero-fills empty buckets, so quiet periods read as quiet instead of vanishing from the line.

Naming stayed as it is for now (interval on some panels, timeRange on others) since renaming stored fields would force a document migration for no functional gain, so the alignment is on the UI side. If that ever becomes awkward when hand-editing YAML, say so and I will reconsider it with a proper migration.

Thanks again for both reports. The viewport one in particular was the kind of thing that only shows up when a board is actually left open on a screen all day.

[SemoTech]

Thanks @Polliog — we upgraded to 1.3.2 and the Caddy WAN Geography board is working well with the new toolbar time picker (48h is great), stable geo map viewport, and Fit data. The dashboard docs page is helpful too.

Still on our side until native support lands: #308 (geo_label / geo_hit for Top Locations rows like City, USA → host.domain.com - agent). Happy to test whenever that ships.