Redacting Coordinate Precision in Application Logs

← Back to Audit Logging for Location Data Access

This page shows how to strip identifying precision from coordinates in log output without losing the ability to debug, by truncating at the logging formatter and keeping a correlation id that points back into the access-controlled audit trail.

# Context & When to Use

The database that holds location data usually has careful access control: roles, row-level security, an audit trail. The log pipeline that sits beside it usually does not. Logs are shipped to an aggregator that half the engineering organisation can search, retained for a year, replicated to a backup region, and occasionally exported to a spreadsheet during an incident. A coordinate at six decimal places in a log line has effectively left the security boundary the database spent so much effort establishing.

This is not hypothetical leakage through some exotic channel. It happens through the most ordinary paths: a debug line that prints the request parameters, an exception message that includes the SQL that failed, an access log that records the full query string, a trace span that captures the URL. None of those were written by someone deciding to log a location; the location arrived as a side effect.

The fix is to make coarsening structural. Apply it in the logging configuration where it covers every record, including ones emitted by libraries, and pair it with a request id so an authorised investigator can still recover the precise envelope from the audit table described in Audit Logging for Location Data Access.

# Runnable Implementation

import logging
import re
from typing import Any

# Matches a decimal degree with more than three fractional digits, in any
# surrounding text: query strings, WKT, JSON, exception messages.
COORD_RE = re.compile(r"(-?(?:1[0-7]\d|\d{1,2})\.\d{3})\d+")
REDACTED_DP = 3   # ~110 m at the equator


def coarsen(text: str) -> str:
    """Truncate every decimal degree in a string to REDACTED_DP places."""
    return COORD_RE.sub(r"\1", text)


class CoarsenCoordinates(logging.Filter):
    """Apply coarsening to the message, its args, and any exception text.

    Installed as a filter rather than a formatter so it runs for records from
    third-party libraries too — which is where the accidental leaks live.
    """

    def filter(self, record: logging.LogRecord) -> bool:
        if isinstance(record.msg, str):
            record.msg = coarsen(record.msg)

        if record.args:
            if isinstance(record.args, dict):
                record.args = {k: coarsen(v) if isinstance(v, str) else v
                               for k, v in record.args.items()}
            else:
                record.args = tuple(coarsen(a) if isinstance(a, str) else a
                                    for a in record.args)

        # Exception text is the most common accidental carrier
        if record.exc_info and record.exc_info[1]:
            exc = record.exc_info[1]
            if exc.args and isinstance(exc.args[0], str):
                exc.args = (coarsen(exc.args[0]),) + exc.args[1:]
        return True


LOGGING: dict[str, Any] = {
    "version": 1,
    "disable_existing_loggers": False,
    "filters": {"coarsen_coords": {"()": CoarsenCoordinates}},
    "formatters": {
        "json": {"format": '{"t":"%(asctime)s","lvl":"%(levelname)s",'
                           '"req":"%(request_id)s","msg":"%(message)s"}'},
    },
    "handlers": {
        "stdout": {
            "class": "logging.StreamHandler",
            "formatter": "json",
            # The filter goes on the HANDLER so every logger inherits it
            "filters": ["coarsen_coords"],
        },
    },
    "root": {"handlers": ["stdout"], "level": "INFO"},
}

Attaching the filter to the handler rather than to individual loggers is the detail that makes it comprehensive: every record that reaches stdout passes through it, whatever emitted it.

# Key Parameters & Options

That last row matters if any part of the stack speaks a national grid. A British National Grid easting like 530034.271 is not a decimal degree and passes the filter untouched, while being just as identifying — add a second pattern for the projected systems your API accepts, using the ranges from Handling Mixed SRID Inputs from Legacy Clients.

# What each level of redaction still leaks

Three places is the knee of the curve: risk falls off sharply between four and three, while debugging value barely moves until coordinates disappear altogether.

# Gotchas & Failure Modes

• Structured logging that bypasses the message. If coordinates are passed as structured fields rather than inside the message string, a filter that only rewrites record.msg misses them. Extend the filter to walk record.__dict__ for known field names, or normalise all logging through one helper.
• The regex matching version numbers. An unbounded \d+\.\d+ pattern will happily truncate PostGIS 3.3.4 and timing values like 1247.891. Bounding the integer part to plausible degree ranges, as above, avoids most of it; test against a corpus of real log lines.
• Coordinates arriving base64-encoded. A cursor token or a WKB hex string carries a location the regex cannot see. Redact those by field name rather than by pattern — see Implementing Cursor-Based Pagination for Spatial Queries for what a cursor typically contains.
• Redaction applied only in production. A developer copying a staging log into a ticket leaks the same data. Apply the filter in every environment; a debugging session that needs full precision should query the audit table.
• Losing the request id. Coarsening without a correlation id makes logs both private and useless. The id is what preserves the investigative path.
• Assuming coarsening is anonymisation. A sequence of 110-metre points still traces a route. Coarsening limits blast radius; it does not make the data non-personal.

# Keeping the investigative path open

Redaction is only acceptable because there is somewhere else to look. The request id printed on every log line is what turns a coarse log entry back into a precise answer, for the small number of people authorised to ask.

The workflow in practice: an engineer sees an error in the aggregator, notes the request id, and — if the investigation genuinely needs the exact area — an authorised colleague queries the audit table for that id. The engineer gets the diagnosis; the precise envelope never leaves the database. That split is the whole point, and it fails only if the id is dropped somewhere along the chain.

# Verification Snippet

import logging

def test_filter_coarsens_every_carrier(caplog):
    logging.getLogger().addFilter(CoarsenCoordinates())

    logging.info("bbox=-0.127761,51.507351,-0.127700,51.507400")
    logging.info("query %s", "POINT(-0.127761 51.507351)")
    try:
        raise ValueError("no feature at -0.127761, 51.507351")
    except ValueError:
        logging.exception("lookup failed")

    text = caplog.text
    assert "-0.127761" not in text
    assert "-0.127" in text          # coarse value survives
    assert "51.507" in text
    assert "PostGIS 3.3.4" == coarsen("PostGIS 3.3.4")   # version untouched

# Belt and braces: scan shipped logs for anything with 4+ decimal places
grep -REn '(-?[0-9]{1,3}\.[0-9]{4,})' /var/log/api/*.log | head
# (no output expected)

# Related

• Audit Logging for Location Data Access — where full precision is kept, under access control
• JWT Authentication for Spatial Scopes — the subject identifier that accompanies the request id
• Observability for Spatial Endpoints — trace attributes have the same exposure question

← Back to Audit Logging for Location Data Access