# Precision Mapping & Rounding Strategies

A vibration-monitoring fleet reports rms_velocity at one sample per second. The hourly rollup averages those 3,600 readings and rounds the result to four decimals before writing it to the long-term bucket. For eleven months the dashboards look fine — until a reliability engineer sums the daily energy counters for a compliance report and finds the total is 0.7% higher than the raw stream says it should be. Nothing failed. No task errored. The gap is pure rounding bias: every window used math.round() (which rounds halves away from zero), so across tens of millions of windows the systematic upward push on .5 cases accumulated into a real, auditable discrepancy. This is the failure that precision mapping exists to prevent — the silent, compounding divergence between what a sensor measured and what the downsampled series claims it measured.

Precision mapping is the explicit contract that says, for each measurement, exactly how many significant digits survive aggregation and which rounding rule resolves the boundary cases. It is a specialization of the broader downsampling and aggregation pipeline problem: the pipeline decides when and what to aggregate; precision mapping decides how the numeric result is quantized so that the rollup stays mathematically faithful to the raw stream, run after run, tier after tier.

# Problem framing: where precision quietly breaks

Three distinct failures hide under the umbrella of “rounding”:

• Accumulated bias. A directional rounding rule (half-up, half-down, ceil, floor) applied to millions of aggregate results pushes long-horizon sums and integrals in one direction. On cumulative counters — energy, throughput, packet totals — the error is not noise that averages out; it is a drift that grows with sample count.
• Representation error. IEEE 754 float64 cannot store 12.3455 exactly. Rounding a binary approximation with a decimal rule produces results that disagree with a base-10 calculation, and the disagreement changes depending on the order of operations inside aggregateWindow.
• Cross-tier inconsistency. When the raw stream, the 1h rollup, and the 1d rollup each round independently — or round to different decimal places — an alert threshold evaluated against one tier fires at a different value than the same threshold against another, producing phantom or missing alerts.

The fix is not “round more carefully in one place.” It is to treat the rounding rule and decimal precision as a versioned pipeline parameter, apply it identically everywhere the series is materialized, and continuously validate that raw and downsampled aggregates still agree within a declared tolerance.

# Prerequisites

Confirm the environment before wiring precision logic into a scheduled task:

• InfluxDB v2.7+ (OSS or Cloud) — the option task semantics and the math standard library behavior described here are v2-native.
• Flux v0.194+ — for the math package functions (math.floor, math.mod, math.round) used below.
• A source bucket for raw writes (this page uses raw_telemetry) and a destination rollup bucket (downsampled_telemetry).
• An API token with read on the source bucket and write on the destination.
• For offline validation: Python 3.9+ with pandas v1.4+ and numpy — the decimal module ships with the standard library.
• A written precision contract per measurement: the rounding mode and the number of decimal places (or significant figures) each series will carry.

The rollup bucket must exist before the task references it — a missing destination is a per-run hard failure, not a warning.

# Core concept: rounding modes and quantization bias

Rounding to d decimal places is a two-step operation: scale by the quantization factor, resolve to an integer with a chosen rule, then unscale. The quantization step is:

$$ q = 10^{-d}, \qquad \operatorname{round}_d(x) = \frac{\operatorname{round}(x \cdot 10^{d})}{10^{d}} $$

The choice of the inner integer rule is what determines whether error cancels or accumulates. Over N windows whose fractional parts land on exactly .5, a directional (half-up) rule contributes a cumulative bias of roughly:

$$ E_{\text{half-up}} \approx N \cdot \tfrac{1}{2} q, \qquad E_{\text{half-even}} \approx 0 $$

Half-even (banker’s rounding) sends .5 to the nearest even integer, so ups and downs are equally likely and the expected bias is zero — which is exactly why it is the correct default for cumulative counters and any series that will be summed or integrated over long horizons. The trade-off table below matches each rule to the telemetry it suits:

The subtlety in Flux: math.round() implements round-half-away-from-zero, not half-even. To get bias-free rounding you must build half-even from math.floor and math.mod. The rest of this page does exactly that and then proves it stayed faithful.

# Step-by-step implementation

# Step 1 — Fix the precision contract

Decide, per measurement, the decimal places and the rounding rule, and record the decision as data you can query later. For industrial_vibration.rms_velocity we choose four decimals, half-even, because the field is averaged and later summed into energy estimates. Carrying the mode as a tag (precision_mode) makes the contract self-documenting inside the rollup bucket and lets downstream consumers detect a change in policy.

# Step 2 — Implement half-even in Flux

Flux has no built-in banker’s rounding, so derive it. A block-body function ends with return; if/else is an expression (no end terminator); and % is not valid on floats, so use math.mod.

import "math"

// Round x to the nearest integer using half-to-even (banker's) rounding.
round_half_even = (x) => {
    f = math.floor(x: x)
    diff = x - f
    result =
        if diff < 0.5 then f
        else if diff > 0.5 then f + 1.0
        else if math.mod(x: f, y: 2.0) == 0.0 then f
        else f + 1.0
    return result
}

The diff == 0.5 branch is the whole point: only when the value sits exactly on the boundary does the even-integer tie-break apply, and that is where math.round() would have injected directional bias.

# Step 3 — Apply the rule inside the aggregation task

Aggregate first, then quantize with scale → round → unscale, then tag the contract and route to the destination. Anchoring range() to task.every keeps each run processing exactly one window, matching the idempotent scheduling discipline covered in Flux scripting for task automation.

import "math"

option task = {name: "sensor_precision_downsample", every: 15m, offset: 1m}

round_half_even = (x) => {
    f = math.floor(x: x)
    diff = x - f
    result =
        if diff < 0.5 then f
        else if diff > 0.5 then f + 1.0
        else if math.mod(x: f, y: 2.0) == 0.0 then f
        else f + 1.0
    return result
}

from(bucket: "raw_telemetry")
    |> range(start: -task.every)
    |> filter(fn: (r) => r._measurement == "industrial_vibration")
    |> filter(fn: (r) => r._field == "rms_velocity")
    |> aggregateWindow(every: 15m, fn: mean, createEmpty: false)
    // Round to 4 decimals: scale, apply half-even, then unscale.
    |> map(fn: (r) => ({ r with _value: round_half_even(x: r._value * 10000.0) / 10000.0 }))
    |> set(key: "precision_mode", value: "half_even_4dp")
    |> to(bucket: "downsampled_telemetry", org: "production_ops")

Two callouts. Rounding inside map() before to() means the destination stores already-quantized values, so every Grafana panel and API consumer reads an identical number — no client re-rounds differently. And matching aggregateWindow(every: 15m) to the task’s every: 15m guarantees one point per series per run; splitting the aggregation finer would straddle the window boundary and reintroduce partial-window artifacts. Dynamic decimal-place selection driven by rolling variance — carrying more digits for quiet signals, fewer for noisy ones — is developed in optimizing aggregation precision for high-frequency sensor data.

# Step 4 — Keep the rule consistent across tiers

When a second tier rolls the 15m series up to 1h, it must apply the same round_half_even at the same four decimals. Re-rounding an already-rounded value to the same precision is a no-op, so a consistent contract makes the tiers idempotent with respect to each other. Divergent decimal places between tiers is the root of the cross-tier alerting inconsistency described earlier; the threshold implications are handled in threshold tuning for aggregation.

# Configuration reference

# Common failure modes and fixes

1. math.round() used where half-even was needed (cumulative drift). Symptom: long-horizon sums of a rounded series exceed the raw sum by a small, steadily growing percentage. Root cause: math.round() rounds halves away from zero, so .5 cases are pushed one-directionally across millions of windows. Fix: replace it with the round_half_even function from Step 2 on any series that is summed or integrated.

2. Rounding before aggregating (variance destruction). Symptom: the mean of a rounded raw stream differs from the rounded mean, and low-amplitude signal detail vanishes. Root cause: quantizing each raw point before mean discards sub-quantum information the average would otherwise have preserved. Fix: always aggregate first, then round the aggregate — quantize the result, never the inputs.

// Wrong: |> map(fn:(r)=>({r with _value: round_half_even(x:r._value*10000.0)/10000.0}))
//        |> aggregateWindow(every: 15m, fn: mean, createEmpty: false)
// Right: aggregate, THEN round.
|> aggregateWindow(every: 15m, fn: mean, createEmpty: false)
|> map(fn: (r) => ({ r with _value: round_half_even(x: r._value * 10000.0) / 10000.0 }))

3. Silent float64 → int demotion. Symptom: the destination field’s type flips to integer and later writes are rejected with a field-type conflict, or decimals are truncated. Root cause: an unguarded int() cast or a mixed integer/float branch in a custom function coerces the column type. Fix: keep all arithmetic in floats (note the 2.0, 10000.0, + 1.0 literals in round_half_even) so the field type stays float end to end.

4. Inconsistent decimal places across tiers (phantom alerts). Symptom: the same threshold fires against the 1h rollup but not the 1d rollup for the same underlying value. Root cause: tiers were built at different precisions, so the quantized values disagree at the last digit. Fix: apply one contract (half_even_4dp) to every tier; verify with the drift check below.

5. Offset too small for late data (windows quantized before packets land). Symptom: rounded aggregates omit points that exist in raw_telemetry seconds after the boundary. Root cause: offset closed the window before the edge gateway flushed its batch, so the mean — and its rounded value — is computed on incomplete data. Fix: set offset above the observed p99 delivery lag, the same tuning discussed under continuous query migration to tasks.

# Verification and testing

Validate raw-vs-rollup drift offline. Compare an independently re-aggregated reference against the pipeline’s output using decimal arithmetic, which bypasses float64 representation error before computing statistics.

import pandas as pd
from decimal import Decimal, ROUND_HALF_EVEN

def validate_precision_drift(raw_series: pd.Series, downsampled_series: pd.Series, epsilon: float = 1e-6) -> dict:
    """
    Compare two index-aligned series (an independently re-aggregated reference
    vs. the pipeline's downsampled output, equal length) to detect drift.
    Decimal arithmetic sidesteps float64 error; results cast to float for stats.
    """
    q = Decimal("0.0001")
    raw_dec = raw_series.apply(lambda x: Decimal(str(x)).quantize(q, rounding=ROUND_HALF_EVEN))
    down_dec = downsampled_series.apply(lambda x: Decimal(str(x)).quantize(q, rounding=ROUND_HALF_EVEN))

    diff = (raw_dec - down_dec).abs().astype(float)
    drift = diff > epsilon
    return {
        "max_drift": float(diff.max()),
        "mean_drift": float(diff.mean()),
        "drift_points_pct": float(drift.mean() * 100),
        "validation_passed": not bool(drift.any()),
    }

raw = pd.Series([12.3455, 12.3445, 12.3465, 12.3435])
downsampled = pd.Series([12.346, 12.344, 12.346, 12.344])
print(validate_precision_drift(raw, downsampled))

Add an in-database drift deadman. A rollup that silently diverges is worse than one that errors. A companion task re-aggregates the raw window, compares it to the stored rollup, and writes to an alert bucket when the absolute difference exceeds the tolerance:

import "math"

option task = {name: "precision_drift_check", every: 1h, offset: 5m}

reference = from(bucket: "raw_telemetry")
    |> range(start: -task.every)
    |> filter(fn: (r) => r._measurement == "industrial_vibration" and r._field == "rms_velocity")
    |> aggregateWindow(every: 15m, fn: mean, createEmpty: false)

stored = from(bucket: "downsampled_telemetry")
    |> range(start: -task.every)
    |> filter(fn: (r) => r._measurement == "industrial_vibration" and r._field == "rms_velocity")

join(tables: {ref: reference, out: stored}, on: ["_time"])
    |> map(fn: (r) => ({ r with drift: math.abs(x: r._value_ref - r._value_out) }))
    |> filter(fn: (r) => r.drift > 0.0001)
    |> to(bucket: "_alerts")

If that query writes rows, a tier has diverged from the raw truth — wire _alerts to a notification hook. The math package reference for the functions used here is in the official Flux math documentation.

# Integration points

Precision mapping sits between the scheduler and the storage substrate. The scheduling cadence and window semantics that wrap the quantization task belong to Flux scripting for task automation; the bucket topology and per-tier retention that determine how long each quantized series lives are owned by bucket architecture and tiering boundaries and retention policy design. Downstream, the rounded values feed alert evaluation — where mismatched precision between raw and rollup silently distorts firing rates — covered in threshold tuning for aggregation. When a window has no data to round because packets never arrived, the recovery behavior is the concern of fallback chains for missing data.

# Frequently Asked Questions

# Why not just use math.round() in Flux?

math.round() rounds halves away from zero, which is a directional rule. On any series you later sum or integrate — energy counters, throughput totals — that bias accumulates across millions of windows into a measurable, one-sided error. Build half-even from math.floor and math.mod for bias-free rounding on summed data; math.round() is only safe where a small one-sided bias is genuinely acceptable.

# Should I round before or after aggregating?

Always after. Rounding raw points before aggregateWindow discards sub-quantum detail that the aggregate would have used, changing the computed mean. Aggregate at full precision, then quantize the single result per window.

# How many decimal places should I keep?

Keep only the digits that exceed your sensor’s noise floor. Carrying digits below the ADC resolution stores quantization noise as if it were signal and inflates storage without adding information. Match the decimal places to the signal-to-noise ratio, and consider variance-driven dynamic precision for wide-dynamic-range sensors.

# Will rounding change my field’s data type?

It can, if a cast or a mixed integer/float branch sneaks in. Keep every literal a float (10000.0, 2.0, + 1.0) so the _value column stays float64 end to end; a demotion to int causes field-type conflicts on later writes.

# How do I keep multiple rollup tiers consistent?

Apply one precision contract — the same rounding rule and the same decimal places — to every tier, and tag each series with precision_mode. Re-rounding an already-quantized value at the same precision is a no-op, so consistent tiers stay idempotent; the drift deadman confirms they have not diverged.

# Related

• Threshold Tuning for Aggregation — aligning alert boundaries with the precision of rounded rollups.
• Continuous Query Migration to Tasks — porting legacy rollups where implicit coercion hid precision loss.
• Fallback Chains for Missing Data — what a rounding task does when a window has nothing to quantize.
• Optimizing Aggregation Precision for High-Frequency Sensor Data — variance-driven dynamic decimal precision.

Up: Downsampling & Aggregation Pipeline Design