Encoding and serialisation

Validate incoming data before using it. Check its size, encoding, structure and values, including data from devices on a private network.

Overview#

Treat every byte from a device, broker, SDK, file, webhook, or discovery response as untrusted, even on a private network. Physical security devices are long lived, heterogeneous, and frequently expose legacy parsers.

Processing pipeline#

bounded read -> frame detection -> integrity check -> structural parse
             -> schema validation -> semantic validation -> authorization
             -> state change or storage

Set limits before allocation: total message size, nesting depth, collection count, field/string length, decompressed size, number of XML entities, attachment count, and processing time. Reject trailing or concatenated data unless the framing explicitly permits it.

Binary protocols#

Document byte order, bit numbering, signedness, length unit, escape rules, checksum coverage, alignment, and counter wrap. Before reading a field:

1. prove the minimum remaining length;
2. validate declared lengths against frame and configured maximum;
3. use checked arithmetic for offsets and lengths;
4. reject impossible enum/reserved combinations as specified;
5. distinguish incomplete stream data from malformed frames.

A checksum detects some transmission errors; it isn't cryptographic integrity or peer authentication.

JSON#

RFC 8259 defines JSON syntax and interoperability considerations. Applications must still decide:

• duplicate member names;
• integer/float precision and range;
• unknown fields and enum values;
• Unicode normalisation and display safety;
• absent versus null;
• canonicalization if signing or hashing;
• schema/version negotiation.

Reject non finite numbers unless explicitly defined by another encoding. Avoid round tripping large credential identifiers through a number type that loses precision.

XML and SOAP#

Use namespace aware parsing, a local schema set for the applicable edition, and hardened parser defaults. Disable external entity resolution and network retrieval unless a narrowly controlled use case requires it. Cap entity expansion and document size. Don't select elements by local name alone when namespaces distinguish semantics.

The authoritative XML specifications are maintained by the W3C. Protocols such as ONVIF also publish their own schemas and Web Services Description Language (WSDL); validate against the version actually negotiated or documented.

Compact and schema driven encodings#

CBOR (RFC 8949), Protocol Buffers, and vendor binary SDKs can reduce size but still require bounds, schema versioning, unknown field policy, and canonicalization rules. “Generated parser” doesn't remove semantic validation.

Logging and evidence#

Never log credentials, authorisation headers, cookies, private keys, biometric templates, card secrets, full access tokens, or unnecessary personal data. Store a bounded sanitised excerpt, message type, sizes, parse outcome, source identity, trace ID, and integrity hash/reference when raw evidence belongs in a separately controlled store.

Schema evolution#

Prefer additive compatible change. Producers shouldn't reuse a field with new meaning; consumers should define unknown field and unknown enum behaviour; both should expose schema/protocol version. A mapper version belongs in provenance so historical data can be interpreted after corrections.

Section overview · Wiki home