HTTP and REST API engineering

HTTP provides request and response behaviour for web APIs. Define the resources, permissions and physical effects associated with each operation.

Overview#

HTTP supplies message semantics and versioned transports. REST is an architectural style, not a separate wire protocol, schema language, authentication method, or guarantee of good resource design. A physical security API should document both the HTTP contract and the domain contract. HTTP FIELDING

Current protocol bases#

HTTP versions preserve common semantics but differ materially in framing, connection, prioritization, flow control, and failure behaviour. Gateways must translate without creating ambiguous lengths, pseudo header errors, authority confusion, or inconsistent normalisation. HTTP1 HTTP2 HTTP3

Resource contract#

A durable API defines nouns and state transitions before routes:

Don't model high impact commands as harmless field updates without an explicit audit and completion model.

Methods, safety, and idempotency#

Per RFC 9110, safe methods are read only by defined semantics; idempotent methods can be repeated with the same intended effect. Server logging or accounting doesn't make a safe method unsafe, but a GET endpoint that unlocks a door violates method semantics. HTTP

An “idempotency key” is an application contract: define key scope, authenticated principal, target, canonical request fingerprint, retention window, concurrent duplicate behaviour, response replay, and conflict result. It can't make two semantically different requests equivalent.

Concurrency and conditional requests#

Use validators such as ETags with If-Match for changes and If-None-Match for cache validation or create if absent behaviour. Strong versus weak validators have different comparison rules. A typical optimistic update is:

1. GET /credentials/{id} and record the strong ETag.
2. Validate intended transition locally.
3. PUT or PATCH with If-Match: "recorded-tag".
4. Treat 412 Precondition Failed as a concurrency conflict, refetch, and reconsider, don't overwrite blindly.

Read only conditional request fixture#

This synthetic protocol fragment illustrates cache validation for a read. api.example, the credential ID, token text, and ETag are documentation values; the fragment contains no client and shouldn't be adapted into a state changing credential operation.

GET /v1/credentials/cred_demo_001 HTTP/1.1
Host: api.example
Authorization: Bearer replace-with-short-lived-test-token
Accept: application/json
If-None-Match: "demo-etag-7"

The API must authorise the read for the exact tenant/resource. A matching current validator can produce 304 Not Modified; otherwise the server can return the authorised representation and a current validator. Even a safe method needs request, response, rate, cache, privacy, and log bounds.

Status and error model#

Use status codes consistently, then return a stable machine readable error shape. RFC 9457 defines Problem Details for HTTP APIs. PROBLEM

For asynchronous physical actions, return a job/operation resource with correlation ID, state, timestamps, result source, and expiry. Distinguish “controller acknowledged” from “door sensor confirmed.”

Pagination, filtering, and time#

• Prefer opaque cursors over mutable offset pagination for event streams.
• Bind a cursor to tenant, principal, query filters, sort order, snapshot/watermark, and expiry.
• Define inclusive/exclusive timestamp boundaries, ordering tie breakers, late events, and clock uncertainty.
• Cap page size, query complexity, expanded relationships, date range, and export volume.
• Allowlist filter/sort fields; parameterized storage queries don't replace authorisation.

Authentication and authorisation#

• Use HTTPS with certificate validation. Mutual TLS can provide client/workload identity but still needs resource authorisation.
• For OAuth 2.0 deployments follow RFC 9700, the current OAuth 2.0 Security Best Current Practice; use flows appropriate to public/confidential clients, sender constraint where needed, exact redirect validation, PKCE, scoped/audience bound short lived tokens, and refresh token controls. OAUTH BCP
• Validate token issuer, audience, signature/algorithm policy, time bounds, token type, client/subject, and required scope/claims. Don't accept a token merely because it parses.
• Authorise object and action together; prevent cross tenant identifier substitution.
• Separate read video/events, enrol credentials, unlock/override, administer identities, export evidence, and manage devices.
• Recheck authorisation for long running exports, streams, and jobs.

Browser sessions also require CSRF, CORS, origin, cookie, SameSite, and content security design. CORS is a browser read control mechanism, not API authentication.

Browser based OAuth clients#

RFC 10017 is the current IETF Best Current Practice for OAuth 2.0 browser based applications. It evaluates architectural patterns and browser specific threats; apply it together with RFC 9700 and the authorisation server’s supported profile. BROWSER BCP

• A browser application is a public client and can't keep a distributed static client secret confidential.
• Use Authorisation Code flow with PKCE and exact redirect URI registration. Don't use the implicit grant for new applications.
• Bind state to the initiating browser session and intended return location; use OIDC nonce where the OIDC profile requires it.
• Prefer an architecture that keeps access/refresh tokens out of JavaScript, such as a backend for frontend, when consequence and deployment constraints support it.
• If tokens are exposed to browser code, keep their scope and lifetime minimal and account explicitly for XSS, malicious dependencies/extensions, storage, refresh, and multi tab/session behaviour.
• A backend for frontend session needs secure, HttpOnly cookies, CSRF protection, origin checks, session rotation, logout, and server side authorisation. Moving a token behind a cookie doesn't remove browser threats.
• Restrict third party scripts, use an effective Content Security Policy, avoid sensitive tokens/codes in URLs and telemetry, and prevent open redirects.
• Treat refresh token rotation or sender constraint as additional defenses under the chosen profile, not substitutes for preventing script compromise.
• Keep browser authentication/session success separate from authorisation for tenant, object, operation, and high impact physical actions.

Cross device authorisation/authentication flows have additional transaction binding and phishing concerns; see WebAuthn, FIDO, and passkeys.

Message integrity and webhooks#

RFC 9421 defines HTTP Message Signatures, but an application profile must still specify key discovery, algorithm policy, created/expiry time, nonce/replay behaviour, and the exact covered components. Always cover authority, method, target, content digest where applicable, and security critical fields according to the profile. A valid signature doesn't authorise the requested operation. HTTP SIG

Parser and intermediary safety#

• Reject ambiguous HTTP/1.1 framing and conflicting length/transfer metadata; keep proxy/backend parsing aligned.
• Normalise a path once under a documented URI policy; avoid decoding or case rules that differ across authorisation and routing layers.
• Validate Host/authority against the selected tenant and endpoint.
• Bound header fields, trailers, body, decompressed body, multipart parts, streams, concurrent requests, and HPACK/QPACK state.
• Don't send untrusted post transition protocol bytes optimistically on HTTP/1.1. RFC 9931, published March 2026, adds requirements to prevent request smuggling/parser risks during rejected Upgrade or CONNECT transitions. HTTP TRANSITION
• Treat redirects as new authorisation/egress decisions; never forward credentials across origin by default.

Caching and sensitive data#

Explicitly set cache policy. Credential records, live images, access events, configuration, and exports usually require private/no store semantics tailored to the application. Ensure shared cache keys include all representation varying dimensions and never depend on Authorization behaviour implicitly. Signed or expiring URLs need cache lifetimes no longer than authorisation.

Client reliability contract#

• Set connect, TLS, header, body idle, and total deadlines.
• Cancel abandoned requests and bound response reads.
• Retry only documented transient failures, with exponential backoff, jitter, retry budget, and Retry-After handling.
• Never retry a mutating request solely because no response arrived; its effect may have occurred.
• Use circuit breaking/load shedding without converting an access outage into an unsafe fail open.
• Preserve a correlation ID but don't let clients choose trusted audit identity fields.

Review checklist#

• Exact HTTP versions, TLS policy, ALPN, proxy chain, and timeout behaviour documented
• Resource identities, lifecycle states, command completion, and error model defined
• Safe/idempotent method semantics respected
• ETag/precondition and duplicate request behaviour specified
• Authentication plus per object/per action authorisation enforced
• OAuth/token validation profile and credential rotation documented where used
• Browser clients follow RFC 10017 architecture, PKCE, redirect, cookie/session, CSRF, XSS, and token storage guidance
• Parser, body, decompression, pagination, export, and concurrency limits enforced
• Request smuggling differentials tested manually by the system owner or qualified assessor
• Sensitive caching, logs, traces, URLs, and error fields reviewed

Sources#

• HTTP, RFC 9110 / STD 97: HTTP Semantics, IETF, June 2022.
• HTTP CACHE, RFC 9111: HTTP Caching, IETF, June 2022.
• HTTP1, RFC 9112 / STD 99: HTTP/1.1, IETF, June 2022, updated by RFC 9931.
• HTTP2, RFC 9113: HTTP/2, IETF, June 2022.
• HTTP3, RFC 9114: HTTP/3, IETF, June 2022.
• HTTP TRANSITION, RFC 9931: Security Considerations for Optimistic Protocol Transitions in HTTP/1.1, IETF, March 2026.
• TLS13, RFC 8446: TLS 1.3, IETF, August 2018.
• OAUTH BCP, RFC 9700 / BCP 240: Best Current Practice for OAuth 2.0 Security, IETF, January 2025.
• BROWSER BCP, RFC 10017: OAuth 2.0 for Browser Based Applications, IETF.
• HTTP SIG, RFC 9421: HTTP Message Signatures, IETF, February 2024.
• PROBLEM, RFC 9457: Problem Details for HTTP APIs, IETF, July 2023.
• FIELDING, Architectural Styles and the Design of Network based Software Architectures, Chapter 5, Roy Fielding, 2000.

Section overview · Wiki home