Search analytics: System implementation

Defines a privacy-reduced search-event pipeline that reconstructs query sessions, preserves retrieval and ranking versions, and produces trustworthy relevance and failure diagnostics.

Package status: reference context ready for human review. The contract and test scenarios are complete, but no claim is made that an adopting implementation has passed them.

Decision

Emit immutable search events with stable query and result-set identities, then derive sessions and metrics downstream. Store enough versioned context to reproduce product behavior without logging raw sensitive queries by default.

Scope

• Event envelope, query lifecycle, result impressions, interactions, reformulations, zero results, errors, consent, retention, and data quality.
• Join rules between server decisions and client-visible interactions.
• Operational and relevance datasets with explicit late-event handling.

Outside this block

• Choosing business KPIs or treating clicks as ground truth.
• General-purpose product analytics unrelated to search.

Contract

• A query event has an event ID, pseudonymous session ID, query ID, event time, receive time, surface, normalized query fingerprint, filter summary, and profile versions.
• A result impression references one immutable ordered result-set ID; interactions reference both query and item position.
• Raw query text is excluded unless an approved purpose, access policy, consent or other legal basis, and retention rule exist.
• Client retries preserve event ID, and ingestion de-duplicates by producer plus event ID.
• Late events are accepted within a declared window and dashboards disclose data completeness.
• Bot, test, staff, and unsupported traffic classifications remain fields rather than destructive deletion from the raw governed stream.

Implementation guidance

• Use a vendor-neutral event envelope and version the payload schema independently.
• Create a server-side search-decision event before rendering and a client impression only when results become viewable.
• Resolve sessions using an inactivity rule in a derived model so the raw event remains unchanged.
• Enforce tenant and data-class access in warehouse models as well as ingestion.

Failure handling and safeguards

• Quarantine schema-invalid events with reason and redacted headers; do not partially coerce identifiers.
• If client and server clocks diverge, retain both times and order using declared rules.
• If consent state is unavailable, apply the conservative collection profile rather than inferring consent.

Verification and operations

• Measure ingestion lag, duplicate rate, schema rejection, unjoined interactions, late events, consent coverage, and event-volume discontinuities.
• Reconcile query counts against service logs using privacy-safe aggregates.
• Publish metric definitions with exclusions, windows, and data-freshness indicators.

The executable-looking examples in this package are fixtures and acceptance contracts. Run python tools/validate.py from the collection root to check package structure and metadata; then implement and execute the scenarios in the target repository.

Adoption assumptions

• Names and numeric values in example.yaml are an adoption profile, not universal defaults.
• The adopting team must map actors, data classes, error vocabulary, and ownership to its system.

References

• cloudevents