feat: add the redisvl[otel] observability foundation (FDE-86)

[vishal-bala]

Motivation

SemanticCache is gaining OpenTelemetry metrics and spans, so that applications can see hit rates, match distances against the threshold, and where a lookup spends its time. This change adds the plumbing that instrumentation builds on, following OpenTelemetry's guidance for libraries: depend on the API only, leave the SDK and exporters to the application, and cost next to nothing when no SDK is configured. It emits no telemetry itself.

Changes

The redisvl[otel] extra and redisvl.utils.otel

The otel extra depends on opentelemetry-api>=1.30.0, the first release that accepts the histogram bucket advice the cache metrics will set. The internal redisvl.utils.otel module returns a tracer and a meter from the global providers, under the redisvl instrumentation scope with RedisVL's version and the semantic-conventions 1.44.0 schema URL. With no SDK configured they are the API's own no-ops, at about 2 µs for one span and two histogram records on an Apple M4 Pro.

When the API is not installed, or REDISVL_OTEL_DISABLED is true, the module returns stand-ins with the same interface, so instrumented code never branches. The metric and attribute names the instrumentation will use are defined in the same module. RedisVL never constructs a provider and never initialises redis-py's own observability; both belong to the application.

Tests and CI

The SDK the tests need sits in a non-default otel-test dependency group, so a plain uv sync stays free of OpenTelemetry and the existing import check keeps running without optional dependencies. A shared otel_sdk fixture configures in-memory exporters and resets the set-once global providers on teardown. Three tests cover the instrumentation scope with an SDK, the off switch with an SDK configured, and the path with no SDK in a fresh interpreter: the global providers stay the API's proxies, redis-py's observability is not initialised, and the per-call cost is bounded.

A new CI leg runs the default suite without OpenTelemetry. Every other leg has it, because mistralai brings in the SDK and fastmcp the API, so the leg installs every extra except those two and skips the MCP tests, which need fastmcp.

Minor

• make install and the test matrix sync the otel-test group.

Notes

This PR sits in a stack on top of the KNN lookup change. On its own it adds an extra that emits nothing, so it should merge together with the instrumentation above it rather than alone.

Next Steps

1. Record SemanticCache operation-duration and match-distance histograms.
2. Emit spans for cache operations and embeddings.
3. Document the feature, with an example and a starter dashboard.

---

Note

Low Risk
Adds an optional extra and internal helpers with no-op defaults; no production instrumentation or runtime behavior change until follow-up PRs wire it in.

Overview
Adds optional OpenTelemetry support via redisvl[otel] (opentelemetry-api>=1.30.0, also in all) and internal redisvl.utils.otel, which exposes cached get_tracer() / get_meter() under the redisvl scope, shared metric/attribute name constants, REDISVL_OTEL_DISABLED, and no-op stand-ins when the API is missing or disabled so callers never branch. The library does not install an SDK or touch redis-py observability.

otel-test is a non-default uv group with opentelemetry-sdk for tests; make install and the main CI matrix sync --group otel-test. A new Tests without OpenTelemetry job installs extras without mistralai/mcp (which pull OTEL transitively), asserts opentelemetry is not importable, and runs the suite with MCP tests ignored.

Adds an otel_sdk pytest fixture (in-memory exporters + provider reset) and unit tests for scope, the off switch, and API-only no-SDK behavior with a microsecond overhead bound.

^{Reviewed by Cursor Bugbot for commit 5d9a2d5. Bugbot is set up for automated code reviews on this repo. Configure here.}