OptiServe

Jointly optimize cost, latency and accuracy in serverless ML workflows.
Profile each AWS Lambda function once, then search for the memory size and model variant per function that meets your constraints — analytically, offline, in milliseconds.

Interactive demo · Usage · Architecture · Audit & target design · Developer guide

---

The problem

A serverless ML pipeline has two knobs per function — how much memory to buy and which model variant to run — and every setting trades latency against cost against accuracy. The demo's six-function pipeline has 103,680 distinct deployments, and no two of them agree on which is best.

What OptiServe does

		Needs AWS?
1. Measure	Profile a deployed Lambda across memory sizes and fit rt(m) = a₀ + a₁·e^(−m/a₂) per model variant. Adaptive sampling, resumable checkpoints, guaranteed config restore.	Yes — the only step that does
2. Model	Reduce a probabilistic workflow graph — branches, parallel joins, retries, back-edges — to an expected end-to-end latency and cost, in closed form.	No
3. Optimize	Greedily search the joint (memory × variant) space along the probability-refined critical path.	No

Strategy	Optimizes	Subject to
BPBC	min latency	budget + accuracy
BCPC	min cost	latency + accuracy
BAPB	max accuracy	latency + budget

Measured, not asserted. On that 103,680-configuration pipeline a greedy run evaluates about 103 configurations — 0.1% of the space — in ~50 ms, and hits the brute-force optimum at 126 of 198 feasible constraint points (median 100%). The evaluation harness reports where it does not; the demo has the full per-strategy breakdown.

Quickstart

No AWS needed — the analytical half runs on synthetic models and injected pricing:

from optiserve import WorkflowGraph, ModelVariant, ApplicationPerformanceModeling, ApplicationOptimizer

grid = [128, 256, 512, 1024, 2048]
workflow = (WorkflowGraph()
    .add_ml_function(1, [ModelVariant("small", lambda m: 900 - 0.1*m, accuracy=0.70),
                         ModelVariant("large", lambda m: 1400 - 0.1*m, accuracy=0.95)], grid)
    .add_ml_function(2, [ModelVariant("fast", lambda m: 600 - 0.05*m, accuracy=0.60),
                         ModelVariant("accurate", lambda m: 950 - 0.05*m, accuracy=0.92)], grid)
    .add_edges([("Start", 1, 1.0), (1, 2, 1.0), (2, "End", 1.0)])
    .validate())

app = ApplicationPerformanceModeling(workflow.to_networkx(), delay_type="SFN")
app.cost_calculator.aws_pricing_units = {"compute": 1.6667e-5, "request": 2e-7}

optimizer = ApplicationOptimizer(app)
result = optimizer.BPBC(budget=optimizer.maximal_cost,
                        accuracy_constraint=0.75,
                        accuracy_formula=lambda a, b: (a + b) / 2)
print(result.response_time_ms, result.cost, result.memory_config, result.model_config)

Runnable version: examples/optimize_workflow.py. Full API walkthrough: docs/usage.md.

Install

Python 3.11+. AWS credentials come from the standard credential chain; OptiServe never reads or stores them, and there is no .env to fill in.

python -m pip install -e .                 # core library
python -m pip install -e ".[experiments]"  # + notebook / benchmark deps

Conda, or Docker with no local Python

conda env create -f environment.yml && conda activate optiserve

docker build --target runtime -t optiserve:latest . && docker run --rm optiserve:latest --help
docker compose run --rm tests         # full local stack
docker compose run --rm integration   # against a moto server, never real AWS

Command line

optiserve optimize --workflow demo/workflow_pipeline.json --strategy bpbc --accuracy 0.6
optiserve profile  --function my-lambda --yes --checkpoint-dir output/ckpt

optimize is fully offline and reads a declarative JSON spec — it never evaluates expressions from the file, so reading a spec is not equivalent to running it. profile is the only command that touches AWS, and it refuses to mutate a live function without --yes.

Design notes

• One package imports boto3. Everything above optiserve/aws/ is cloud-free, which is why the modelling and optimization engines are fully testable offline — and why the demo is generated by running the library rather than mocking it. 137 tests pass offline in 9 s.
• Published results and correct results are different artifacts. An audit found defects in the greedy strategies that cannot be fixed without moving published numbers. Both behaviours ship as named presets (OptimizerCompat.CORRECTED, the default, and PUBLISHED), each with its own frozen golden baseline. Say which preset produced any number you report.
• Topology-only work is memoized. Cycle discovery and the DAG-reduction predicate never depend on the latencies the optimizer varies — a measured ~2× with byte-identical output, proven by an equivalence test rather than asserted.
• Dependencies are pinned exactly. The golden suite asserts byte-identical CSV output, so a float-repr change in numpy or pandas is a behavioural change for this package. Bump deliberately, with baselines regenerated in the same commit.

Project layout

optiserve/      the framework (aws, cost, profiling, modeling, workflow,
                optimization, evaluation, datasets, visualization,
                observability, cli)
demo/           interactive visual demo + the script that generates its data
docs/           usage, architecture, target design, developer guide
examples/       runnable usage examples + a declarative workflow spec
experiments/    benchmark functions + evaluation notebooks
tests/          unit, golden-master, and moto-backed integration tests
scripts/        maintenance utilities (dependency sync, .mdl migration)

The fitted .mdl performance models live in modeled_functions/. They are gitignored: they are produced by profiling against live AWS and cannot be regenerated without it.

License

MIT — see LICENSE.