Skip to content

Document the Milvus object storage check - #1088

Open
Sabnaj-42 wants to merge 2 commits into
milvus-inplace-adoptionfrom
milvus-object-storage-check
Open

Sabnaj-42 wants to merge 2 commits into
milvus-inplace-adoptionfrom
milvus-object-storage-check

Conversation

@Sabnaj-42

Copy link
Copy Markdown

A new Milvus has an empty etcd. On an object storage path that still holds another cluster's write-ahead logs, log ids restart, the old logs fence the writer, and writes time out while the status looks healthy. The operator never looked at S3, and the default path is files for every Milvus in a bucket.

Before the first pod starts, the operator counts the keys of the meta etcd. If there are none, a Job runs milvus-archiver storage-check against the rendered milvus.yaml.

  • Path holds objects: the Milvus does not start. Condition ObjectStorageChecked=False, reason PathInUse, names the bucket and path. It rechecks every minute and starts by itself once the path is free.
  • Etcd holds data (adoption): passes with no Job.
  • The check cannot run: a warning, then the Milvus starts as before.
  • Never runs for a provisioned Milvus or a restore. Annotation kubedb.com/milvus-skip-object-storage-check: "true" skips it.

WipeOut still never deletes object storage, the same as the official Milvus operator.

Stacked on milvus-inplace-adoption; merge that first.

Signed-off-by: sabnaj <sabnaj@appscode.com>
Copilot AI balanced review requested due to automatic review settings November 2, 2026 07:54

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.

@github-actions

github-actions Bot commented Nov 2, 2026 •

Copy link
Copy Markdown

Visit the preview URL for this PR (updated for commit a9fcb61):

https://kubedb-v2-hugo--pr1088-milvus-object-storag-gv5zkd4v.web.app

(expires Thu, 12 Nov 2026 12:09:34 GMT)

🔥 via Firebase Hosting GitHub Action 🌎

Sign: 0f29ae8ae0bd54a99bf2b223b6833be47acd5943

Use milvuses.kubedb.com, wait for pods to exist, replace the Service
selector, note slow rollback, dead pods and the second Milvus on one Secret.

Signed-off-by: sabnaj <sabnaj@appscode.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants