Skip to content

fix: wait for Mooncake master exit before replacement - #213

Open
ji-huazhong wants to merge 1 commit into
Ascend:mainfrom
ji-huazhong:fix/mooncake-master-restart
Open

ji-huazhong wants to merge 1 commit into
Ascend:mainfrom
ji-huazhong:fix/mooncake-master-restart

Conversation

@ji-huazhong

Copy link
Copy Markdown
Collaborator

Wait for captured master processes to release their listeners after SIGTERM, escalating to SIGKILL only for those still alive. Cover both delayed shutdown and forced termination before starting the replacement master.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 10, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-10T12:36:55.407475Z 20e3e29 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@ji-huazhong
ji-huazhong force-pushed the fix/mooncake-master-restart branch from 20e3e29 to 7c6e83a Compare November 10, 2026 12:33

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 20e3e2940d

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

@ji-huazhong
ji-huazhong force-pushed the fix/mooncake-master-restart branch 2 times, most recently from 0338bff to 6f8edc7 Compare November 10, 2026 12:47
Signed-off-by: ji-huazhong <hzji210@gmail.com>
@ji-huazhong
ji-huazhong force-pushed the fix/mooncake-master-restart branch from 6f8edc7 to fa5e868 Compare November 10, 2026 12:51
@ji-huazhong

Copy link
Copy Markdown
Collaborator Author

cc @huniu20

@huniu20

huniu20 commented Nov 11, 2026

Copy link
Copy Markdown
Contributor

Overall LGTM. A few comments:

  • Over-matching: pgrep -f mooncake_master matches full command lines, so processes like tail -f /tmp/mooncake_master.log get caught. They now get SIGKILL after the timeout, and the caller could even kill itself. Please filter by proc.name() == "mooncake_master" and exclude os.getpid(), or switch to psutil.process_iter(["name"]).

  • AccessDenied: terminate() can raise psutil.AccessDenied, for example when the master belongs to another user. That exception escapes raw, and the PIDs already sent SIGTERM are never waited on. Please catch it and raise a RuntimeError that includes the PID.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants