Skip to content

Install v2 GitHub actions for v2 sync connections - #5244

Open
midigofrank wants to merge 9 commits into
sandbox-sync-uifrom
v2-github-actions
Open

midigofrank wants to merge 9 commits into
sandbox-sync-uifrom
v2-github-actions

Conversation

@midigofrank

@midigofrank midigofrank commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

Description

This PR makes v2 GitHub sync connections use @v2 of the OpenFn deploy and pull GitHub Actions. These actions sync an openfn.yaml CLI workspace with openfn project pull and openfn project deploy. Legacy connections stay on @v1 and behave as before.

Stacked on sandbox-sync-ui.

Installing the actions

  • New templates, priv/github/deploy_v2.yml and priv/github/pull_v2.yml, are picked when sync_version is true.
  • Deploy only runs on pushes that touch the project's workspace folder or its own workflow file. It can also be dispatched by hand with force, and a commit with [openfn force] in its message forces a deploy. The v2 "Configure OpenFn" commit uses this marker.
  • Pull and deploy share a concurrency group per branch, so they never push to the same branch at once.
  • The pull run name includes the project id. openfn-pull.yml is shared by every project in the repo, and the id is how Lightning finds this project's runs.
  • The API secret is now written before the deploy workflow is pushed, because the workflow can run as soon as it lands on the branch.
  • For v2, Lightning no longer commits a config file. Verifying the connection no longer checks for one, and disconnecting leaves openfn.yaml in place.

Workspace folder and local alias

  • New workspace_path column on project_repo_connections sets where the CLI workspace lives in the repo (default .). It can be changed on reconnect, and the change is audited as repo_connection_updated.
  • New virtual local_alias is passed to the first pull. The CLI uses it to name the project's local copy.
  • config_path and workspace_path are validated as plain relative paths inside the repo because they get written into the workflow files. The alias is validated because it ends up in a filename.
  • The config format, workspace folder and local alias fields sit in a collapsed "Advanced settings" section.
  • Before connecting, the form checks the chosen branch and workspace (VersionControl.check_workspace/2):
    • If another local project already uses the alias, the Connect button stays disabled until a different alias or folder is picked. The CLI won't pull while the alias clashes. The warning links to Advanced settings.
    • If another project already syncs to the folder, the form warns, and the confirm box says that project's files will be replaced. Deploying (importing) from another project's folder is the normal case and gets no warning.
    • If the repo's shared openfn-pull.yml uses the other format (v1 vs v2), the confirm box warns that connecting replaces it and stops syncs for the projects already using it.

Sync status (connected v2 connections)

  • VersionControl.get_sync_status/1 reads the branch's .openfn/branches/<branch>/checkout.yaml and .projects/*.yaml to tell whether the branch tracks this project. It also fetches the latest pull and deploy runs (new GithubClient.list_workflow_runs/4). A workflow GitHub doesn't list yet (a 404, e.g. not yet run and not on the default branch) counts as having no runs.
  • The "verify connection" banner is replaced by a sync health block with one row each for repository setup, branch tracking, last pull and last deploy. It has a manual refresh and polls while the first sync is setting up.
  • The connected panel shows the workspace folder and the project's local alias.

Validation steps

  1. Point VersionControl.endpoint_url/0 at a public URL (e.g. ngrok) so the actions can reach your local Lightning.
  2. In project settings → Sync, connect a repo with the v2 format. Optionally set a workspace folder and local alias under "Advanced settings". Try a folder another project already uses to see the alias clash and takeover warnings. Try both directions.
  3. Check that openfn-pull.yml and openfn-<project>-deploy.yml land in the repo, no config file is committed, and the first sync run commits the workspace.
  4. Back on the Sync tab, the status block should go from "Setting up" to "Sync healthy" and show the last pull/deploy runs.
  5. Change a workflow in the workspace on the branch and push. Deploy should run. Pushing outside the workspace folder should not trigger it.
  6. Reconnect with a different workspace folder and check the audit trail. Disconnect and confirm openfn.yaml is left in place.
  7. Connect a legacy-format project and confirm nothing changed.

Additional notes for the reviewer

  1. Do not merge as is: the v2 templates point at midigofrank/cli-deploy-action@v2 and midigofrank/cli-pull-action@v2 for testing (df7f500). They need to go back to the OpenFn actions once those @v2 tags are published.
  2. workspace_projects/2 makes one GitHub request per local project copy. That's fine for the few copies a workspace holds, and it's marked with a ponytail: comment.
  3. The status checks rely on the CLI's file layout (checkout.yaml, .projects/<alias>@<host>.yaml), taken from the @openfn/project and @openfn/cli sources. If the CLI changes that layout, the checks break.

AI Usage

Please disclose whether you've used AI anywhere in this PR (it's cool, we just
want to know!):

  • I have used Claude Code
  • I have used another model
  • I have not used AI

You can read more details in our
Responsible AI Policy

Pre-submission checklist

  • I have performed an AI review of my code (we recommend using /review
    with Claude Code)
  • I have implemented and tested all related authorization policies.
    (e.g., :owner, :admin, :editor, :viewer)
  • I have updated the changelog.
  • I have ticked a box in "AI usage" in this PR

@midigofrank
midigofrank added this pull request to stack #5245 October 13, 2026 14:01
@github-project-automation github-project-automation Bot moved this to New Issues in Core Oct 13, 2026
Comment thread priv/github/deploy_v2.yml Outdated
jobs:
deploy-to-lightning:
runs-on: ubuntu-latest
name: A job to deploy to Lightning

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we rename this to Deploy changes to OpenFn

Comment thread priv/github/pull_v2.yml Outdated
runs-on: ubuntu-latest
permissions:
contents: write
name: A job to pull changes from Lightning

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And rename this to Pull changes from OpenFn

Comment thread priv/github/deploy_v2.yml
contents: write
steps:
- name: openfn deploy
uses: midigofrank/cli-deploy-action@v2

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will need to be saved under openfn - is this just becuase you've built a test action in your own repo?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, this is just for testing. I'm going to revert it

v2 connections can now point at a workspace folder inside the repo
instead of assuming the repo root. That lets several projects share one
repo. The folder is passed to the pull and deploy actions, and the
deploy workflow only triggers on changes under it. Reconnecting can move
the folder, which is audited as repo_connection_updated. Both paths are
validated so they stay inside the repo, because they get written into
the workflow files.

A first pull can set a local alias for the CLI's project copy. The Sync
tab warns when that alias clashes with another project already in the
workspace.

The Sync tab now shows a status block for v2 connections. It covers the
repo setup, which project the branch's checkout tracks, and the latest
pull and deploy runs, and it polls while the first sync is being set up.
v2 no longer checks for a config file, since the CLI owns the workspace.

The API secret is now set before the deploy workflow is pushed, so the
first run doesn't fail without it. The configure commit for a v2 deploy
carries [openfn force] so that first deploy goes through.
The GitHub actions, config.json and the alias clash check all read
LightningWeb.Endpoint.url() separately. They now go through
VersionControl.endpoint_url/0 so they always agree on the host the CLI
names its project copies after. Because the function is public, local
testing can swap in a tunnel URL such as ngrok.
"A job to deploy to Lightning" becomes "Deploy project to OpenFn", and
"A job to pull changes from Lightning" becomes "Pull project from OpenFn".
GitHub only lists runs for a workflow once it has run or is on the default
branch. A pull-direction deploy workflow on another branch returned 404 and
failed the whole sync status.
Every project connected to a repo shares openfn-pull.yml, and v1 and v2
take different inputs. Connecting with the other format replaces the file
and stops syncs for the projects already using it. check_workspace/2 now
reads the file on the default branch for both formats, and the confirm box
warns about it next to the file it lists.
The CLI won't pull a project whose alias is already taken in the
workspace, so the connect button stays disabled until the clash is
gone. Taking over a folder that another project syncs to is no longer
a dead end. It's a warning, and the confirm box says that project's
files will be replaced. Importing from another project's folder (deploy)
is the normal case, so that still gets no warning. Both problems now
share one alert.

The config format, workspace folder and local alias fields move into a
collapsed "Advanced settings" section, and the warning links straight
to it. This replaces the "Use the alias" button. The run rows now show
the sync direction, and the status pill sits at the end of the row.
@midigofrank
midigofrank marked this pull request as ready for review October 15, 2026 17:29
@github-actions

Copy link
Copy Markdown

Security Review ⚠️

Other Security Observations

  • priv/github/deploy_v2.yml:31 and priv/github/pull_v2.yml:46 — the v2 workflow templates installed into every connected user's repo reference midigofrank/cli-deploy-action@v2 and midigofrank/cli-pull-action@v2, a personal GitHub namespace rather than the openfn organization the CHANGELOG advertises. These actions run with contents: write and the project's OpenFn API key (${{ secrets[inputs.apiSecretName] }} / secrets.<api_secret_name>), so if that personal account is compromised or force-pushes the v2 tag, every v2-sync customer executes attacker-controlled code with their OpenFn API token. Commit df7f500fc5 ("Point v2 GitHub actions at midigofrank forks for testing") suggests this is temporary, but nothing on the branch reverts it before merge.

S0, S1 and S2 pass: new web-layer events (save, reconnect, delete-connection, initiate-sync) still gate on can_install_github / can_initiate_github_sync and derive project_id from socket.assigns.project rather than params (lib/lightning_web/live/project_live/github_sync_component.ex:46,57,88,99); check_workspace/2 guards via authorize_installation_access (lib/lightning/version_control/version_control.ex:219); get_sync_status/1 operates on the connection loaded by the parent LV with get_repo_connection_for_project(project.id); workspace_path is persisted and audited via a new repo_connection_updated event inside the same Repo.transact (lib/lightning/version_control/version_control.ex:87,104, lib/lightning/version_control/audit.ex:60); and workspace_path / local_alias are format-validated and rejected on absolute paths or .. traversal (lib/lightning/version_control/project_repo_connection.ex:93-108,135-139).

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: New Issues

Development

Successfully merging this pull request may close these issues.

2 participants