Repository navigation
Install v2 GitHub actions for v2 sync connections - #5244
midigofrank wants to merge 9 commits into
Conversation
| jobs: | ||
| deploy-to-lightning: | ||
| runs-on: ubuntu-latest | ||
| name: A job to deploy to Lightning |
There was a problem hiding this comment.
Can we rename this to Deploy changes to OpenFn
| runs-on: ubuntu-latest | ||
| permissions: | ||
| contents: write | ||
| name: A job to pull changes from Lightning |
There was a problem hiding this comment.
And rename this to Pull changes from OpenFn
| contents: write | ||
| steps: | ||
| - name: openfn deploy | ||
| uses: midigofrank/cli-deploy-action@v2 |
There was a problem hiding this comment.
This will need to be saved under openfn - is this just becuase you've built a test action in your own repo?
There was a problem hiding this comment.
Yeah, this is just for testing. I'm going to revert it
fd9c604 to
7f61ec9
Compare
v2 connections can now point at a workspace folder inside the repo instead of assuming the repo root. That lets several projects share one repo. The folder is passed to the pull and deploy actions, and the deploy workflow only triggers on changes under it. Reconnecting can move the folder, which is audited as repo_connection_updated. Both paths are validated so they stay inside the repo, because they get written into the workflow files. A first pull can set a local alias for the CLI's project copy. The Sync tab warns when that alias clashes with another project already in the workspace. The Sync tab now shows a status block for v2 connections. It covers the repo setup, which project the branch's checkout tracks, and the latest pull and deploy runs, and it polls while the first sync is being set up. v2 no longer checks for a config file, since the CLI owns the workspace. The API secret is now set before the deploy workflow is pushed, so the first run doesn't fail without it. The configure commit for a v2 deploy carries [openfn force] so that first deploy goes through.
The GitHub actions, config.json and the alias clash check all read LightningWeb.Endpoint.url() separately. They now go through VersionControl.endpoint_url/0 so they always agree on the host the CLI names its project copies after. Because the function is public, local testing can swap in a tunnel URL such as ngrok.
"A job to deploy to Lightning" becomes "Deploy project to OpenFn", and "A job to pull changes from Lightning" becomes "Pull project from OpenFn".
7f61ec9 to
88718f5
Compare
GitHub only lists runs for a workflow once it has run or is on the default branch. A pull-direction deploy workflow on another branch returned 404 and failed the whole sync status.
Every project connected to a repo shares openfn-pull.yml, and v1 and v2 take different inputs. Connecting with the other format replaces the file and stops syncs for the projects already using it. check_workspace/2 now reads the file on the default branch for both formats, and the confirm box warns about it next to the file it lists.
The CLI won't pull a project whose alias is already taken in the workspace, so the connect button stays disabled until the clash is gone. Taking over a folder that another project syncs to is no longer a dead end. It's a warning, and the confirm box says that project's files will be replaced. Importing from another project's folder (deploy) is the normal case, so that still gets no warning. Both problems now share one alert. The config format, workspace folder and local alias fields move into a collapsed "Advanced settings" section, and the warning links straight to it. This replaces the "Use the alias" button. The run rows now show the sync direction, and the status pill sits at the end of the row.
Security Review
|
Description
This PR makes v2 GitHub sync connections use
@v2of the OpenFn deploy and pull GitHub Actions. These actions sync anopenfn.yamlCLI workspace withopenfn project pullandopenfn project deploy. Legacy connections stay on@v1and behave as before.Stacked on
sandbox-sync-ui.Installing the actions
priv/github/deploy_v2.ymlandpriv/github/pull_v2.yml, are picked whensync_versionis true.force, and a commit with[openfn force]in its message forces a deploy. The v2 "Configure OpenFn" commit uses this marker.concurrencygroup per branch, so they never push to the same branch at once.openfn-pull.ymlis shared by every project in the repo, and the id is how Lightning finds this project's runs.openfn.yamlin place.Workspace folder and local alias
workspace_pathcolumn onproject_repo_connectionssets where the CLI workspace lives in the repo (default.). It can be changed on reconnect, and the change is audited asrepo_connection_updated.local_aliasis passed to the first pull. The CLI uses it to name the project's local copy.config_pathandworkspace_pathare validated as plain relative paths inside the repo because they get written into the workflow files. The alias is validated because it ends up in a filename.VersionControl.check_workspace/2):openfn-pull.ymluses the other format (v1 vs v2), the confirm box warns that connecting replaces it and stops syncs for the projects already using it.Sync status (connected v2 connections)
VersionControl.get_sync_status/1reads the branch's.openfn/branches/<branch>/checkout.yamland.projects/*.yamlto tell whether the branch tracks this project. It also fetches the latest pull and deploy runs (newGithubClient.list_workflow_runs/4). A workflow GitHub doesn't list yet (a 404, e.g. not yet run and not on the default branch) counts as having no runs.Validation steps
VersionControl.endpoint_url/0at a public URL (e.g. ngrok) so the actions can reach your local Lightning.openfn-pull.ymlandopenfn-<project>-deploy.ymlland in the repo, no config file is committed, and the first sync run commits the workspace.openfn.yamlis left in place.Additional notes for the reviewer
midigofrank/cli-deploy-action@v2andmidigofrank/cli-pull-action@v2for testing (df7f500). They need to go back to the OpenFn actions once those@v2tags are published.workspace_projects/2makes one GitHub request per local project copy. That's fine for the few copies a workspace holds, and it's marked with aponytail:comment.checkout.yaml,.projects/<alias>@<host>.yaml), taken from the@openfn/projectand@openfn/clisources. If the CLI changes that layout, the checks break.AI Usage
Please disclose whether you've used AI anywhere in this PR (it's cool, we just
want to know!):
You can read more details in our
Responsible AI Policy
Pre-submission checklist
/reviewwith Claude Code)
(e.g.,
:owner,:admin,:editor,:viewer)