CommitChange's version of Houdini

This is a Rails 7.0 app.

The frontend is written in a few custom frameworks, the largest of which is called Flimflam. We endeavor to migrate to React as quickly as possible to increase development comfort and speed.

All backend code and React components should be well-tested

Prerequisites

Houdini is designed and tested to run with the following:

• Ruby 3.3
• Node 16
• PostgreSQL 16
• run on Heroku-24

Dev Setup

Get the code

git clone https://github.com/Commitchange/houdini
git checkout supporter_level_goal

Get your .env file

If you don't already have access to the CommitChange 1Password vault, ask to be added. Then download the .env file in 1Password and place it in the root directory.

Note: Double check that your .env file has the '.' in front of the file name.

Dockerized

This is a work-in-progress method of running a development environment. The standard is still the bare metal instructions below.

Mac users can ignore this, but if your host machine is Linux, you might run into permission issues with the tmp files created by the postgres image. To proactively avoid this, run cp docker-compose.override.yml.example docker-compose.override.yml and change the values inside the newly copied file to match the output of echo $(id -u):$(id -g)

One-time setup:

touch ~/.netrc #prevents docker compose from creating it as a directory if you don't have it yet

docker compose run web bin/rails db:setup

Running:

docker compose up

Restoring the DB from Prod (Linux):

# Enter `password` when prompted for a password after the download step.
docker compose exec web script/restore_from_heroku.sh

Restoring the DB from Prod (Mac). The above command will work on Mac, but will take an hour or more due to differences in how docker handles storage. Use the below to reduce how long it takes (will still take a long time).

curl -o ./tmp/shared/latest.dump `heroku pg:backups:url -a commitchange`

# Enter `password` when prompted for a password.
docker compose exec db -e CC_PROD_DUMP_PATH="/tmp/shared/latest.dump" script/pg_restore_local_from_production.sh

Docker Development Commands

Use bin/cc-run for streamlined Docker commands:

# Generic commands - run any Rails/Bundle/Yarn command
bin/cc-run rails db:migrate
bin/cc-run rails db:setup
bin/cc-run rails generate model User
bin/cc-run rails routes | grep users
bin/cc-run bundle install
bin/cc-run bundle update
bin/cc-run yarn install
bin/cc-run yarn add react

# Convenient shortcuts
bin/cc-run console     # Rails console
bin/cc-run test        # Run tests
bin/cc-run bash        # Bash shell in web container
bin/cc-run worker      # Bash shell in worker container

Optional: Create a Shell Alias

For even faster commands, you can optionally create a shell alias:

# Add to ~/.bashrc or ~/.zshrc
alias ccr='bin/cc-run'

# Then reload your shell
source ~/.bashrc  # or source ~/.zshrc

Usage with alias:

ccr rails db:migrate
ccr console
ccr test
ccr bash

One-time setup (Ubuntu)

You'll want to run the next commands as root or via sudo (for Ubuntu 18.04 users or anyone running ProgresSQL 10, change "postgresql-12" below to "postgresql-10"). You could do this by typing sudo /bin/sh running the commands from there.

apt update
apt install curl -yy
curl -sL https://deb.nodesource.com/setup_16.x | bash -
curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | apt-key add -
echo "deb https://dl.yarnpkg.com/debian/ stable main" | tee /etc/apt/sources.list.d/yarn.list
apt update
apt install git postgresql-12 libpq-dev libjemalloc-dev libvips42 yarn -yy

You'll run the next commands as your normal user.

Note: in the case of a production instance, this might be your web server's user.

Note: We use rbenv to have more control over the exact version of Ruby. This tool is useful for switching between multiple Ruby versions on the same machine and for ensuring that each project you are working on always runs on the correct Ruby version. You could also build ruby from source.

Note: We recommend building Ruby with jemalloc support as we do in these instructions. In practice, it manages memory far more efficiently in Rails-based projects.

Tip: To get out of the root shell, run exit

Get the latest rbenv

git clone https://github.com/rbenv/rbenv.git ~/.rbenv

Add rbenv to bashrc:

echo 'eval "$(~/.rbenv/bin/rbenv init - bash)"' >> ~/.bashrc

Note: close and reopen your terminal.

Download the rbenv install feature:

git clone https://github.com/rbenv/ruby-build.git "$(rbenv root)"/plugins/ruby-build

Ruby install

cd houdini
rbenv install 3.0

Run the following command as the postgres user and then enter your admin password at the prompt.

Note: For development, Houdini expects the password to be 'password'. This would be terrible for production but for development, it's likely not a huge issue.

Tip: To run this, add sudo -u postgres to the beginning of the following command.

createuser admin -s -d -P

One-time setup (Mac)

Set your Ruby version with rbenv.

brew install rbenv
rbenv versions # see which ruby versions are already installed
rbenv install  # the app currently uses version 3.4.12
rbenv local # rbenv local --unset reverses the action

# To switch between rbenv versions installed locally, use the following command:
rbenv shell 3.4.12

Set your Node version with NVM.

brew install nvm
brew info nvm # command that shows the remaining steps to complete to install nvm properly
mkdir ~/.nvm
nvm install 16
nvm use 16
# Add the following lines to your ~/.bashprofile or ~/.zshrc:
echo 'export NVM_DIR="$HOME/.nvm"' >> ~/.zshrc
echo '[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh"  # This loads nvm' >> ~/.zshrc
echo '[ -s "$NVM_DIR/bash_completion" ] && \. "$NVM_DIR/bash_completion"  # This loads nvm bash_completion' >> ~/.zshrc

# Reference Stack Overflow post: https://stackoverflow.com/questions/53118850/brew-install-nvm-nvm-command-not-found

Make sure you've installed Yarn.

yarn --version
brew install yarn

Set your Postgres version with homebrew.

brew install postgresql@16
brew switch postgres@16

# To start postgres locally run:
brew services start postgresql@16

You might get segmentation faults if you don't configure pg with the correct macports. One of these should work.

gem install pg -- --with-pg-config="${HOMEBREW_PREFIX}/opt/libpq/bin/pg_config"
# Or?
bundle config build.pg --with-pg-config="${HOMEBREW_PREFIX}/opt/libpq/bin/pg_config"

You may also need to set the env variable PGGSSENCMODE=disable to resolve segmentation faults.

Create necessary postgres users in the psql console.

psql postgres # if this doesn't work, make sure postgres is running
CREATE ROLE admin WITH SUPERUSER CREATEDB LOGIN PASSWORD 'password';
CREATE ROLE postgres WITH SUPERUSER CREATEDB LOGIN PASSWORD 'password';

You may need to disable AirPlay Receiver in your System Settings if it is hogging port 5000.

System configuration (all)

There are a number of steps for configuring your Houdini instance for startup

Run bin/setup

bin/setup

Startup

run foreman for development

When you run foreman in dev, you start up the server, the job runner and webpack.

bin/dev

If you get ActiveRecord::NoDatabaseError errors, run bin/rails db:create:all to make sure all the databases are built.

Frontend

Assets get compiled from /client to /public/client

Documentation

You can get generate documentation for the Ruby source by running:

bundle exec yard doc

Alternatively, you can have it run in local webserver and autoupdate by running:

bundle exec yard server -r

Providing the complete corresponding source code

Note: This is not legal advice and provides a suggestion which may be compliant. You should talk with your legal counsel if you have questions or concerns with how to comply with the various licenses of Houdini

Providing the complete, corresponding source code (CCS) of your project is a requirement of some of the licenses used by Houdini. There are two methods for doing so right now:

1. Providing a tarball of the current running code
2. Providing a link to Github where the code is pulled from

The easiest method is to provide a tarball. Houdini automatically provides a link on the Terms & Privacy page which generates a tarball for the current running code at runtime. For this to work though, the following characteristics must be true:

• Your have to have committed any changes you made to the project in HEAD in your git repository
• The .git folder for your repository must be a direct subfolder of your $RAILS_ROOT
• Your web server must be able to run git archive.

Style

Ruby

• 2 spaces for tabs

New frontend code

• All new front end code should be written in Typescript and React (using TSX files). Please use the React Generators for creation.
• 2 spaces for tabs

Legacy Javascript

• 2 spaces for tabs
• Comma-led lines
• ES6 imports

Git

• No need to rebase, just merge

How to build releases at CommitChange

Releases go through the private deploy-houdini repo, which holds the built assets, and are then pushed to Heroku. Ask Eric for access to deploy-houdini and the commitchange Heroku app.

One-time setup

Install the GitHub CLI (brew install gh, then gh auth login) and the Heroku CLI (brew tap heroku/brew && brew install heroku, then heroku login). Then, from your houdini checkout:

gh auth setup-git    # lets git use your gh login for the private repo
git remote add private https://github.com/CommitChange/deploy-houdini.git
heroku git:remote --remote=production -a commitchange
git fetch private
git branch PRIVATE_MASTER private/master
git branch -u private/master PRIVATE_MASTER
git branch PRIVATE_PROD_DEPLOY private/prod_deploy
git branch -u private/prod_deploy PRIVATE_PROD_DEPLOY

Building the production assets runs rake assets:precompile, which boots the app. It needs the Ruby gems installed (bundle install) and a Postgres database at BUILD_DATABASE_URL (default postgres://admin:password@localhost/commitchange_development_legacy) with the schema loaded.

Build for production

1. Merge your changes into supporter_level_goal and push.
2. From an up-to-date supporter_level_goal, run ./create_new_release.sh. It checks out PRIVATE_MASTER, merges your commit, and commits a CCS_HASH file recording it.
3. Merge that into the deploy branch, build the assets if JS or assets changed, and push:

git checkout supporter_level_goal && git pull
./create_new_release.sh
git push private HEAD:master
git checkout PRIVATE_PROD_DEPLOY
git pull
git merge PRIVATE_MASTER
npm run build-all-production    # only if JS or assets changed
git add public
git commit -m "Build for <commit>"
git push private HEAD:prod_deploy
git push production HEAD:master

git push production HEAD:master is the step that deploys. Check the release with heroku releases -a commitchange.

Build for staging

• Run ./create_new_staging_release.sh from your houdini checkout. It uses gh workflow run to start the create-release workflow in deploy-houdini, and doesn't change anything on your machine.
• Once the deploy finishes, increase ASSET_VERSION by 1 in https://dashboard.heroku.com/apps/commitchange-test/settings.
• To copy the latest production backup onto staging, run the command below. It overwrites everything in the staging database.

heroku pg:backups:restore $(heroku pg:backups:url --app commitchange) --app commitchange-test

Production config

Environment variables

The following environnment variables MUST be set. If you miss any of them, the site may not run or if it does run, it may silently be broken. Do not ignore this information!

• ACTIVE_RECORD_ENCRYPTION_DETERMINISTIC_KEY
• ACTIVE_RECORD_ENCRYPTION_KEY_DERIVATION_SALT
• ACTIVE_RECORD_ENCRYPTION_PRIMARY_KEY
• AIRBRAKE_API_KEY
• AIRBRAKE_PROJECT_ID
• ASSET_VERSION
• AWS_ACCESS_KEY
• AWS_SECRET_ACCESS_KEY
• AWS_SES_ACCESS_KEY
• AWS_SES_SECRET_KEY
• BEACON_SECRET_KEY
• CUSTOM_HOST
• CYPHER_KEY
• DATABASE_URL
• DEVISE_SECRET_KEY
• ERROR_PAGE_URL
• FACEBOOK_API_SECRET
• FACEBOOK_APP_ID
• FEE_SWITCHOVER_TIME
• FROALA_KEY
• GOOGLE_API_KEY
• GOOGLE_AUTH_CLIENT_ID
• GOOGLE_TRACK_ID
• JEMALLOC_ENABLED
• JEMALLOC_VERSION
• MAILCHIMP_API_KEY
• MAILCHIMP_OAUTH_CLIENT_ID
• MAILCHIMP_OAUTH_CLIENT_SECRET
• MAILCHIMP_REDIRECT_URL
• MAILCHIMP_USERNAME
• MAINTENANCE_PAGE_URL
• MEMCACHIER_PASSWORD
• MEMCACHIER_SERVERS
• MEMCACHIER_USERNAME
• MINIMUM_RECAPTCHA_SCORE
• OPENREDIS_SECURE_URL
• OPENREDIS_URL
• ORG_NAME
• PAPERTRAIL_API_TOKEN
• RACK_TIMEOUT_SERVICE_TIMEOUT
• RAILS_ENV
• RECAPTCHA_ENTERPRISE
• RECAPTCHA_ENTERPRISE_API_KEY
• RECAPTCHA_ENTERPRISE_PROJECT_ID
• RECAPTCHA_SECRET_KEY
• RECAPTCHA_SITE_KEY
• S3_BUCKET_NAME
• SECRET_KEY_BASE
• SECRET_TOKEN
• Settings.aws.bucket_name
• STRIPE_API_KEY
• STRIPE_API_PUBLIC
• STRIPE_CONNECT_WEBHOOK_SECRET
• STRIPE_WEBHOOK_SECRET
• TWITTER_API_SECRET

Creating issues

• I'm a community member - You should file an issue upstream in https://github.com/houdiniproject/houdini

• I work for CommitChange and...

  • this is a CommitChange issue - create an issue in https://github.com/commitchange/tix
  • this may be an issue upstream - create an issue in https://github.com/commitchange/tix and maybe upstream. If you do file upstream, link to it the upstream issue in the tix issue.