Deployment Guide for ShipClojure

This guide will walk you through the process of deploying your ShipClojure application to Fly.io, including setting up a PostgreSQL database, configuring secrets, and enabling automated deployments through GitHub Actions.

Prerequisites

Before starting, ensure you have:

  1. Installed the Fly CLI

  2. Created a Fly.io account and logged in:

       fly auth login

  3. A GitHub repository for your project

Step 1: Create Fly.io Apps

Create two applications on Fly.io - one for production and one for staging:

fly apps create [YOUR_APP_NAME]
fly apps create [YOUR_APP_NAME]-staging

Make sure these names match the app set in your fly.toml file.

Step 2: Create a PostgreSQL Database

Create a PostgreSQL database on Fly.io to store your application data:

fly postgres create --name [YOUR_DB_NAME]

When prompted, select:

  • Organization: Your personal or team organization
  • Region: Choose the region closest to your users
  • Configuration: Development (single node) or High Availability (multiple nodes)
  • VM size: Based on your needs (e.g., shared-cpu-1x)

After creation, you'll receive database credentials - save these in a secure place.

Step 3: Attach the Database to Your App

Connect your database to your application:

fly postgres attach [YOUR_DB_NAME] --app [YOUR_APP_NAME]
fly postgres attach [YOUR_DB_NAME] --app [YOUR_APP_NAME]-staging

This creates a DATABASE_URL environment variable in your apps.

Step 4: Configure Application Secrets

4.1: Create a Local Secrets File

First, copy the example secrets file:

cp saas-secrets.example.edn resources/.prod-secrets.edn

Edit this file with your actual production secrets:

{:oauth2-providers {:google {:client-id "your-client-id", :client-secret "your-client-secret"}}
 :db {:dbtype "postgresql"
      :port 5432
      :host "your-db-host" ; Will be set by DATABASE_URL
      :user "postgres"     ; Will be set by DATABASE_URL
      :password "your-password" ; Will be set by DATABASE_URL
      :dbname "your-dbname" ; Will be set by DATABASE_URL
      :serverTimezone "UTC"
      :idleConnectionTestPeriod 30
      :autoReconnect true
      :characterEncoding "UTF-8"}
 :stripe {:api-key "your-stripe-api-key"
          :webhook-signing-secret "your-webhook-secret"}
 :jwt {:access-token "your-access-token-secret"
       :refresh-token "your-refresh-token-secret"}
 :email {:api-key "your-email-service-api-key"}
 :totp {:secret "your-totp-secret"}
 :cookie-secret "your-cookie-secret"
 :cookie-name "saas-session"}

4.2: Generate Secure Random Secrets

For security-critical values, generate secure random secrets:

# Generate JWT secrets
ACCESS_TOKEN_SECRET=$(openssl rand -hex 64)
REFRESH_TOKEN_SECRET=$(openssl rand -hex 64)

# Generate cookie secret
COOKIE_SECRET=$(openssl rand -hex 32)

# Update your .prod-secrets.edn file with these values

Set the cookie secret on both apps:

fly secrets set COOKIE_SECRET=$(openssl rand -hex 32) --app [YOUR_APP_NAME]
fly secrets set COOKIE_SECRET=$(openssl rand -hex 32) --app [YOUR_APP_NAME]-staging

4.4: Prevent Search Engine Indexing on Staging

fly secrets set ALLOW_INDEXING=false --app [YOUR_APP_NAME]-staging

Step 5: Set Up GitHub Actions for CI/CD

5.1: Create a Fly API Token

Generate a token for GitHub Actions to deploy your app:

fly tokens create org

This will output a token that starts with FlyV1. Save this token securely.

5.2: Add the Token to GitHub Secrets

  1. Go to your GitHub repository
  2. Navigate to Settings > Secrets and variables > Actions
  3. Click "New repository secret"
  4. Name: FLY_API_TOKEN
  5. Value: Paste the token you generated
  6. Click "Add secret"

Unknown type: ':html-inline'.{:type :html-inline, :content [{:type :text, :text "<a id=\"org405661f\">"}]}Unknown type: ':html-inline'.{:type :html-inline, :content [{:type :text, :text "</a>"}]}

5.3: Add Your Production Secrets to GitHub

  1. Create another GitHub secret named PROD_SECRETS_CONTENT
  2. Copy the entire content of your resources/.prod-secrets.edn file
  3. Paste it as the value of this secret

Unknown type: ':html-inline'.{:type :html-inline, :content [{:type :text, :text "<a id=\"orgcecc88d\">"}]}Unknown type: ':html-inline'.{:type :html-inline, :content [{:type :text, :text "</a>"}]}

5.4: Create CI/CD Workflow File

Create a file at .github/workflows/ci.yml with the following content:

name: CI & Deploy

on:
  push:
    branches: [main, dev]
  pull_request:
    branches: [main, dev]
  workflow_call:

jobs:
  lint:
    name: Lint
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: ./.github/actions/cache-clojure-deps
        with:
          key-label: 'lint'
      - uses: jdx/mise-action@v2
        with:
          install_args: 'babashka clj-kondo java clojure node'
      - name: Lint
        run: bb deps && bb lint-init && bb lint-ci

  tests:
    name: Run Tests
    runs-on: ubuntu-latest
    services:
      postgres:
        image: postgres:14
        env:
          POSTGRES_PASSWORD: postgres
          POSTGRES_USER: postgres
          POSTGRES_DB: postgres
        options: >-
          --health-cmd pg_isready
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5
        ports:
          - 5432:5432
    steps:
      - uses: actions/checkout@v4
      - uses: ./.github/actions/cache-clojure-deps
        with:
          key-label: 'tests'
      - uses: jdx/mise-action@v2
        with:
          install_args: 'babashka java clojure'
      - name: Run tests
        run: bb deps && bb test
        env:
          DB_HOST: localhost
          DB_PORT: 5432
          DB_USER: postgres
          DB_PASSWORD: postgres
          DB_NAME: postgres

  deploy-production:
    name: Deploy to Production
    runs-on: ubuntu-latest
    needs: [lint, tests]
    if: github.ref == 'refs/heads/main'
    steps:
      - uses: actions/checkout@v4

      # Create the secrets file from GitHub secret
      - name: Create production secrets file
        run: |
          mkdir -p resources
          echo "${{ secrets.PROD_SECRETS_CONTENT }}" > resources/.prod-secrets.edn

      - uses: superfly/flyctl-actions/setup-flyctl@master

      - name: Deploy to Production
        run: flyctl deploy --remote-only --app [YOUR_APP_NAME]
        env:
          FLY_API_TOKEN: ${{ secrets.FLY_API_TOKEN }}

  deploy-staging:
    name: Deploy to Staging
    runs-on: ubuntu-latest
    needs: [lint, tests]
    if: github.ref == 'refs/heads/dev'
    steps:
      - uses: actions/checkout@v4

      # Create the secrets file from GitHub secret
      - name: Create production secrets file
        run: |
          mkdir -p resources
          echo "${{ secrets.PROD_SECRETS_CONTENT }}" > resources/.prod-secrets.edn

      - uses: superfly/flyctl-actions/setup-flyctl@master

      - name: Deploy to Staging
        run: flyctl deploy --remote-only --app [YOUR_APP_NAME]-staging
        env:
          FLY_API_TOKEN: ${{ secrets.FLY_API_TOKEN }}

Make sure to replace [YOUR_APP_NAME] and [YOUR_APP_NAME]-staging with your actual app names.

Step 6: Push to GitHub

Assuming you've already set up your repository as described in Getting Started, push your changes to GitHub:

git add .
git commit -m "Configure deployment"
git push -u origin main

If you haven't set up your repository yet, follow these steps:

# If you cloned from ShipClojure and set upstream as described in Getting Started
git add .
git commit -m "Configure deployment"
git push -u origin main

# OR if you need to initialize a new repository
git add .
git commit -m "Configure deployment"
git remote add origin <ORIGIN_URL>
git push -u origin main

Step 7: Verify Deployment

After pushing to the main branch, GitHub Actions will:

  1. Run linting and tests
  2. If successful, deploy to your production environment

You can monitor the deployment in:

  • GitHub Actions tab of your repository
  • Fly.io dashboard: https://fly.io/apps/[YOUR_APP_NAME]
  • Using the CLI: fly status -a [YOUR_APP_NAME]

Deployment Workflow

With this setup:

  • Every commit to main will deploy to production after passing tests and linting
  • Every commit to dev will deploy to staging after passing tests and linting
  • Pull requests to either branch will run tests and linting without deploying

Troubleshooting

Database Connection Issues

If your app can't connect to the database:

# Check if the DATABASE_URL is set
fly secrets list -a [YOUR_APP_NAME]

# You can manually set it if needed
fly secrets set DATABASE_URL=postgres://postgres:password@[YOUR_DB_NAME].internal:5432/postgres -a [YOUR_APP_NAME]

Deployment Failures

Check the deployment logs:

fly logs -a [YOUR_APP_NAME]

Accessing the Database Console

To directly access your PostgreSQL database:

fly postgres connect -a [YOUR_DB_NAME]

This opens a psql shell where you can run SQL commands.

Scaling Your Database

If you need more resources for your database:

# Scale up the VM
fly machine update [MACHINE_ID] --vm-memory 1024 --app [YOUR_DB_NAME]

# Scale out by adding a replica (for HA clusters)
fly machine clone [MACHINE_ID] --region [REGION] --app [YOUR_DB_NAME]

Security Considerations

  • Rotate your Fly API token periodically
  • Update your application secrets regularly
  • Review GitHub repository access controls to protect your workflows
  • Monitor your application and database logs for suspicious activity

By following this guide, you've set up a complete CI/CD pipeline for your ShipClojure application, with automated testing and deployment to both staging and production environments on Fly.io.