JAMStack CI/CD with Lerna, NextJS, CDK, and Github Actions

With infrastructure as a code as a tool under my belt, I figured it was time to tackle CI/CD with Github. Up until now, I either deployed my apps with the help of a PaaS provider like Netlify or Vercel, or I triggered CDK deployments manually from my command line.

I wanted to see if it was feasible to handle CI/CD myself and I also simply wanted to learn how to use Github actions.

If there's any confusion, for this document "Github Workflow and Actions" will be our "CI/CD Pipeline".

Workflow

Here's a diagram of the actual cloud architecture behind this site, and numbers to assist with providing a rough overview of the CI/CD steps that I ended up adding.

Here's an interactive filetree of the code behind this site.

1. Github Actions YAML

Github actions allow you to build CD/CD workflows and are defined using YAML. They need to live in a special .github/workflows directory.

I have one workflow to run some actions on PR updates...

on:
  pull_request:
    branches:
      - master
# truncated...

...and one that handles deploying to AWS on merges into master.

on:
  push:
    branches:
      - master
# truncated...

2. Workflow Gets Triggered

The following is my actual deployment workflow file. On merges to master, there are a few actions that run. One to check out my project, one to set up AWS credentials for the entire workflow, and one that uses a NodeJs Docker image to run scripts that are defined in my project.

on:
  push:
    branches:
      - master

jobs:
  build-deploy:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v2

      - name: Configure AWS Credentials
        uses: aws-actions/configure-aws-credentials@v1
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: us-east-1

      - name: Use Node.js
        uses: actions/setup-node@v1
        with:
          node-version: "12.x"

      # `yarn` returned some registry errors so I opted for `npm`
      - name: Install
        run: npm i

      # Step 3 starts here !!!
      - name: Build
        run: yarn build

      - name: Export
        run: yarn export

      # Step 4 starts here !!!
      - name: Deploy
        run: yarn cdk deploy --require-approval=never

3. Lerna Scripts

"Lerna is a tool for managing JavaScript projects with multiple packages."

I recently learned how to use a really neat tool, Lerna, by accident after trying to figure out how this project was managing a nearly identical stack to mine.

I used Lerna to simplify some scripts given my multiple packages — there are only two for now, but it'll become even more handy if the number grows. The scripts that get run in the workflow are defined below:

{
  "name": "root",
  "private": true,
  "scripts": {
    "postinstall": "lerna exec npm i",
    "build": "lerna exec yarn build",
    "clean": "lerna run clean",
    "watch": "lerna run watch --stream",
    "test": "lerna run test --",
    "export": "lerna run export",
    "cdk": "lerna exec yarn cdk --scope=infrastructure --",
    "dev": "lerna exec yarn dev --scope=application"
  },
  "devDependencies": {
    "lerna": "3.22.1"
  }
}

Lerna runs the build and export scripts in each respective package if they are defined.

"build": "lerna exec yarn build"

This compiles CDK TypeScript code, and builds the NextJS app.

"export": "lerna run export",

This exports statically generated files from the NextJS app.

"cdk": "lerna exec yarn cdk --scope=infrastructure --",

The workflow runs yarn cdk passing along necessary arguments and flags (deploy --require-approval=never). This triggers the Cloudformation stack update via the CDK, updating all the intrastructure, uploading lambda code and statically generated assets, and invalidating the CloudFront cache.

4. Update Cloudformation

WHen deploying with the CDK from my commandline, it usually occupies my terminal for 5-10 minutes, and I'm always scared of accidentally stopping it the process and possibly casuing Cloudformation to hang up in some borked limbo state.

Offloading this to a Github workflow has been a total gamechanger since it frees up my terminal. It just makes sense!

This is pretty much the end of the whole process. The build-deploy workflow usually completes in around 5 minutes.

Infrastructure

Here's a bullet list summary of the infrastructure that lives as code and gets deployed via the CI/CD pipeline.

• Frontend-ish
  • Statically generated HTML/CS/JS via NextJS
• Backend-ish
  • API Gateway
    • Lambda Integration
    • Dynamo DB
• Everything else
  • S3 to serve frontend assets
  • Cloudfront as a CDN & request router
    • /api/* → API Gateway
    • (*) → S3
  • Route 53
    • Hosted Zone
    • Domain Name
    • DNS Resolution
  • ACM

It's pretty nice (and mindblowing at first) to be able to push some code, and have magical Github Octo-elves deploy all of the above resources for you while you sit back and eat chips.

It makes me feel like I have superpowers 🦄.

Closing Thoughts

Getting to the first successful deployment CI/CD came with a learning curve and cognitive over head. Scripts that I could run successfully locally didn't necessarily work for the CI/CD pipeline due to environment differnces. This took some trial and error.

My optimistic mindset was that whatever issues I encountered, there was a 99% chance that someone else encountered it before me, solved it, and open-sourced a Github Action to solve it for everyone else in their workflows. And that weren't the case there would be at least be docs/stackoverflow posts.

Also, after learning Lerna I seemingly started to notice it everywhere.

Large projects that use Lerna.

• ⚡️ NextJS
• 🧪 Optimizely
• 🚀 Apollo Server
• ☁️ AWS CDK

Next

The next thing I might tackle is creating dynamic, immutable PR environments like <{PR-title}-{hash:8}>.thekevinwang.com