2022, Week 31
Week 31 is from Monday, August 1, 2022 until (and including) Sunday, August 7, 2022.
WeekendHot Productive Tired
Over the past weekend, I built a fun little feature that I’m calling Regeneration History. You can see it on the right side column (desktop width only).
I also watched a lot of Big Timber in the background, and worked on some CloudWatch dashboards for Lambda function metrics.
I self-host & deploy my site on AWS so I can view some additional bits of info that you forfeit when using a fully managed platform-as-a-service like Netlify or Vercel.
I use serverless-next.js to build my site. This package implements Next.js’ ISR by storing statically generated pages in an S3 bucket. Anytime a page is re-generated, the S3 object is modified, and if you have versioning enabled, the object will get a new version rather than being overwritten.
For example, the S3 object for /wiki/cloud/aws/s3
will be an
.html file, located in the bucket at this particular path:
S3 Object Versioning
Regular traffic to a page, which implies regular static regeneration, will result in an object version dataset like so this...
August 2, 2022, 17:23:09 (UTC-04:00)
August 2, 2022, 07:08:11 (UTC-04:00)
July 31, 2022, 15:56:05 (UTC-04:00)
July 31, 2022, 00:34:25 (UTC-04:00)
July 30, 2022, 23:59:41 (UTC-04:00)
An audit log or append-only-like log such is certainly useful in other contexts, but I don’t think there is much value in this particular case of a website’s static files.
For fun though, in case you wanted to build some sort a time-travel feature (thinking about something similar to the React.js tic-tac-toe time-travel feature), you could accomplish that this mechanism... but again, not particularly useful.
Tuesday, Aug 2, 2022Big Ship
Terraform Core Versioned Docs
Today, I, along with my colleague Bryce, delivered versioned docs for the Core docs on terraform.io. This includes:
While the trivial on the surface — you get a nifty little dropdown — the underlying architecture is massively more complex. More on that in a dedicated post maybe...
Wednesday, Aug 3, 2022
Automated GitHub Releases
Mermaid Rabbit Hole & GitHub GraphQL API Rate LimitPoor choices
After work, I went into one of the worst rabbit holes in recent times.
The Problem: I had an initial implementation of
being rendered in MDX. However, these diagrams would render at build time
only, which takes place on my machine. Subsequent builds that would take
place on AWS Lambda in a regeneration lambda, during ISR would fail
with an difficult to diagnose error like:
_projectRoot is undefined. Unable to create a BrowserFetcher.
This lead me to use https://github.com/alixaxel/chrome-aws-lambda.
However this ended up resulting in file trace issues that were caused
by some incompatibility with the
target: "serverless" config that
...The ugly part...
Throughout this process, I had built my site several times. My site
makes calls to the GitHub GraphQL API for data to statically render
pages — mostly the pages on this
wiki portion of the
site. I hadn’t yet optimized these calls, but I quickly realized that
I was exceeding the 5000 req/hr rate limit.
I continuously monitored my usage at https://docs.github.com/en/graphql/overview/explorer
and continued debugging two issues simultaneously —
excessive API calls to the GitHub API.
I needed to replace my troublesome code, which was already deployed on AWS, with a new deployment but the static page generation during the build process was blocked my GitHub API rate limit.
Both the deployed code and build attempts were simultaneously exacerbating the rate limit.
GitHub resets your rate limit every hour, so I stayed up throughout 3 of these hour windows... from roughly 1:30AM to 4:30AM. In the first two windows, I had failed to find/solve the culprit, and rate limit was reached within minutes. I eventualy optimized and deleted some repeat API calls, and delete other unecessary ones.
Here's one for fetching all the files in a tree shape. I deleted this.
Logical Nullish Assignment
One takeaway and learning was this logical nullish assigment operator.
In this bare bones example,
fetch only gets called once, despite the
callApi function being called three times.
crude cache example
I eventually optimized a few repeat API calls, resolved my rate limit nightmare, never ended up getting Mermaid diagrams to render on Lambda, and eventually went to bed.
Awful life decision of staying up.
- Use a different GitHub account's PAT for API calls.
Thursday, Aug 4, 2022Dead
I basically felt like garbage all day, and was hit by back to back Vercel gotchas at work.
Next.js && Nested Middleware incompatibility
From JJ Kasper:
There should be a build error if nested middleware is used past 12.1.7-canary.9
Nix Shell + Mermaid CLI
mmdc is the CLI command for the Mermaid CLI.