Promise.all is too much of a good thing

Sometimes, your application architecture or constraints force you to do a bunch of database lookups in rapid succession. Running them sequentially, one after another, can take a long time. With great new syntax in Javascript, like async and await, running multiple operations concurrently is easier than ever. By doing so, you can complete a big operation faster, but you risk shooting yourself in the foot and bringing your entire system to its knees. Good application performance needs a holistic view: making one operation faster at the expense of all the others is not a good engineering trade-off.

Recursive Common Table Expressions with Postgres

This week, I’ve been helping a client with some performance problems with their API. We’ve been focusing on tuning some Postgres queries to reduce latency and improve response times.

One area of the app that was struggling was a query that processes a tree structure. The app allows users to create folders and upload files to them.

The database structure is designed in a way that resembles the following:

create table folders (
    id bigserial primary key,
    name text not null,
    parent_id bigint,
    foreign key (parent_id) references folders(id)
);

create table files (
    id bigserial primary key,
    name text not null,
    folder_id bigint,
    size bigint not null,
    foreign key (folder_id) references folders(id)
);

Here, we have a table called folders and a table for files in each folder.

The parent_id column in the folders table is a self-referencing foreign key that allows us to form a tree by pointing each folder to its parent folder.

How to structure a workshop for a tech conference

View of audience at a tech conference workshop from the point of view of the presenter Workshops are challenging to design. You have no idea how many attendees will show up, or what their relative skill level or previous experience will be. You’ll have some attendees that have done similar things in the past, and will breeze through your content and exercises. Others will be entirely new to programming in general! For example, I recently did a workshop on vue.js, and I had participants who had not done any front-end coding since jQuery, and many who had done angular or react in the past.

MassTransit Transactional Outbox Table Schemas

MassTransit has a transactional outbox feature that integrates with entity framework so that your published messages are persisted in the same database transaction as your model changes.

If you use Entity Framework database migrations, this works out fine, it will add itself to your database context and migrate the database along with your own schema changes.

But if you’re like me and don’t trust code-first migrations, you need the raw SQL CREATE TABLE commands.

Solved (Sorta): My Mac Thinks its in UTC Timezone

I was working on a project and was trying to convert an ISO-8601 timestamp string from my server into localtime for display. Most libraries should handle that automatically, since the js Date object is pretty much always in local time. But it kept showing me a time that was off by a few hours.

Eventually I figured out that certain systems in my environment were using UTC time, even though I live in Kansas City and it should be using America/Chicago.

Optimizing build pipelines in Azure DevOps

Build time before optimization: ~50 minutes

We have a monolith application hosted in Azure DevOps. We use the Pipelines product to run builds and tests on every pull request. After merging, the builds run again. Build times were creeping up towards an hour, leading to some pain points for our developers:

  • It’s really frustrating to feel pressure to release a hotfix, but it takes multiple hours to prep a release artifact
  • Even an insignificant code change due to PR feedback led to delaying the merge another hour
  • Developers get distracted during builds and move on to new tickets, increasing work-in-process counts and delaying merges

I wanted to get it down to under 20 minutes. Here’s how we did it.