Today has been a day of struggle with Effect. I probably spoiled it by saying nice things about Effect yesterday, which inevitably made today tougher. I’m also writing code to procrastinate from a writing task, or to let the writing thoughts percolate, so my mind isn’t calm at all.

Anyway, for fun I’m going to make some of my learning notes public.

All right, so today I’m trying to port one of Val Town’s healthcheck endpoints to use Effect. I think it’s a pretty place to start because it’s relatively unentangled with the rest of the application and it has some caching, timeout, and database logic.

Background (you can skip this part if you don’t care)

This endpoint has a migration checker in it. We use Drizzle for our database client and to manage migrations, and migrations are automatically applied whenever we deploy. But we have two services that are deployed: a React Router frontend and a Fastify backend. Only one of these services runs the migrations and they deploy simultaneously, so we want to prevent the race condition in which a server is deployed with new code and old database schemas.

So we had code that basically looks at the data migrations in code by looking at Drizzle’s metadata:

const when = JSON.parse(
Fs.readFileSync("./db/meta/_journal.json", "utf8")
).entries.at(-1)?.when;

And then compares that with the database migrations applied to the database by looking at Drizzle’s bookkeeping tables:

db.execute<{ id: number; created_at: number }>(sql`
select id, created_at
from
${sql.identifier(this._drizzle_table_schema)}.${sql.identifier(
this._drizzle_table_name
)}
order by created_at desc limit 1`),

Previously, I read that _journal.json file from disk when the server started up using Fs.readFileSync. But I’m trying to get all idiomatic with Effect to really learn the system, so I ended up with something like this to replicate the first code snippet:

Effect.gen(function* () {
const fs = yield* FileSystem.FileSystem;
const content = yield* fs.readFileString("./db/meta/_journal.json", "utf8");
const parsed = yield* safeJsonParse(content);
const validated = yield* Schema.decodeUnknownEither(Migrations)(parsed);
const when = validated.at(-1).when;
return when;
})

Beautiful, probably not, but it does the job. The problem is that if I run this effect, it’ll keep reading from the filesystem every time that the healthcheck runs. We don’t want or need that - the ./db/meta/_journal.json file isn’t going to change. So how do I cache this thing?

This is where I got a little caught up in the sauce: the Caching Effects documentation shows a lot of nifty built-in ways to cache effects, but the examples show them using Effect.cached inside of each Effect.gen call. Which means that they’re creating a new cached version of the Effect every time. I don’t want that: I want a cached version that will be cached between runs.

I went down a small rabbit hole writing the obvious next thing:

const getWhenCached = Effect.cached(
Effect.gen(function* () {
const fs = yield* FileSystem.FileSystem;
const content = yield* fs.readFileString("./db/meta/_journal.json", "utf8");
const parsed = yield* safeJsonParse(content);
const validated = yield* Schema.decodeUnknownEither(Migrations)(parsed);
const when = validated.at(-1)!.when;
return when;
})
);

But this is an effect that produces an effect. The type of getWhenCached looks like

Effect.Effect<
Effect.Effect<
number,
ParseError | ParseError | PlatformError, FileSystem.FileSystem
>,
never, never>

This isn’t what I want: I don’t want a cached function factory, I want the cached function. It took me a while to figure out that in this case I probably want to use Effect.runSync, which runs the Effect.cached layer and gives me the result. Then I can refactor from Effect.cached(Effect.gen(…)) into Effect.gen(…).pipe(Effect.cached) to make it look cooler, and tada:

const getWhen = Effect.runSync(
Effect.gen(function* () {
const fs = yield* FileSystem.FileSystem;
const content = yield* fs.readFileString("./db/meta/_journal.json", "utf8");
const parsed = yield* safeJsonParse(content);
const validated = yield* Schema.decodeUnknownEither(Migrations)(parsed);
const when = validated.at(-1)!.when;
return when;
}).pipe(Effect.cached)
);

The lesson: Effect.runSync is a useful little tool for running effects ahead of time.

• General vibe: once everything is in place, I really like how Effect code can be both terse and also ‘correct’ in terms of error handling, timeouts, retries, etc.
• Bugs today: yep, looks like I found a bug in db.execute for Effect’s drizzle integration.