TIL about Effect.fn

I started reading some of effect-solutions and learned about Effect.fn. This method helps with one of the core weirdnesses of Effect, that so much of the documentation doesn’t really show you functions with arguments, which are a big part of programming. So a classic Effect pattern that I’ve been using is like this:

export function downloadFile(
key: string,
location: string
): Effect.Effect<
void,
NotFoundError | PlatformError | UnknownException,
FileSystem.FileSystem
> {
return Effect.gen(function* () {
const fs = yield* FileSystem.FileSystem;
let dest = localPath(key);
yield* downloadAndFileToTemp(key);
yield* fs.copyFile(dest, location);
}).pipe(Effect.withSpan("downloadFile"));
}

That is, a function that takes arguments and then returns an Effect that uses those arguments from the function’s scope. The Effect doesn’t take arguments. This feels a little kludgy to me, and Effect.fn makes it simpler:

export const downloadFile = Effect.fn("downloadFile")(function* (
key: string,
location: string
) {
const fs = yield* FileSystem.FileSystem;
let dest = localPath(key);
yield* downloadAndFileToTemp(key);
yield* fs.copyFile(dest, location);
});

This is an improvement! Instead of manually tracing, it automatically traces, and I have one less function to declare.

There are downsides, though:

• In the first example, I defined some explicit types for the return, requirements, and errors of the effect (the part that says void, NotFoundError…). There are no examples I can find of doing the same for Effect.fn, which has a totally different call signature. I opened an issue for this one.
• If you want to pipe the function through some stuff, you need to provide a magic second argument to Effect.fn and use flow. Which is fine, but is another learning curve relative to the way that piping is taught everywhere else, using .pipe().

Unfortunately, this also led me into another battle with OpenTelemetry.

I really don’t like OpenTelemetry

Boy, I don’t like the OpenTelemetry SDK. Let me count the ways:

The scattered opentelemetry packages are the worst case of dependency hell. In our project, we have 14 OpenTelemetry dependencies (api, core, exporter-trace-otlp-proto, instrumentation-aws-sdk, instrumentation-express, instrumentation-http, instrumentation-ioredis, instrumentation-net, instrumentation-pg, resources, sdk-node, sdk-trace-base, sdk-trace-web, semantic-conventions). Great. The node_modules/@opentelemetry directory weighs 146MB. Not a great start: and some of this is due to some modules distributing triple sources - esm, esnext, and src, three copies of everything.

Then you’ve got the peer dependencies. Sentry relies heavily on OpenTelemetry now and wants to integrate tightly with it, so it has @sentry/opentelemetry which has a peerDependencies on four OpenTelemetry modules. And then @effect/opentelemetry does the same - it has peer dependencies on eight OpenTelemetry modules.

Then you’ve got the versioning scheme: OpenTelemetry’s experimental modules are versioned 0.x, so they get stricter matching behavior: specifying @opentelemetry/sdk-logs: "^0.203.0" in your package.json does not permit upgrades to 0.208.0 because it’s a 0.x package. This makes the peerDependency situation even worse because it makes it harder for two packages with peerDependencies onto sdk-logs to resolve to the same version.

And then, to be clear, the consequences of duplicate OpenTelemetry modules are not minor! If conflicting peerDependency and root requirements cause more than one copy of otel modules to be installed, you’ll get separate tracers and disconnected tracers, mucking up your whole observability situation.

Now: look at me, a big critic. Why don’t I fix this stuff? How could this be better? I don’t know exactly, but I wish that:

• They stopped using 0 versions for so many OpenTelemetry APIs.
• They stopped distributing packages in triplicate.
• OpenTelemetry packages were more macro and less micro so there were fewer of them to have to wrangle.
• NPM has better warnings and tooling for identifying when duplicate dependencies are installed.

And to be clear - I’m just talking about the SDK. The protocol itself is another matter, which David Cramer has written an extensive piece about.