Published on November 21, 2025 3:07 AM GMT

Let’s put aside the many, many philosophical confusions about what “preferences” even are.  There’s a very basic sense that preferences are confusing even in practice.  It’s that for most things that you could have preferences about, you probably don’t know what your preferences about those things are, and you don’t have any reliable way of introspecting upon yourself to figure them out.

Let’s take a very basic example: food.  Even more narrowly, let’s consider only your reflexive (system 1) taste-sensation-preferences about food.  For the sake of this hypothetical, let’s use “streamed broccoli”^[1].  For that food item, when you put it into your mouth, is your immediate, instinctual reaction “good” or “bad”?  (Let’s call this a fast-preference, moving forward.)

I claim that knowing the answer to the previous question is a way in which you do know what at least one of your preferences are.

Now let’s consider a food item that you haven’t had before.  I will use the example of fresh durian, since I only tried it for the first time a couple weeks ago^[2].  (I recommend picking one that is reputed to share as little in common with foods that you’re familiar with as possible.)

I claim that you don’t “know” what your fast-preference for that food item will be.  You certainly have some priors, established from eating many different foods in the past, so you might be reasonably well-calibrated if you make a prediction, but for a food item that’s novel in the relevant sensory dimensions, I think you will have a hard time doing much better than a naive baseline of “what percentage of foods have I liked in the past”, purely by introspection^[3].

Now consider that understanding your own fast-preferences about foods is playing on easy mode in a bunch of ways:

• It’s pretty low-dimensional.
• The valence of this particular kind of preference is highly legible to most people - you’re not very likely to be uncertain about whether you liked a thing or not^[4].
• The feedback is unusually direct, distinct, and immediate.
• You have a pretty large sample size.
• It’s (relatively) easy to disambiguate from your other preferences about food, like “how much do I morally endorse eating this food^[5]”, or “how much do I endorse eating this food, given how I expect eating it to make me feel later”, or “how much do I appreciate the artistry of this food”, etc.

Most kinds of preferences you have over most other salient features of your life are way harder to understand than your fast-preferences about food.

Now let’s consider the referent^[6] preference people are pointing at when they ask if you “like” doing your job or not:

• It’s got much larger dimensionality than “fast-preference about food item [x]”, and you could decompose it into a much larger number of smaller preferences than you’d be able to do for the food fast-preference.
• It’s not totally obvious what you’re supposed to be paying attention to, if you’re trying to figure out whether you “like” or “dislike” doing your job.  You don’t have a custom-built sensory organ that evolution’s spent a lot of time integrating with your brain’s reward system, specifically for interacting with your job.
• As a result, the feedback you have access to is quite indirect, often difficult to disentangle from other experiences or causes of feelings you’re having, and in ambiguous cases can be difficult to evaluate on timescales shorter than “months”.  Is it your job making your tired, or the mold that’s secretly growing in your apartment?  Are you depressed because your boss’s expectations are unreasonable, or because you stopped going to the gym a month ago?  Who knows!
• You have a very small sample size.
• Related preferences, like “how much do I morally endorse doing this job”, empirically seem much more likely to directly affect people’s senses of how much they like doing their job, independent of other factors, but not always in ways that are obvious to them.

And now you’re telling me that I’m supposed to figure out what kinds of people I like?  Seems rough.

1. If you’ve never had steamed broccoli before, substitute any food that you’re familiar with.

2. As above, you may have to pick your own food item here.

3. You might be able to do better by asking other people with preferences similar to yours, of course.

4. Your preference for a specific food item might change over time, but at any given moment when you’re eating that food item, I think you’re unlikely to be in a state of substantial uncertainty about whether you like it or not.

5. Some people I know claim that such considerations feed directly into a visceral disgust response when eating a food.  Putting aside the fact that this is contingent on them knowing a specific fact while eating the food, I think that there is not much of a relationship for the overwhelming majority of people.

6. Let’s also pretend that all of these people are, in fact, pointing at the same thing… hah.