The whole ebook/ereader ecosystem just bums me out whenever I think about it.

For instance, there is a “Kindle Notebook” feature which saves highlights and notes you make in Kindle books, but getting these annotations out of the Kindle is a humiliating experience. You have to either log in to the Kindle Cloud Reader web application and pull the highlights from JavaScript, or log in to a Kindle app and email yourself an HTML file. Each of these options require individually navigating to a book and going through steps in a GUI. (You used to have a third sad option, to connect your Kindle with a cable (!) and copy a text file off of it, but this is no longer possible with modern Kindles.)

The methods all produce different results, of course. Highlights for books you bought from the Kindle store sync to the Kindle Cloud Reader, but highlights from books you add outside of the Kindle store sync only between native apps (e.g. macOS and iOS; I assume Android and Windows too). Via Bookcision you can get structured data from the Cloud Reader, but the official export from Kindle comes only in HTML that doesn’t even associate a user’s notes with highlights from the text of the book.

And you might think, well, highlights and notes in physical books are also sort of constrained to the medium, maybe it makes sense that Amazon constrains Kindle highlights and notes to their ecosystem as well? But this is unsatisfying for two reasons:

1. It’s an artificial limitation by Amazon that restricts users, not an inherent limitation of the technology
2. Physical books have the advantage of being physically manipulable: you can add sticky notes as tabs and flip through pages very quickly to find annotations, but this is extremely inconvenient for ebooks

Book publishers also do their best to become a villain in this story (although as usual they’re too insignificant a player for anyone to notice): they set a limit on the amount of a book you can highlight, and if you highlight more than this amount, exporting your highlights will be truncated. One might think this is a reasonable compromise as long as its set high enough, because someone could extract the whole text of a book this way, but only if they hadn’t thought about this issue even a tiny bit. For a user to accomplish this, they’d have to painstakingly highlight the entire book across thousands of page turns, and what they’d have to show for it is a giant text blob stripped of newlines and formatting. Why would anyone do this in a world where Anna’s Archive already has a full fidelity copy of the book for free?

In my limited experience, this doesn’t matter; books seem to allow highlighting at least 10% of total content, and you probably can’t make use of even that amount. Still, it’s a pathetic waste of everyone’s time, and Amazon has managed to fuck up its implementation between apps, showing different values for the amount you’re allowed to export depending on whether you’re trying to do it from the Cloud Reader or a native app.

---

Anyway, I have added Kindle Notebook exports to relevant pages in the books section. The site can now parse exports from both Bookcision and the Kindle apps.

Like the rest of the books section, I think this will be more useful to me than to anyone else. Highlights aren’t really valuable on their own, but my highlights help remind me of something that I remember was important. It’ll be nice to be able to find them easily with cmd-f.

Here’s how it works under the hood.

The bookcision.html partial parses Bookcision exports

{{- range $index, $hilite := .highlights -}}
<figure>
{{- with $hilite.text }}
<blockquote>{{ . }}</blockquote>
{{- end }}

{{- with $hilite.note }}
<aside class="note">{{ . }}</aside>
{{- end }}

{{- with $hilite.location -}}
<figcaption>
Loc {{ .value }}
</figcaption>
{{- end -}}
</figure>
{{- end -}}

The kindleNotebook.html partial parses exports from the Kindle apps

This is a horrible partial that parses HTML with regex, cobbled together with ChatGPT. If Amazon ever changes their export format, it’ll break.

{{/* Usage: {{ partial "bookKindleNotebook.html" (dict "res" $kindleResource) }} */}}
{{- $res := .res -}}
{{- $raw := readFile $res.File.Filename -}}

{{/* Match:
- <div class="sectionHeading">Section</div>
- <div class="noteHeading">Heading</div><div class="noteText">Text</div>
*/}}
{{- $re := `(?s)<div class="(sectionHeading|noteHeading)">\s*(.+?)\s*</div>(?:\s*<div class="noteText">\s*(.+?)\s*</div>)?` -}}
{{- $matches := findRE $re $raw -}}
{{- $count := len $matches -}}

{{- $consumed := newScratch -}}

<article class="book-notes">
{{- range $i, $m := $matches }}
{{- $idx := printf "%d" $i -}}

{{- if not ($consumed.Get $idx) }}
{{- $class := replaceRE $re "$1" $m -}}
{{- $content1 := replaceRE $re "$2" $m -}}
{{- $content2 := replaceRE $re "$3" $m | default "" -}}

{{- if eq $class "sectionHeading" }}
<h3>{{ $content1 | safeHTML }}</h3>

{{- else if eq $class "noteHeading" }}
{{- $heading := $content1 -}}
{{- $text := $content2 -}}

{{- $isHighlight := in $heading "Highlight(" -}}
{{- $isNoteHeading := and (not $isHighlight) (hasPrefix $heading "Note -") -}}

{{- if $isHighlight }}
{{- $page := replaceRE `(?s).*Page\s+([0-9]+).*` "$1" $heading | default "" -}}
{{- $loc  := replaceRE `(?s).*Location\s+([0-9]+).*` "$1" $heading | default "" -}}

{{- $noteText := "" -}}

{{/* Look ahead for an attached Note entry */}}
{{- $nextIndex := add $i 1 -}}
{{- if lt $nextIndex $count }}
{{- $next := index $matches $nextIndex -}}
{{- $nextClass := replaceRE $re "$1" $next -}}
{{- $nextHeading := replaceRE $re "$2" $next -}}
{{- $nextBody := replaceRE $re "$3" $next | default "" -}}

{{- $nextIsNote := and (eq $nextClass "noteHeading") (hasPrefix $nextHeading "Note -") -}}
{{- if $nextIsNote }}
{{- $notePage := replaceRE `(?s).*Page\s+([0-9]+).*` "$1" $nextHeading | default "" -}}
{{- $noteLoc  := replaceRE `(?s).*Location\s+([0-9]+).*` "$1" $nextHeading | default "" -}}

{{/* Heuristic:
- Page matches or is missing on one side
- Location equal or slightly ahead (<= 10) if both present
*/}}
{{- $pageOK := or (eq $page "") (eq $notePage "") (eq $page $notePage) -}}
{{- $locOK := or
(eq $loc "")
(eq $noteLoc "")
(and
(ne $loc "")
(ne $noteLoc "")
(ge (sub (int $noteLoc) (int $loc)) 0)
(le (sub (int $noteLoc) (int $loc)) 10)
)
-}}

{{- if and $pageOK $locOK }}
{{- $noteText = $nextBody -}}
{{- $consumed.Set (printf "%d" $nextIndex) true -}}
{{- end -}}
{{- end -}}
{{- end -}}

<figure {{ with $page }}data-page="{{ . }}"{{ end }} {{ with $loc }}data-location="{{ . }}"{{ end }}>
{{- if $text }}
<blockquote>{{ $text | safeHTML }}</blockquote>
{{- end }}
{{- if $noteText }}
<aside>{{ $noteText | safeHTML }}</aside>
{{- end }}
<figcaption>
{{- with $page }}Page {{ . }}{{ end -}}
{{- if and $page $loc }}, {{ end -}}
{{- with $loc }}Location {{ . }}{{ end -}}
</figcaption>
</figure>

{{- else if $isNoteHeading }}
{{/* Standalone note not paired to a highlight */}}
{{- $page := replaceRE `(?s).*Page\s+([0-9]+).*` "$1" $heading | default "" -}}
{{- $loc  := replaceRE `(?s).*Location\s+([0-9]+).*` "$1" $heading | default "" -}}

<figure class="note-only" {{ with $page }}data-page="{{ . }}"{{ end }} {{ with $loc }}data-location="{{ . }}"{{ end }}>
{{- if $text }}
<aside>{{ $text | safeHTML }}</aside>
{{- end }}
<figcaption>
{{- with $page }}Page {{ . }}{{ end -}}
{{- if and $page $loc }}, {{ end -}}
{{- with $loc }}Location {{ . }}{{ end -}}
</figcaption>
</figure>

{{- else }}
{{/* Fallback for any odd noteHeading formats */}}
<figure>
{{- if $text }}
<blockquote>{{ $text | safeHTML }}</blockquote>
{{- end }}
</figure>
{{- end }}
{{- end }}
{{- end }}
{{- end }}
</article>

The template for individual book pages calls those partials when data is present Example bookcision.json file

A simplified version of the template for the book pages.

{"asin":"B00KWG9M2E","title":"Getting Things Done: The Art of Stress-Free Productivity","authors":"David Allen and James Fallows","highlights":[{"text":"Why Things Are on Your Mind Most often, the reason something is on your mind is that you want it to be different than it currently is, and yet: you haven’t clarified exactly what the intended outcome is; This consistent, unproductive preoccupation with all the things we have to do is the single largest consumer of time and energy. —Kerry Gleeson you haven’t decided what the very next physical action step is; and/or you haven’t put reminders of the outcome and the action required in a system you trust. That’s why it’s on your mind. Until those thoughts have been clarified and those decisions made, and the resulting data has been stored in a system that you absolutely know you will access and think about when you need to, your brain can’t give up the job. You can fool everyone else, but you can’t fool your own mind. It knows whether or not you’ve come to the conclusions you need to, and whether you’ve put the resulting outcomes and action reminders in a place that can be trusted to resurface appropriately within your conscious mind.* If you haven’t done those things, it won’t quit working overtime. Even if you’ve already decided on the next step you’ll take to resolve a problem, your mind can’t let go until and unless you park a reminder in a place it knows you will, without fail, look. It will keep pressuring you about that untaken next step, usually when you can’t do anything about it, which will just add to your stress.","isNoteOnly":false,"location":{"url":"kindle://book?action=open&asin=B00KWG9M2E&location=652","value":652},"note":null},{"text":"Before you can achieve any of that, though, you’ll need to get in the habit of keeping nothing on your mind. And the way to do that, as we’ve seen, is not by managing time, managing information, or managing priorities. After all: you don’t manage five minutes and wind up with six; you don’t manage information overload—otherwise you’d walk into a library and die, or the first time you connected to the Web, you’d blow up; and you don’t manage priorities—you have them. Instead, the key to managing all of your stuff is managing your actions.","isNoteOnly":false,"location":{"url":"kindle://book?action=open&asin=B00KWG9M2E&location=711","value":711},"note":null},{"text":"There is no reason to ever have the same thought twice, unless you like having that thought.","isNoteOnly":false,"location":{"url":"kindle://book?action=open&asin=B00KWG9M2E&location=784","value":784},"note":null},{"text":"We (1) capture what has our attention; (2) clarify what each item means and what to do about it; (3) organize the results, which presents the options we (4) reflect on, which we then choose to (5) engage with.","isNoteOnly":false,"location":{"url":"kindle://book?action=open&asin=B00KWG9M2E&location=833","value":833},"note":null},{"text":"Still others have good systems but don’t (4) reflect on the contents consistently enough to keep them functional. They may have lists, plans, and various checklists available to them (created by capturing, clarifying, and organizing), but they don’t keep them current or access them to their advantage. Many people don’t look ahead at their own calendars consistently enough to stay current about upcoming events and deadlines, and they consequently become victims of last-minute craziness.","isNoteOnly":false,"location":{"url":"kindle://book?action=open&asin=B00KWG9M2E&location=857","value":857},"note":"Reflecting requires upkeep - there is a cost to this system. I never wanted to pay that cost but now is the time to embrace it."}]}

Example kindle-notebook.html file

A simplified version of the template for the book pages.

<!DOCTYPE html PUBLIC
"-//W3C//DTD XHTML 1.0 Strict//EN"
"XHTML1-s.dtd" >
<html xmlns="http://www.w3.org/TR/1999/REC-html-in-xml" xml:lang="en" lang="en">
<head>
<meta charset="UTF-8">
<style>
.bodyContainer {
font-family: Arial, Helvetica, sans-serif;
text-align: center;
padding-left: 32px;
padding-right: 32px;
}

.notebookFor {
font-size: 18px;
font-weight: 700;
text-align: center;
color: rgb(119, 119, 119);
margin: 24px 0px 0px;
padding: 0px;
}

.bookTitle {
font-size: 24px;
font-weight: 700;
text-align: center;
color: #333333;
margin-top: 22px;
padding: 0px;
}

.authors {
font-size: 18px;
font-weight: 700;
text-align: center;
color: rgb(119, 119, 119);
margin-top: 22px;
margin-bottom: 24px;
padding: 0px;
}

.citation {
font-size: 18px;
font-weight: 500;
text-align: center;
color: #333333;
margin-top: 22px;
margin-bottom: 24px;
padding: 0px;
}

.sectionHeading {
font-size: 24px;
font-weight: 700;
text-align: left;
color: #333333;
margin-top: 24px;
padding: 0px;
}

.noteHeading {
font-size: 18px;
font-weight: 700;
text-align: left;
color: #333333;
margin-top: 20px;
padding: 0px;
}

.noteText {
font-size: 18px;
font-weight: 500;
text-align: left;
color: #333333;
margin: 2px 0px 0px;
padding: 0px;
}

.highlight_blue {
color: rgb(178, 205, 251);
}

.highlight_orange {
color: #ffd7ae;
}

.highlight_pink {
color: rgb(255, 191, 206);
}

.highlight_yellow {
color: rgb(247, 206, 0);
}

.notebookGraphic {
margin-top: 10px;
text-align: left;
}

.notebookGraphic img {
-o-box-shadow:      0px 0px 5px #888;
-icab-box-shadow:   0px 0px 5px #888;
-khtml-box-shadow:  0px 0px 5px #888;
-moz-box-shadow:    0px 0px 5px #888;
-webkit-box-shadow: 0px 0px 5px #888;
box-shadow:         0px 0px 5px #888;
max-width: 100%;
height: auto;
}

hr {
border: 0px none;
height: 1px;
background: none repeat scroll 0% 0% rgb(221, 221, 221);
}
</style>
<script>
</script>
</head>
<body>
<div class="bodyContainer">
<div class="notebookFor">
Notebook Export
</div>
<div class="bookTitle">
Johnstone, Keith - Impro - improvisation and the theater (1992)
</div>
<div class="authors">
Keith Johnstone
</div>
<div class="citation">

</div>
<hr />
<div class="sectionHeading">
Introduction
</div><div class="noteHeading">
Highlight(<span class="highlight_yellow">yellow</span>) - Page 9 · Location 55
</div>
<div class="noteText">
I first met Johnstone shortly after he had joined the Court as a los-a-script play-reader, and he struck me then as a revolutionary idealist looking around for a guillotine. He saw corruption everywhere.
</div><div class="sectionHeading">
Notes on Myself
</div><div class="noteHeading">
Highlight(<span class="highlight_yellow">yellow</span>) - Page 13 · Location 103
</div>
<div class="noteText">
As I grew up, everything started getting grey and dull. I could still remember the amazing intensity of the world I’d lived in as a child, but I thought the dulling of perception was an inevitable consequence of age—just as the lens of the eye is bound gradually to dim. I didn’t understand that clarity is in the mind. I’ve since found tricks that can make the world blaze up again in about fifteen seconds, and the effects last for hours. For example, if I have a group of students who are feeling fairly safe and comfortable with each other, I get them to pace about the room shouting out the wrong name for everything that their eyes light on. Maybe there’s time to shout out ten wrong names before I stop them. Then I ask whether other people look larger or smaller—almost everyone sees people as different sizes, mostly as smaller. ‘Do the outlines look sharper or more blurred?’ I ask, and everyone agrees that the outlines are many times sharper. ‘What about the colours?’ Everyone agrees there’s far more colour, and that the colours are more intense. Often the size and shape of the room will seem to have changed, too. The students are amazed that such a strong transformation can be effected by such primitive means—and especially that the effects last so long.
</div><div class="noteHeading">
Highlight(<span class="highlight_yellow">yellow</span>) - Page 13 · Location 118
</div>
<div class="noteText">
One afternoon I was lying on my bed and investigating the effects of anxiety on the musculature (how do you spend your afternoons?).
</div><div class="noteHeading">
Highlight(<span class="highlight_yellow">yellow</span>) - Page 14 · Location 135
</div>
<div class="noteText">
At about the age of nine I decided never to believe anything because it was convenient. I began reversing every statement to see if the opposite was also true. This is so much a habit with me that I hardly notice I’m doing it any more. As soon as you put a ‘not’ into an assertion, a whole range of other possibilities opens out—especially in drama, where everything is supposition anyway.
</div><div class="noteHeading">
Highlight(<span class="highlight_yellow">yellow</span>) - Page 22 · Location 285
</div>
<div class="noteText">
When I hear that children only have an attention span of ten minutes, or whatever, I’m amazed. Ten minutes is the attention span of bored children, which is what they usually are in school—hence the misbehaviour.
</div><div class="noteHeading">
Highlight(<span class="highlight_yellow">yellow</span>) - Page 28 · Location 394
</div>
<div class="noteText">
What really got me started again was an advert for a play of mine in the paper, a play called The Martian. I had never written such a play, so I phoned up Bryan King, who directed the theatre. ‘We’ve been trying to find you,’ he said. ‘We need a play for next week, does the title The Martian suit you?’
</div><div class="noteHeading">
Note - Page 28 · Location 396
</div>
<div class="noteText">
WHAT
</div>
</div>
</body>
</html>

This system expects a layout with pages like this in Hugo:

content/books/
book-name/
index.md
bookcision.json
kindle-notebook.html

Take the above and make your own!

If you’re interested in this, you might want to check out:

• clippings.io, a nice simple service for exporting Kindle annotations
• Notado, a bookmarking service that can also pull in Kindle highlights (I haven’t tried this)
• Readwise (invite link), designed around reminding you of your highlights over time
• Bookcision, a free bookmarklet from Readwise that lets you download your highlights from the web without using their paid service at all
• How to annotate literally everything, a blog post about highlighting and making notes on all kinds of content
• Exporting Kindle Highlights for Personal Documents, a blog post about the Kindle app’s process for emailing yourself an HTML file