Immortalizing the words of a dead bird.

A theme in nature documentaries is that one creature’s death is new life for others.

I recently followed another dead-end Twitter link and realized anyone with a website could self-host their tweet archive and make them available however they want. I made a weekend project out of publishing mine.

They’re your words and they always were. Why not own where they live?

What if we all did?

I generated a Twitter account export in late 2022 just before I deleted it. I probably owe the ability to GDPR, and it was a convenient, simple thing to do. The archive is a ZIP file with a bunch of JavaScript, media assets, and a nice HTML page for browsing everything.

I opened the HTML page again and wondered what it would take to re-expose the internet to so many years of unnecessary thoughts.

It didn’t take much, and it was a great way to spend more time getting comfortable with TypeScript.1

I created a new .astro page on my site and used the twitter-archive-reader package to read the ZIP file and give back an archive class with everything parsed and ready to browse—including TypeScript definitions to make it fun.

import TwitterArchive from "twitter-archive-reader"

const archive = new TwitterArchive("my-archive.zip")

await archive.ready()

You don’t need to use a library to do this, especially if you’re only after the tweets. (There’s much more in the archive!) Your ZIP file includes data/tweets.js, which is an almost a perfect JSON blob except for the very beginning:

window.YTD.tweets.part0 = [
// ...

Remove window.YTD.tweets.part0 = and you’ve got a stew going JSON you can import and play with.

It’s an array of objects that represent tweets. Here’s one I’ve purposefully selected because it has an image in it:

{
"tweet": {
"edit_info": {
"initial": {
"editTweetIds": ["691695276218683392"],
"editableUntil": "2016-01-25T19:22:59.584Z",
"editsRemaining": "5",
"isEditEligible": true
}
},
"retweeted": false,
"source": "<a href=\"http://twitter.com\" rel=\"nofollow\">Twitter Web Client</a>",
"entities": {
"user_mentions": [],
"urls": [],
"symbols": [],
"media": [
{
"expanded_url": "https://twitter.com/mattrambles/status/691695276218683392/photo/1",
"indices": ["77", "100"],
"url": "https://t.co/FkWk3HumW1",
"media_url": "http://pbs.twimg.com/media/CZllMEJUYAAFsML.png",
"id_str": "691695275723743232",
"id": "691695275723743232",
"media_url_https": "https://pbs.twimg.com/media/CZllMEJUYAAFsML.png",
"sizes": {
"medium": {
"w": "303",
"h": "305",
"resize": "fit"
},
"small": {
"w": "303",
"h": "305",
"resize": "fit"
},
"thumb": {
"w": "150",
"h": "150",
"resize": "crop"
},
"large": {
"w": "303",
"h": "305",
"resize": "fit"
}
},
"type": "photo",
"display_url": "pic.twitter.com/FkWk3HumW1"
}
],
"hashtags": []
},
"display_text_range": ["0", "100"],
"favorite_count": "0",
"id_str": "691695276218683392",
"truncated": false,
"retweet_count": "0",
"id": "691695276218683392",
"possibly_sensitive": false,
"created_at": "Mon Jan 25 18:52:59 +0000 2016",
"favorited": false,
"full_text": "At what point do higher-than-normal call volumes simply become normal again? https://t.co/FkWk3HumW1",
"lang": "en",
"extended_entities": {
"media": [
{
"expanded_url": "https://twitter.com/mattrambles/status/691695276218683392/photo/1",
"indices": ["77", "100"],
"url": "https://t.co/FkWk3HumW1",
"media_url": "http://pbs.twimg.com/media/CZllMEJUYAAFsML.png",
"id_str": "691695275723743232",
"id": "691695275723743232",
"media_url_https": "https://pbs.twimg.com/media/CZllMEJUYAAFsML.png",
"sizes": {
"medium": {
"w": "303",
"h": "305",
"resize": "fit"
},
"small": {
"w": "303",
"h": "305",
"resize": "fit"
},
"thumb": {
"w": "150",
"h": "150",
"resize": "crop"
},
"large": {
"w": "303",
"h": "305",
"resize": "fit"
}
},
"type": "photo",
"display_url": "pic.twitter.com/FkWk3HumW1"
}
]
}
}
}

While the library was handy and simple to work with, it seemed excessive to commit the ZIP archive into my Astro project. I used the archive reader package temporarily to spit out JSON exactly how I wanted it.

Keeping a JSON blob around as a data source is better than a one-and-done HTML page because I can separate content from presentation with clean stylistic control, the ability to add stats or little features, and better odds of surviving a replatform or redesign.

The result for me is a directory of images and video attachments that came straight from the ZIP file (data/tweets_media/), along with my pared-down JSON where the same tweet looks like this:

{
"id": "691695276218683392",
"body": "At what point do higher-than-normal call volumes simply become normal again? ",
"media": [
{
"kind": "image",
"src": "/images/twitter/media/691695276218683392-CZllMEJUYAAFsML.png"
}
],
"created": "Mon Jan 25 18:52:59 +0000 2016",
"type": "tweet",
"retweets": 0,
"favorites": 0
}

I only a cheated a little bit with some in-between cleanup tasks:

1. Limited image and video references to those I actually have locally.
2. Pulled image and video references out of the tweet content and exclusively into a simpler media array.
3. Un-shortened URLs with tall.
4. Made up my own type property (tweet, retweet, or reply) for easier filtering.
5. Downloaded and replaced the handful of Twitpic images I had.

I’m loving TypeScript (with somebody else’s tsconfig.json), and my cleaned-up data uses these little types:

src/types.ts

type TweetImage = {
kind: "image"
src: string
}

type TweetVideo = {
kind: "video"
src: string
contentType: string
gif: boolean
}

type Tweet = {
id: string
body: string
media: Array<TweetImage | TweetVideo>
created: string
type: "tweet" | "retweet" | "reply"
retweets: number
favorites: number
}

The page loads up the JSON, limits the data to tweets (no replies or retweets), and dumps out an obscenely-long pile of markup:

src/pages/twitter.astro

---
import Breadcrumb from "@components/Breadcrumb.astro"
import Layout from "@layouts/Layout.astro"
import PostBody from "@components/PostBody.astro"
import PostTitle from "@components/PostTitle.astro"
import tweetData from "src/data/tweets.json"
import Tweet from "@components/Tweet.astro"
const tweets: Tweet[] = tweetData as Tweet[]
const pageTitle = "Twitter Archive"
---
<Layout title={pageTitle} description={`Matt’s 2008–2022 tweet archive.`}>
<Breadcrumb items={[{ label: pageTitle, current: true }]} />
<main>
<PostTitle title={pageTitle} />
<p class="text-xl my-12">
Arrived in 2008, said some forgettable things, and left in 2022.
</p>
<div class="wrapper">
{
tweets
.filter((tweet: Tweet) => {
return tweet.type == "tweet"
})
.map((tweet: Tweet) => {
return (
<Tweet
data={tweet}
name="Matt Stein"
handle="mattrambles"
avatarUrl="/images/twitter/avatar.jpg"
/>
)
})
}
</div>
<PostBody />
</main>
</Layout>

The Tweet component renders each random thought:

src/components/Tweet.astro

---
import { formatDate } from "../utils/content"
import {
ArrowPathRoundedSquareIcon,
HeartIcon
} from "@heroicons/vue/24/outline/index.js"
export interface Props {
data: Tweet
name: string
handle: string
avatarUrl: string
}
const {
data: {
id,
created,
body,
media,
retweets,
favorites
},
name,
handle,
avatarUrl
} = Astro.props
---
<div
id={`t` + id}
class={`tweet flex my-12 space-x-3 max-w-xl overflow-x-scroll`}
>
<div class="avatar flex-none w-12 h-12 rounded-full overflow-hidden">
<img src={avatarUrl} alt="" />
</div>
<div class="content -mt-1">
<div class="top">
<b>{name}</b>{" "}
<span>
<span class="opacity-50">@{handle}</span>
<span class="opacity-25 hidden sm:inline"> •</span>
<span class="opacity-50 block mb-3 sm:mb-0 sm:inline"
>{formatDate(created)}</span
>
</span>
</div>
<div class="body">
<div set:html={body} />
{
media.map((mediaItem) => {
return (
<>
{mediaItem.kind == "image" && (
<img
src={mediaItem.src}
class="block rounded-xl mt-2 shadow-xs max-w-full"
loading="lazy"
/>
)}
{mediaItem.kind == "video" && (
<video
class={`rounded-xl mt-2 shadow-xs`}
autoplay={mediaItem.gif}
muted={mediaItem.gif}
loop={mediaItem.gif}
controls={!mediaItem.gif}
>
<source src={mediaItem.src} type={mediaItem.contentType} />
</video>
)}
</>
)
})
}
</div>
<div class="meta mt-3 flex text-sm opacity-75">
<div class="retweets">
<ArrowPathRoundedSquareIcon
class="w-4 h-4 inline-block -mt-0.5 mr-0.5"
/>
{retweets}
</div>
<div class="favorites ml-4">
<HeartIcon class="w-4 h-4 inline-block -mt-0.5 mr-0.5" />
{favorites}
</div>
</div>
</div>
</div>

There’s plenty of room here to come back and add stuff like stats, filters, civilized lazyloading, and navigation. You could surely do something cooler!

That’s the point: you can do whatever you want when it’s your data.

Footnotes

I spent hours making a cleaned-up example repository for you, but couldn’t manage to configure TypeScript to run it with the handful of imports I used. If anybody can help me iron it out I’d be happy to publish a working example. (Everything ran in an Astro page just fine.) ↩