Tokenization

Feeds to Scour
SubscribedAll
Scoured 39 posts in 9.2 ms

What Are Tokens in LLMs?

 🤖LLM  Content type: Blog
bearisland.dev··Hacker News

Finding Optimal Tokenizers

 💬Natural Language Processing  Content type: Blog
blog.aqnichol.com··Hacker News

Augmenting Molecular Language Models with Local $n$-gram Memory

 💬Natural Language Processing  Content type: Academic
arxiv.org·

[NEW MODEL] SupraLabs just released Supra1.5-50M Base (Experimental)!

 🎯Fine-tuning
huggingface.co··r/LocalLLaMA

smile/deep at master · haifengl/smile

 🔥PyTorch  Content type: Code
github.com··Hacker News

A system programmer’s guide to LLM inference

 🧠LLM Inference  Content type: Blog
blog.xiangpeng.systems··Hacker News

Making a Vintage LLM from Scratch

 💬Natural Language Processing
crlf.link··Hacker News

Show HN: In-browser real LLM token counter and cost estimation

 🪟Context Windows
holaclaw.ai··Hacker News

Vibe Diaries: Training Nanochat

 💬Natural Language Processing
vibediary.dev··Hacker News

Agentic RL: Token-In, Token-Out Done Right

 💬Natural Language Processing
qgallouedec-tito.hf.space··Hacker News

No Token Left Behind: Demystifying Token-in-Token-Out in Miles

 🧠LLM Inference  Content type: Blog
lmsys.org··Hacker News

Arithmetic Without Numbers – How LLMs Do Math

 🧠LLMs
alvaro-videla.com··Hacker News

Apple rebuilt its on-device AI stack at WWDC 2026

 🤖Machine Learning  Content type: Blog
ziraph.com··Hacker News

How LLMs Actually Work: A Friendly Map for Humans • oreoro

 💬Natural Language Processing
oreoro.github.io··Hacker News

End-to-End Training for Discrete Token LLM based TTS System

 🤖LLM  Content type: Academic
arxiv.org·

1-bit and 1.58 bit LLM Benchmarking on Jetson Orin Nano Super | Bonsai LM

 🤖Data science
smolhub.com··r/LocalLLaMA

scottpurdy/llmbuffer: LLM conversation buffer with cache optimization and dynamic context.

 🪟Context Windows  Content type: Code
github.com··Hacker News, Hacker News

Show HN: Bosun – a small model that keeps an agent's memory graph clean

 🎯Fine-tuning
huggingface.co··Hacker News

The Token at the Seam – Learning in Robotics

 💬Natural Language Processing  Content type: News  Content type: Blog
atomsfrontier.substack.com··Substack

Breaking the Tokenizer Barrier: On-Policy Distillation across Model Families

 🎯Fine-tuning  Content type: Academic
arxiv.org·

Keyboard Shortcuts

Navigation

Next / previous item
j/k
Open post
oorEnter
Preview post
v

Post Actions

Love post
a
Like post
l
Dislike post
d
Undo reaction
u
Save / unsave
s

Recommendations

Add interest / feed
Enter
Not interested
x

Go to

Home
gh
Interests
gi
Feeds
gf
Likes
gl
History
gy
Changelog
gc
Settings
gs
Browse
gb
Search
/

General

Show this help
?
Submit feedback
!
Close modal / unfocus
Esc

Press ? anytime to show this help