Tokenization

Feeds to Scour
SubscribedAll
Scoured 20 posts in 9.4 ms

What Are Tokens in LLMs?

 💬Natural Language Processing  Content type: Blog

A system programmer’s guide to LLM inference

 💬Natural Language Processing  Content type: Blog

lbj96347/nemotron-3.5-asr-ios: On-device, offline speech recognition for iPhone/iPad using NVIDIA's Nemotron-3.5-ASR Streaming 0.6B (multilingual) via CoreML.SwiftUI app with mic capture + audio file import, RNN-Tdecoding, and live benchmark metrics (latency, RTF, memory).

 💬Natural Language Processing  Content type: Code
github.com··Hacker News

Vibe Diaries: Training Nanochat

 💬Natural Language Processing

LDARNet: DNA Adaptive Representation Network with Learnable Tokenization for Genomic Modeling

 💬Natural Language Processing  Content type: Academic
arxiv.org·

Security updates for Tuesday [LWN.net]

 🏭Code Generation
lwn.net·

Train your own GPT-2 (124M).

 🤖Transformer Architecture  Content type: Blog
medium.com·

How Far Apart Does a Model Think Its Tokens Are?

 🔢Kolmogorov Complexity
lesswrong.com·

What an LLM Actually Does With Your Prompt First

 💬Natural Language Processing
siliconopera.com·
Less-relevant results

Building Java Containers Without a Dockerfile: Azul Zulu and Paketo Buildpacks

 🏗️Build Systems  Content type: Blog
azul.com·

Architecting the Control Plane for Intelligence: System Design of an Enterprise AI Gateway

 🏗️Systems Design  Content type: Blog
medium.com·

How LLMs work | Practical Leaders

 💬Natural Language Processing

Run an Apache Airflow DAG with Docker Compose and PostgreSQL

 ⚙️Data Engineering
pyimagesearch.com·

Microsoft Restores Some GitHub Repos, Keeps Others Offline as Miasma Probe Continues

 🤖AI Agents
thehackernews.com·

SniperRavan/Trace: Ambient AI usage tracker extension for ChatGPT, Claude, Gemini, Grok, and Perplexity.

 🤖Local LLMs  Content type: Code
github.com··r/SideProject

I designed a 0.9B Mamba-2 / GLA hybrid LLM — the AI agents wrote the code. An honest build log.

 ⚙️Data Engineering  Content type: Code
github.com··DEV

maziyarpanahi/openmed: open-source healthcare ai

 🧠Deep Learning  Content type: Code
github.com·

TilelliLab/atome-lm: A ternary, zero-heap tiny language model that runs inside a $2 microcontroller — bit-exact Python <-> C99 <-> Cortex-M3 (QEMU) parity. Apache-2.0.

 🤖Machine Learning  Content type: Code
github.com··r/LLM

Release v0.15.1 · NVIDIA/garak

 🧠Deep Learning  Content type: Code
github.com·

Keyboard Shortcuts

Navigation

Next / previous item
j/k
Open post
oorEnter
Preview post
v

Post Actions

Love post
a
Like post
l
Dislike post
d
Undo reaction
u
Save / unsave
s

Recommendations

Add interest / feed
Enter
Not interested
x

Go to

Home
gh
Interests
gi
Feeds
gf
Likes
gl
History
gy
Changelog
gc
Settings
gs
Browse
gb
Search
/

General

Show this help
?
Submit feedback
!
Close modal / unfocus
Esc

Press ? anytime to show this help