🤖 Transformer Architecture - tomasz · Scour

STS: Efficient Sparse Attention with Speculative Token Sparsity 📊TF-IDF

The Expressive Power of Low Precision Softmax Transformers with (Summarized) Chain-of-Thought 🔢Kolmogorov Complexity

SparseSAM: Structured Sparsification of Activations in Segment Anything Models 👁️Computer Vision

GiLT: Augmenting Transformer Language Models with Dependency Graphs 🔗RAG

Geometric Factual Recall in Transformers 🔢Kolmogorov Complexity

RoiMAM: Region-of-Interest Medical Attention Model for Efficient Vision-Language Understanding 👁️Computer Vision

GQLA: Group-Query Latent Attention for Hardware-Adaptive Large Language Model Decoding 🔢Kolmogorov Complexity

Transformer Interpretability from Perspective of Attention and Gradient 👁️Computer Vision

Variational Linear Attention: Stable Associative Memory for Long-Context Transformers 🔢Kolmogorov Complexity

Representative Attention For Vision Transformers 👁️Computer Vision

Clustering in pure-attention hardmax transformers and its role in sentiment analysis 🔢Kolmogorov Complexity

DemaFormer: Damped Exponential Moving Average Transformer with Energy-Based Modeling for Temporal Language Grounding 💬Natural Language Processing

Elastic Attention Cores for Scalable Vision Transformers 👁️Computer Vision

AttnGen: Attention-Guided Saliency Learning for Interpretable Genomic Sequence Classification 🔢Kolmogorov Complexity

Latent Chain-of-Thought Improves Structured-Data Transformers 🧠LLM Reasoning

Breaking Global Self-Attention Bottlenecks in Transformer-based Spiking Neural Networks with Local Structure-Aware Self-Attention 🎭Anthropic Claude

Conditional Attribute Estimation with Autoregressive Sequence Models 🔢Kolmogorov Complexity

ECG-NAT: A Self-supervised Neighborhood Attention Transformer for Multi-lead Electrocardiogram Classification 🔍Vector Search

Pretraining Language Models with Subword Regularization: An Empirical Study of BPE Dropout in Low-Resource NLP ✂️Tokenization

A Composite Activation Function for Learning Stable Binary Representations 🔍Vector Search

Log in to enable infinite scrolling