Fine-tuning a DistilBERT classifier with numerical and text inputs (opens in new tab)

Text classification is often done through fine-tuning of a pretrained foundation model with domain-specific data. In FreeAgent we use transformer based models to automatically classify incoming bank transactions. Specifically we use a DistilBERT model that is fine-tuned on hundreds of millions of bank transactions with customer-labelled accounting categories. The model inputs are currently text-based, built […]