MaximeRivest/tiny-classifiers
A reproducible pipeline turns LLM‑generated pseudo‑labels into a fast, cheap classifier that can out‑perform a paid API on multiple benchmarks.
The repository delivers a complete recipe for training a 17 M‑parameter text classifier in roughly one minute. Instead of hand‑curating a labeled dataset, it asks a powerful LLM to annotate raw text, then treats those pseudo‑labels as ground truth for a downstream model. The whole flow is scripted in JavaScript, making it runnable in Node or the browser with minimal setup.
At the core is a compact transformer encoder, roughly the size of a distilled BERT, implemented with TensorFlow.js. After the LLM supplies labels, the tiny model is fine‑tuned on the resulting dataset using standard cross‑entropy loss and early‑stopping based on a held‑out slice. The training loop runs on a single consumer‑grade GPU (e.g., RTX 3060) and completes in about 60 seconds, consuming less than a cent of cloud compute.
The authors benchmarked the approach on ten classification tasks ranging from sentiment analysis to topic detection. In a majority of those tasks the tiny classifier achieved higher accuracy than the OpenAI classification API, despite using a fraction of the latency and cost. The reported speedup is roughly 10× faster inference compared to the remote API, and the total training cost is orders of magnitude lower.
The method hinges on the quality of the LLM‑generated labels; noisy pseudo‑labels can cap performance, especially on nuanced domains. Because the pipeline lives in JavaScript, inference speed on CPU is slower than native PyTorch, and the model size, while small, still exceeds what fits comfortably on low‑end mobile browsers. Nonetheless, the trade‑off of ultra‑quick, cheap model iteration makes it attractive for rapid prototyping and edge deployment.
TakeawayA 17 M‑parameter classifier trained from LLM‑generated pseudo‑labels in ~60 s can surpass a commercial API on most of the ten benchmark tasks.