Python
PyTorchAn end-to-end multi-label emotion classification system that predicts five emotions (Anger, Fear, Joy, Sadness, Surprise) from short English text using ensemble transformer models with knowledge distillation for efficient deployment. Built as an IIT Madras course project with a production-ready model deployed to Hugging Face Hub.
Unique Features
10-model transformer ensemble (2 architectures × 5 folds) with weighted logits averaging
Knowledge distillation pipeline compressing large ensemble into deployable DeBERTa-v3-base student model
Hybrid CNN + BiGRU + Self-Attention architecture combining local n-gram patterns with long-range sequence context
Full experiment tracking with Weights & Biases integration
Production-ready model deployed to Hugging Face Hub
Progression from rule-based baseline → classical ML → neural networks → transformers → knowledge distillation
Tech Stack
LanguagePython
ML/DLPyTorch, Hugging Face Transformers, scikit-learn
ModelsDeBERTa-v3-large, RoBERTa-large, CNN + BiGRU + Self-Attention
ToolsWeights & Biases, Hugging Face Hub, GPU with mixed precision (AMP)