Model Distillation for Deep Audio Representations
Bachelor thesis investigating how large audio representation models such as BEATs can be distilled into lightweight student networks for deployment on hearable devices.
Audio Signal ProcessingDeep LearningKnowledge DistillationPyTorchRepresentation LearningModel Compression