论文标题
变压器ASR的适应性多域语言模型
Adaptable Multi-Domain Language Model for Transformer ASR
论文作者
论文摘要
我们为变压器ASR提出了基于适配器的多域变压器语言模型（LM）。该模型由大尺寸的常见LM和小型适配器组成。该模型只能使用小尺寸适配器及其相关层执行多域适应。所提出的模型可以重用完整的微调LM，该LM使用原始模型的所有层进行微调。通过在第二个域之后，可以将约2％的参数和13％的参数扩展到新域。提出的模型也有效地降低了模型维护成本，因为可以省略昂贵且耗时的常见LM前LM预训练过程。使用提出的基于适配器的方法，我们观察到，具有适配器的一般LM可以在单词错误率（WER）方面胜过专用的音乐域LM。
We propose an adapter based multi-domain Transformer based language model (LM) for Transformer ASR. The model consists of a big size common LM and small size adapters. The model can perform multi-domain adaptation with only the small size adapters and its related layers. The proposed model can reuse the full fine-tuned LM which is fine-tuned using all layers of an original model. The proposed LM can be expanded to new domains by adding about 2% of parameters for a first domain and 13% parameters for after second domain. The proposed model is also effective in reducing the model maintenance cost because it is possible to omit the costly and time-consuming common LM pre-training process. Using proposed adapter based approach, we observed that a general LM with adapter can outperform a dedicated music domain LM in terms of word error rate (WER).
