Fix keyword spotting. (#1689) #1556

Job	Run time
ubuntu-20.04 3.10 transducer	7m 30s
ubuntu-20.04 3.10 paraformer	12m 13s
ubuntu-20.04 3.10 zipformer2-ctc	7m 55s
ubuntu-22.04 3.10 transducer	4m 54s
ubuntu-22.04 3.10 paraformer	7m 15s
ubuntu-22.04 3.10 zipformer2-ctc	4m 38s
windows-latest 3.10 transducer	18m 53s
windows-latest 3.10 paraformer	16m 29s
windows-latest 3.10 zipformer2-ctc	16m 51s
macos-latest 3.10 transducer	3m 50s
macos-latest 3.10 paraformer	5m 2s
macos-latest 3.10 zipformer2-ctc	4m 28s
macos-14 3.10 transducer	3m 45s
macos-14 3.10 paraformer	5m 17s
macos-14 3.10 zipformer2-ctc	4m 15s
	2h 3m 15s

Provide feedback