International Research Journal of Education and Technology

Now advanced in leveraging language models to produce cross-modal audio-text representations have beat the restrictions of established teaching methods that rely on prearranged labels. These have enabled the society to build development on issues such as zero-shot classification, which would otherwise be impossible. However, understanding such representations necessitates a huge number of manually annotated audio-text pairs. In this paper, we investigate unsupervised techniques to improving the learning framework for such representations using unpaired text and audio. We investigate domain-unspecific and domain-specific curtain strategies to generate audio-text pairs, which we employ to improve the model. We further demonstrate that when domain-specific curtain is combined with a soft-labeled contrastive loss, we can achieve considerable improvements in zero-shot classification performance on downstream sound event or acoustic scene classification tasks. The suggested model, which can translate text, images, and voice, has been tested on huge datasets in multiple Indian languages and employs cutting-edge techniques such as machine learning, computer vision, and speech recognition to accurately transcribe and translate the input data. The experiment results show that the model is effective at accurately converting text, images, and audio to text, and the potential applications of our proposed model range from language learning to accessibility for nonverbal or non-hearing individuals to cross-language communication. The proposed concept aims to bridge the language gap and improve communication between persons from various linguistic backgrounds.

Keywords— Audio-Text, Zero-shot, Clustering, Unsupervised Technique.