Leveraging Transfer Learning for NLP in Extremely Low-Resource Language Settings

Abstract

Globally, individuals are increasingly gaining access to current technologies. They facilitate unprecedented access to knowledge, justice, and information. To ensure accommodation and universal accessibility, a globally spoken language must be considered, and it is essential to facilitate computers' comprehension of human language. The computational methodologies that exist have advanced significantly; yet these improvements can often require substantial resources, including data generation and processing. Such a reliance on resources hinders the ef icient advancement of cross-lingual transfer techniques, especially for taskslike discourse analysis that require rigorous training and evaluation across more spoken languages and domains. The existing systems consume N resources, and the best language processing methods haven't been able to cover many languages and domains. Transfer learning seeks to solve this problem by leveraging pre-trained modelstrained on large datasets to work in resource-constrained environments. These strategies have gained popularity for their ef ectiveness in managing limited resources across diverse tasks, areas, and languages. This research focuses on the application of transfer learning approachesto Indian languages, notably Hindi and its code-mixed English-Hindi variety, which is commonly found on social networking sites. We examine cross-task and cross-lingual transferstrategies acrossseveral downstream tasks, demonstrating their effectiveness despite limited training data and computational resources. We also provide a syntactic-semantic curriculum-based learning architecture for English-Hindi code-mixed sentiment analysis, resulting in significant performance improvements.