Mustafa Abbas, Al-Khafaji and Huda Karim, Al-Saedi (2022) Generative AI in Enterprise Data Engineering: Integrating Copilot for ETL Automation. International Journal of Trend in Scientific Research and Development, 6 (4). pp. 2390-2395. ISSN 2456-6470

Text ijtsrd50069.pdf Download (779kB)

Abstract

As enterprises grapple with growing volumes and complexity of data, traditional extract transform load (ETL) processes are increasingly strained by scalability demands, evolving business requirements, and the need for rapid delivery of analytics-ready datasets. Conventional automation approaches address some inefficiencies but often fall short in adaptability and context-awareness. This paper explores the integration of generative AI specifically Copilot-style assistants into enterprise data engineering workflows to accelerate and enhance ETL automation. Generative AI introduces a paradigm shift by enabling natural language–driven pipeline generation, automated schema mapping, intelligent error handling, and adaptive optimization, thereby reducing manual intervention and development bottlenecks. Beyond productivity gains, AI-powered ETL fosters greater collaboration between technical engineers and business stakeholders, bridging skill gaps and democratizing data transformation tasks. Key considerations such as governance, data quality, security, and regulatory compliance are examined to ensure responsible deployment at scale. The proposed framework positions generative AI not merely as a coding assistant, but as a strategic enabler for modern data platforms, empowering enterprises to build more resilient, agile, and intelligent data engineering ecosystems.

Item Type:	Article
Subjects:	Q Science > QA Mathematics > QA75 Electronic computers. Computer science
Divisions:	Postgraduate > Master's of Islamic Education
Depositing User:	Journal Editor
Date Deposited:	10 Sep 2025 07:37
Last Modified:	10 Sep 2025 07:37
URI:	http://eprints.umsida.ac.id/id/eprint/16335

Actions (login required)

View Item