Trustworthy Self-Composable Big-Data-as-a-Service: An LLM-Orchestrated Multi-Agent Framework for Automated Data Engineering, AutoML, MLOps Deployment, and Drift-Aware Lifecycle Optimization
arXiv:2606.17915v1 Announce Type: cross Abstract: Big-Data-as-a-Service (BDaaS) platforms require re liable automation across data ingestion, cleaning, feature engi neering, model development, deployment, and post-deployment monitoring. However, existing LLM-based data science agents and AutoML systems mainly focus on isolated workflow stages, leaving limited support for lifecycle-level orchestration, artifact governance, human oversight, and drift-aware adaptation. This paper proposes a trustworthy self-composable BDaaS frame work based on LLM-orchestrated multi-agent collaboration. The propo
The increasing complexity of big data pipelines and the rapid advancement of large language models are converging to necessitate more autonomous and trustworthy data engineering solutions.
This development indicates a significant leap in data platform automation, promising to collapse traditional data engineering, MLOps, and even some AutoML workflows into highly agentic systems.
Current fragmented data science workflows will be integrated and automated by LLM-orchestrated multi-agent systems, transforming how data-driven products are built and maintained.
- · Enterprises with complex data needs
- · Cloud service providers offering BDaaS
- · AI/ML solution developers
- · Data scientists focused on higher-level problem-solving
- · Manual data engineering roles
- · Fragmented point-solution data tools
- · Companies with legacy data infrastructure
- · Consultancies reliant on manual MLOps
BDaaS platforms will become significantly more autonomous and capable, reducing the human effort in data-to-model pipelines.
This automation will accelerate the development and deployment of AI applications across various industries, creating new market opportunities and competitive pressures.
The enhanced trust and drift-aware optimization features could lead to more robust and reliable AI systems, potentially addressing ethical and operational concerns around AI governance.
This signal links to a primary source. Continuum Brief monitors and indexes it as part of the live intelligence stream — we do not republish source content.
Read at arXiv cs.AI