The integration of distributed computing, machine learning, and control theory creates new opportunities for developing scalable and data-driven cyber-physical systems. However, processing high-volume sensor data and supporting reliable model evaluation require architectures capable of combining distributed data ingestion, parallel computation, and machine learning workflows.
This presentation explores an architectural approach based on Apache Kafka and Apache Spark for building scalable data pipelines that support data-driven modeling and machine learning in complex application environments. The proposed approach considers streaming data acquisition, distributed feature processing, model evaluation, and the potential integration of predictive models with control and optimization components.
Environmental monitoring and smart-building applications provide a motivating context for investigating how distributed infrastructure can support real-time analytics, system modeling, and future data-driven control strategies. Particular attention is given to scalability, data quality, processing latency, fault tolerance, and the reproducibility of computational experiments.
The presentation aims to identify research opportunities at the intersection of distributed systems, machine learning, and control theory, including benchmark dataset generation, scalable model evaluation, and the integration of learning-based methods into complex cyber-physical systems. It also outlines potential directions for interdisciplinary collaboration within InterCoML.
Prof. Cezary Orłowski is a Professor of Computer Science and researcher specializing in software engineering, information systems, artificial intelligence, knowledge management, and distributed data processing. His research focuses on intelligent software development environments, multi-agent systems, enterprise architecture, requirements engineering, data modeling, and advanced knowledge representation techniques, including Retrieval-Augmented Generation (RAG). He is actively involved in projects related to digital transformation, IoT-based environmental monitoring systems, governance of AI-assisted software development processes, and the application of generative AI in Software Development Life Cycle (SDLC) environments. His current work explores the design of agent-driven software engineering platforms that integrate governance, traceability, competency models, and autonomous AI agents into a coherent development ecosystem.