Publications

Kubernetes has become the dominant platform for orchestrating containerized applications in modern cloud environments. However, its default scheduling mechanisms primarily focus on CPU and memory resources, overlooking network performance–a critical factor for latency-sensitive applications in domains like 5G and edge computing. This paper introduces NAS, a novel network-aware scheduling framework that integrates real-time network metrics such as latency and bandwidth using an eBPF-based Cilium service mesh. By incorporating these metrics into the scheduling process, NAS optimizes pod placement for performance-sensitive applications. Experimental results show that NAS reduces average latency by 52.66% compared to the default Kubernetes scheduler and 2.68% compared to Diktyo while minimizing maximum latency spikes by 85.61% and 7.23%, respectively. Further, NAS effectively distributes workloads and provides co-location benefits by considering microservices dependencies and network costs during pod placement.

Campus high-performance computing (HPC) centers must continuously look for ways to provide reliable, secure, and cost-effective computing services to meet the scientific computing needs of researchers and students. With the increasing maturity of Generative AI (GenAI), campus researchers, students, and staff increasingly seek to incorporate AI tools into their scientific workflows. The Rosen Center for Advanced Computing at Purdue University has recently released two Generative AI inference tools, AnvilGPT and Purdue GenAI Studio, to the research and campus communities. These services support over 1000 users who use 10+ open-source GenAI models to aid their work. Building on HPC's long history of using open-source tools, these services are based on customized open-source frameworks and hosted entirely on-prem. This paper argues that building custom GenAI services from open-source frameworks is a scalable and cost-effective solution for providing access to Generative AI models. This paper shares the methodology and resources required to develop and host these services and seeks to be a resource for other research computing centers that wish to leverage their HPC investment to create similar services.

Optimizing resource utilization and performance in cloud-native environments has become increasingly critical as applications grow in complexity and demand specialized resources such as graphics processing units (GPUs). This study explores how enhancements to Kubernetes control plane, specifically through advanced schedulers and autoscalers, can improve resource efficiency and application performance. Focusing on environments sensitive to network conditions and requiring GPU resources, the research investigates how these control plane components have been optimized to meet the growing demands of modern applications. The research focuses on two key areas: implementing network-aware scheduling and developing GPU autoscaling mechanisms based on real-time demand. Network-aware scheduling strategies have been proposed and tested to reduce communication overhead between microservices. In addition, autoscaling mechanisms have been developed to dynamically manage GPU resources, ensuring that workloads are handled with optimal performance under varying conditions. The results demonstrate that extending the Kubernetes control plane with custom scheduling and autoscaling techniques can significantly enhance resource utilization and application performance, reducing latency, boosting workload performance, and enabling better resource allocation in cloud-native environments.