Mind the Memory Gap: Unveiling GPU Bottlenecks in Large-Batch LLM Inference
- Pol G. Recasens
- Ferran Agullo
- et al.
- 2025
- CLOUD 2025
Publications
7 results for Yue Zhu
Towards Efficient Key-Value Cache Management for Prefix Prefilling in LLM Inference
- Yue Zhu
- Hao Yu
- et al.
- 2025
- CLOUD 2025
Scalable and Efficient LLM Serving with the vLLM Production Stack
- Junchen Jiang
- Yue Zhu
- 2025
- OSSNA 2025
How Low Can LoRA Go: System-Level Throughput, Energy, and Model Quality Tradeoffs when Fine-Tuning Adapters
- Connor Espenshade
- Umesh Deshpande
- et al.
- 2025
- ISCA 2025
Optimizing GPU Multiplexing for Efficient and Cost-Effective Access to Diverse Large Language Models in GPU Clusters
- Yue Zhu
- Chen Wang
- et al.
- 2024
- MASCOTS 2024
GPU OPTIMIZATIONS FOR EFFICIENT AND COST-EFFECTIVE ACCESS TO DIVERSE LARGE LANGUAGE MODELS IN RESEARCH CLUSTER
- Chen Wang
- Yue Zhu
- et al.
- 2024
- MLSys 2024
Towards Pareto Optimal Throughput in Small Language Model Serving
- Pol G. Recasens
- Yue Zhu
- et al.
- 2024
- EuroSys 2024