A few questions I have enjoyed exploring:
Systems ML for systems How can deployed applications tune themselves as workloads and infrastructure change?
OPPerTune appeared at NSDI 2024.
Applications in production often need to retune configuration parameters as workloads and infrastructure change. OPPerTune lets an application declare which settings it can change, try suggested values, and report how well they worked. From this feedback, it learns better settings without needing access to the application's internals.
I led the design and development of OPPerTune. I later led its open-source release, production deployments, and integration with Microsoft's prediction-serving platform.
ML for systems LLMs Can operators turn requirements into useful reward functions?
Reward Copilot appeared at the NeurIPS 2024 Workshop on Machine Learning for Systems.
A reward function combines goals such as latency, throughput, and resource use into the feedback from which a reinforcement-learning algorithm learns. Reward Copilot asks an LLM to propose reward functions from operator requirements, tests the resulting policies, and feeds their performance back to the LLM for refinement.
I started Reward Copilot to make reward-function design easier, motivated by challenges I encountered while building OPPerTune.
Systems ML for systems Can an ensemble of smaller models make inference cheaper on one GPU?
Exploratory work at Microsoft Research.
Model ensembles can improve prediction quality by combining several models, but running them independently can multiply inference costs. CEISER explores replacing one model with an ensemble of smaller models that shares a single GPU while preserving similar accuracy.
I explored efficient single-GPU execution by merging the models' computation graphs and fusing GPU kernels, allowing the ensemble to share work instead of running each model independently.