Can AI Agents Make GPUs More Energy Efficient?

Most GPU clusters are provisioned for peak performance, not energy efficiency. The result is expensive infrastructure sitting on significant unused headroom.

What if the serving stack could continuously optimize itself?

In this session, we’ll explore an architecture built around self-improving agents that continuously tune GPU inference. The agents experiment with batching, tensor parallelism, prefix caching, expert routing, and serving configurations, then use real workload and energy measurements to decide what to change next.

The goal is ambitious: get 8 GPUs to deliver workloads you might otherwise provision 16 GPUs for, while using dramatically less energy.

At Traversaal, we’ve been building this approach in the open at energy.traversaal.ai. Instead of charging per token, the inference API is priced around the energy actually consumed. That changes the optimization problem: every watt saved can translate directly into lower inference cost.

What we'll cover

• Measuring energy consumption at the request and workload level

• Building agents that experiment with serving configurations

• Batching, caching, parallelism, and routing as optimization levers

• Creating feedback loops so the system improves across runs

• Where unused capacity exists in today's GPU deployments

• What energy-based inference pricing could look like

We'll also walk through what we've learned from building the system, where the approach works, where it doesn't, and the challenges of letting agents autonomously optimize infrastructure.

Who should come: ML and infrastructure engineers, agent builders, researchers, and anyone spending real money running GPUs.

Schedule

4:00 PM · Doors open, food & drinks
4:30 PM · Talk: Energy-priced inference and the self-improving serving stack
5:15 PM · Open discussion & Q&A
6:00 PM · Wrap

Hosted by Hamza Farooq (Traversaal) and Julius Ritter at AGI House SF.