Infrastructure Optimizer spawns containers on discounted virtual machine instances (Amazon Spot Instances, Azure Spot Virtual Machines, and Google Preemptible VM Instances) and dynamically relocates containers between such instances, based on availability and price. Because the reliability of Spot Instances is not guaranteed (they can be reclaimed by the cloud provider any time with very short notification), cloud users can only use them for workloads that can tolerate a sudden failure. Infrastructure Optimizer dramatically improves the usability of Spot Instances for critical applications, with cost reduction of up to 90% compared to regular on-demand instances. With Infrastructure Optimizer, we can dynamically relocate containers between Spot instances and on-demand instances, dramatically improving the reliability of the workload running on Spot instances. This makes Spot instances much easier to use and more suitable for mission-critical applications.
Infrastructure Optimizer uses Cornell patented user-level live migration technology. User VMs are run as nested or second layer VMs (sVMs), while the Spot Instances form the first layer VMs (fVMs). Nested virtualization allows sVMs to be migrated according to a scheduling policy. Users set a high maximum bid, which can be as high as the On-Demand prices. The Infrastructure Optimizer runtime monitors the spot price and live migrates sVMs to other, possibly cheaper fVMs – either just another Spot Instance type, or even Spot Instances in another availability zone, another region, or another cloud. In the worst case, the sVMs can be live migrated to regular On-Demand Instances that are always available, so the user only needs to pay as much as the On-Demand price. When the price approaches the maximum bid in the middle of an instance hour, the Infrastructure Optimizer runtime migrates the sVMs to avoid being terminated. The Infrastructure Optimizer Instance Scheduler optimizes placement of groups of sVM on different types of fVMs, either Spot Instances or regular instances.
