Perceptive bidding strategy for Amazon EC2 spot instance market

Article type: Research Article

Authors: Khandelwal, Veena^{a; *} | Gupta, Chandra Prakash^a | Chaturvedi, Anand Kishore^b

Affiliations: [a] Department of Computer Science and Engineering, Rajasthan Technical University, Kota, India | [b] MLV Textile and Engineering College, Bhilwara Affiliated to Rajasthan Technical University, Kota, India

Correspondence: [*] Corresponding author: Veena Khandelwal, Department of Computer Science and Engineering at Rajasthan Technical University, Kota, India. E-mail: [email protected].

Abstract: Spot pricing is an auction based market mechanism introduced by Amazon EC2 to sell available cloud resources at a low cost without any availability guarantee. Bid prices submitted by users directly affect reliability of the instance acquired. In this context, we propose a Perceptive Bidding Strategy (PBS) for spot instances which presents different bid prices for different time during a day across all Amazon EC2 regions to assist users optimize their bids for obtaining cheaper spot instances with greater reliability. The strategy is firmly grounded on one-day-ahead spot price predictions using Random Forests. The bid price is not static but varies with job size to minimize cost and volatility of spot instances. We also develop a spot billing simulator to validate the bidding strategy against historical data. The simulator utilizes bid prices generated by PBS and Amazon’s real spot price traces of the bidding period for empirical investigations. We evaluate PBS for varying job sizes for a period of two weeks on different compute instance types and demonstrate that PBS greatly reduces job execution costs up to 80% while maintaining high reliability up to 80 to 90%. We compare PBS with several static bidding strategies and show that PBS provides spot instances with reliability for varying job sizes and greatly reduces risk of high execution costs.

Keywords: Amazon EC2, perceptive bidding strategy, spot instances, compute intensive instances

DOI: 10.3233/MGS-180282

Journal: Multiagent and Grid Systems, vol. 14, no. 1, pp. 83-102, 2018