Smart-Transfer: A Cost-Minimized Inter-Service Data Storage and Transfer Scheme

Abstract

Cloud storage services are a widely used tool in both industry and research. However, the wide variety of services offered by cloud providers raises the question of which storage services can most cheaply meet the needs of an application, particularly in cases where the performance required by that application varies over time. To address this problem, we propose Smart-Transfer, a unique scheme to reduce the long-term cost of storing large data sets using cloud storage services. In contrast to most existing works, we study the storage cost-minimization problem by leveraging various available storage services that can provide different levels of performance at different pricing cost, under the constraint of data-access performance requirement. The key idea of Smart-Transfer is to continually transfer large data sets between different cloud storage services so as to always make the data ready within a specified period of time in at least one service that can satisfy the performance requirement while avoiding overpaying for unnecessarily high performance guarantees. While the selected storage service to satisfy a particular data-access request may not be the cheapest, the accumulative data-transfer and data-storage cost over a long period of time to satisfy a sequence of data-access requests can be minimized for the system. Simulation results show that Smart-Transfer performs well under various system parameters and request patterns, and can significantly reduce the cost compared to other schemes.