Scalable data management in distributed information systems

[EN] In the era of cloud computing and huge information systems, distributed applications should manage dynamic workloads; i.e., the amount of client requests per time unit may vary frequently and servers should rapidly adapt their computing efforts to those workloads. Cloud systems provide a solid basis for this kind of applications but most of the traditional relational database systems are unprepared to scale up with this kind of distributed systems. This paper surveys different techniques being used in modern SQL, NoSQL and NewSQL systems in order to increase the scalability and adaptability in the management of persistent data. © 2011 Springer-Verlag.This work has been supported by EU FEDER and Spanish MICINN under research grants TIN2009-14460-C03-01 and TIN2010-17193Pallardó Lozoya, MR.; Esparza Peidro, J.; García Escriva, JR.; Decker, H.; Muñoz Escoí, FD. (2011). Scalable data management in distributed information systems. Lecture Notes in Computer Science. 7046:208-217. https://doi.org/10.1007/978-3-642-25126-9_31S2082177046Helland, P.: Life beyond distributed transactions: an apostate’s opinion. In: 3rd Biennial Conf. on Innov. Data Syst. Research (CIDR), Asilomar, CA, USA, pp. 132–141 (2007)Finkelstein, S., Jacobs, D., Brendle, R.: Principles for inconsistency. In: 4th Biennial Conf. on Innov. Data Syst. Research (CIDR), Asilomar, CA, USA (2009)Chang, F., Dean, J., Ghemawat, S., Hsieh, W.C., Wallach, D.A., Burrows, M., Chandra, T., Fikes, A., Gruber, R.: Bigtable: A distributed storage system for structured data. In: 7th Symp. on Operat. Syst. Design and Implem. (OSDI), pp. 205–218. USENIX Assoc., Seattle (2006)Cooper, B.F., Baldeschwieler, E., Fonseca, R., Kistler, J.J., Narayan, P.P.S., Neerdaels, C., Negrin, T., Ramakrishnan, R., Silberstein, A., Srivastava, U., Stata, R.: Building a cloud for Yahoo! IEEE Data Eng. Bull. 32, 36–43 (2009)DeCandia, G., Hastorun, D., Jampani, M., Kakulapati, G., Lakshman, A., Pilchin, A., Sivasubramanian, S., Vosshall, P., Vogels, W.: Dynamo: Amazon’s highly available key-value store. In: 21st ACM Symp. on Operat. Syst. Princ. (SOSP), Stevenson, Washington, USA, pp. 205–220 (2007)Stonebraker, M., Madden, S., Abadi, D.J., Harizopoulos, S., Hachem, N., Helland, P.: The end of an architectural era (it’s time for a complete rewrite). In: 33rd Intnl. Conf. on Very Large Data Bases (VLDB), pp. 1150–1160. ACM Press, Vienna (2007)Lomet, D.B., Fekete, A., Weikum, G., Zwilling, M.J.: Unbundling transaction services in the cloud. In: 4th Biennial Conf. on Innov. Data Syst. Research (CIDR), Asilomar, CA, USA (2009)Campbell, D.G., Kakivaya, G., Ellis, N.: Extreme scale with full SQL language support in Microsoft SQL Azure. In: Intnl. Conf. on Mngmnt. of Data (SIGMOD), pp. 1021–1024. ACM, New York (2010)Levandoski, J.J., Lomet, D., Mokbel, M.F., Zhao, K.K.: Deuteronomy: Transaction support for cloud data. In: 5th Biennial Conf. on Innov. Data Syst. Research (CIDR), Asilomar, CA, USA, pp. 123–133 (2011)Helland, P., Campbell, D.: Building on quicksand. In: 4th Biennial Conf. on Innov. Data Syst. Research (CIDR), Asilomar, CA, USA (2009)Muñoz-Escoí, F.D., García-Escrivá, J.R., Pallardó-Lozoya, M.R., Esparza-Peidro, J.: Managing scalable persistent data. Technical Report ITI-SIDI-2011/003, Instituto Tecnológico de Informática, Universitat Politècnica de València, Spain (2011)Agrawal, D., El Abbadi, A., Antony, S., Das, S.: Data management challenges in cloud computing infrastructures. In: 6th Intnl. Wshop. on Databases in Networked Information Systems (DNIS), Aizu-Wakamatsu, Japan, pp. 1–10 (2010)Stonebraker, M.: The case for shared nothing. IEEE Database Eng. Bull. 9, 4–9 (1986)Alonso, G., Kossmann, D., Roscoe, T.: SwissBox: An architecture for data processing appliances. In: 5th Biennial Conf. on Innov. Data Syst. Research (CIDR), Asilomar, CA, USA, pp. 32–37 (2011)Baker, J., Bond, C., Corbett, J.C., Furman, J.J., Khorlin, A., Larson, J., Léon, J.M., Li, Y., Lloyd, A., Yushprakh, V.: Megastore: Providing scalable, highly available storage for interactive services. In: 5th Biennial Conf. on Innov. Data Syst. Research (CIDR), Asilomar, CA, USA, pp. 223–234 (2011)Curino, C., Jones, E.P.C., Popa, R.A., Malviya, N., Wu, E., Madden, S., Balakrishnan, H., Zeldovich, N.: Relational cloud: A database-as-a-service for the cloud. In: 5th Biennial Conf. on Innov. Data Syst. Research (CIDR), Asilomar, CA, USA, pp. 235–240 (2011)Das, S., Agrawal, D., El Abbadi, A.: ElasTraS: An elastic transactional data store in the cloud. CoRR abs/1008.3751 (2010)Vogels, W.: Eventually consistent. Commun. ACM 52, 40–44 (2009)Breitbart, Y., Korth, H.F.: Replication and consistency: being lazy helps sometimes. In: 16th ACM Symp. on Princ. of Database Syst., PODS 1997, pp. 173–184. ACM, New York (1997)Brantner, M., Florescu, D., Graf, D.A., Kossmann, D., Kraska, T.: Building a database on S3. In: Intnl. Conf. on Mngmnt. of Data (SIGMOD), pp. 251–264. ACM Press, Vancouver (2008)Lakshman, A., Malik, P.: Cassandra: a decentralized structured storage system. Operating Systems Review 44, 35–40 (2010)Burrows, M.: The Chubby lock service for loosely-coupled distributed systems. In: 7th Symp. on Operat. Syst. Design and Implem. (OSDI), pp. 335–350. USENIX Assoc., Seattle (2006)Junqueira, F.P., Reed, B.: The life and times of a ZooKeeper. In: 28th Annual ACM Symp. on Princ. of Distrib. Comp. (PODC), p. 4. ACM Press, Calgary (2009)MacCormick, J., Murphy, N., Najork, M., Thekkath, C.A., Zhou, L.: Boxwood: Abstractions as the foundation for storage infrastructure. In: 6th Simp. on Operat. Syst. Design and Impl. (OSDI), pp. 105–120. USENIX Assoc., San Francisco (2004)Stonebraker, M., Cattell, R.: Ten rules for scalable performance in ”simple operation” datastores. Commun. ACM 54, 72–80 (2011)Amazon Web Services LLC: Amazon SimpleDB (2011), http://aws.amazon.com/simpledb/Lamport, L.: The part-time parliament. ACM Trans. Comput. Syst. 16, 133–169 (1998)Bernstein, P.A., Reid, C.W., Das, S.: Hyder - a transactional record manager for shared flash. In: 5th Biennial Conf. on Innov. Data Syst. Research (CIDR), Asilomar, CA, USA, pp. 9–20 (2011)Bonnet, P., Bouganim, L.: Flash device support for database management. In: 5th Biennial Conf. on Innov. Data Syst. Research (CIDR), Asilomar, CA, USA, pp. 1–8 (2011)Microsoft Corp.: Windows Azure: Microsoft’s cloud services platform (2011), http://www.microsoft.com/windowsazure/VoltDB, Inc.: VoltDB technical overview: Next generation open-source SQL database with ACID for fast-scaling OLTP applications (2010), Downloadable from: http://voltdb.com/_pdf/VoltDBTechnicalOverviewWhitePaper.pd

Estimation de performances et de consommation énergétique de systèmes de stockage à base de mémoire flash dans les systèmes embarqués

Controlling and optimizing embedded system performance and power consumption is critical. In this context, estimation techniques are used when performing measurement campaigns is difficult due to time or financial constraints. This work targets the performance and power consumption evaluation of the secondary storage service in an embedded operating system using NAND flash memory. One way to manage flash memory is to used dedicated Flash File Systems (FFS). One can observe a lack of work in the literature concerning FFS performance and power consumption estimation techniques.The contributions presented in this thesis rely on a three steps performance and power consumption modeling methodology. During the exploration phase, we identify through micro-benchmarking the main elements of a FFS based system impacting performance and power consumption of the embedded system. In the modeling phase, this impact is represented by building models of various types. The main models types are the functional, performance and power consumption models. Models parameters are extracted through measurements on a real platform. During the simulation phase the models are implemented in a simulator. This tool allows obtaining performance and power consumption estimations concerning a flash-based storage system processing a given I/O workload.Maitriser et optimiser les performances et la consommation énergétique dans les systèmes embarqués est aujourd'hui crucial. Pour ce faire, des techniques d'estimation de ces métriques sont utilisées dans des environnements où la réalisation de mesures est difficile. Ce travail cible l'évaluation des performances et de la consommation énergétique du service du stockage secondaire dans un système d'exploitation embarqué utilisant une mémoire flash NAND. L'un des moyens de gérer ce type de média est l'utilisation de systèmes de fichiers dédiés (Flash File Systems, FFS), pour lequel on peut constater un manque de travaux dans la littérature concernant les techniques d'estimation des performances et de la consommation. Les contributions apportées dans cette thèse s'articulent autour d'une méthodologie de modélisation pour l'estimation des performances et de la consommation des systèmes de stockage embarqués de type FFS. Cette méthodologie est divisée en trois phases. En phase d'exploration on identifie, via des micro-benchmarks, les éléments du système de stockage impactant les performances et la consommation du système embarqué. En phase de modélisation, cet impact est représenté sous la forme de modèles de différents types, dont les principaux sont les modèles fonctionnels, de performances et de consommation. Les paramètres de ces modèles sont extraits via des mesures. En phase de simulation, les modèles sont implémenté dans un simulateur, développé dans le cadre de cette thèse, permettant d'obtenir des estimations concernant les performances et la consommation d'un système de stockage à base de mémoire flash soumis à une charge d'entrées / sorties donnée

Flash Device Support for Database Management

International audienceWhile disks have o ered a stable behavior for decades -thus guaranteeing the timelessness of many database design deci- sions, ash devices keep on mutating. Their behavior varies across models, across rmware updates and possibly in time for the same model. Many researchers have proposed to adapt database algorithms for existing ash devices; others have tried to capture the performance characteristics of ash devices. However, today, we neither have a reference DBMS design nor a performance model for ash devices: database researchers are running after ash memory technology. In this paper, we take the reverse approach and we de ne how ash devices should support database management. We ad- vocate that ash devices should provide DBMS with more control over IO behavior without sacri cing correctness or robustness, exposing the full potential of the underlying ash chips in terms of performance. We suggest two approaches: (a) keep the narrow block device interface, or (b) provide a rich interface that allows a DBMS to explicitly control IO behavior. We believe that these approaches are natural evolutions of the current generation of ash devices, whose complexity and opacity is ill-suited for database manage- ment. We describe the design space for the two proposed approaches, discuss how they would bene t many existing techniques proposed by the database research community, and identify a set of new research issues. 1