previous day
all days

View: session overviewtalk overview

09:00-10:00 Session 9
Big Data, Big Systems, Big Challenges (A Personal Experience)
SPEAKER: Vadim Kotov

ABSTRACT. I have been fortunate to work in two remarkable research communities, Akademgorodok in Siberia and Silicon Valley in California, and that my professional career stretched over two very different, yet both exciting epochs of the computer systems and science evolution: steady accumulation of knowledge and technology in 1960s-1980s and, then, “Internet Big Bang” and Information Revolution in 1990s-2010s. Since arriving to Silicon Valley in 1991, I have been working on concurrent and distributed systems, expanding my research done in Akademgorodok. The systems I worked on were becoming more and more large and sophisticated (clusters, data centers, global enterprise IT infrastructures, planetary scale computing,…). They were generating more and more data that, in turn, required even larger and complex system to process them. The term “Big Data” is now used to describe collections of data so huge that it becomes difficult to process them using traditional computer systems and applications. In this talk, I track the trends in the development of large computer systems which I witnessed working first at Hewlett-Packard Laboratories and then at Carnegie-Mellon University, Silicon Valley Campus. This is not a general survey, I exemplify those trends by the systems in the analysis or/and design of which I or my colleagues participated. The driving force behind the computer system progress is unprecedented accumulation of complex data and huge global data traffic. Big Data is a popular metaphor labeling the situation. Big Data requires Big Systems that become the main trend in the current system architecture. They include: powerful data centers using of tens of thousands servers; enterprise IT “systems of systems” consisting of globally distribute data centers; Grid and Utility Computing; Cloud Computing; smart embedded systems. Big Data distributed among storages in a data center or in multiple data centers requires new type of database software organization and parallel frameworks for running applications. SAP HANA and Apache Hadoop are intended to help. New software paradigms include service-oriented architecture, autonomous, and multi-agent computing. In service-centric software organization, the basic programming entity is service, a well-defined procedure that does not depend on the state of other services. The services interact with each other by exchanging service requests and service results. The version proposed at HP Labs is discussed. To overcome growing complexity of management for large-scale distributed systems, autonomous and multi-agent architecture replaces the centralized management by distributed self-management provided by autonomic agents that interact to manage collectively. Four distributed management algorithms, providing increased servers' utilization and reduced the overall network traffic, has been studied. Embedded computers are becoming a key component of all kinds of complex cyber-physical systems. The given examples are Smart Homes, Smart (electrical) Grid, and driverless cars. As computer systems play more and more important role in our life, can we trust them? System dependability should guarantee their availability (readiness for service), reliability (continuity of uninterrupted service), safety (no catastrophic consequences for users), integrity (no unforeseen changes), and maintainability (ability to quickly repair and modify). NASA-sponsored the High-Dependability Computing Program is an example of efforts to improve system dependability. In conclusion, few remarks about what is ahead.

10:30-23:00Excursion + Conference Dinner
Disclaimer | Powered by EasyChair Smart Program