Build Stuff'14

Google and eBay operate some of the largest Internet sites on the planet, and each maintains its leadership through continuous innovation in infrastructure and products. While substantially different in their detailed approaches, both organizations have lessons to teach about building and maintaining large-scale systems. This session tells several war stories from Google and eBay focusing on how to scale code, infrastructure, performance, and operations. It details hard-won lessons learned in scaling those companies' computing systems, organizations, and technology stacks. It finally describes how we put some of those lessons into practice in the context of real-time games at KIXEYE, and offers concrete suggestions you can apply to your own organization.

Come watch the unveiling of the magic behind developing drugs and making music in a way simple enough to inspire even a 5 year old. We will explore how this provides us with lessons in writing scalable, maintainable code, and how it allows us to tackle problems in new ways. Yes, there will be experiments. Yes, there will be live-coded music. And yes, you really can make cleaner and more scalable code when you think about solutions through the eyes of a child. 

In this talk we present Scalable Distributed (SD) Erlang - an extension of distributed Erlang functional programming language for reliable scalability. The work is a part of the RELEASE project that aims to improve the scalability of Erlang programming language. We start by providing an overview of the RELEASE project and discussing distributed Erlang limitations. Then we introduce SD Erlang, its design, motivation, and the main two components, i.e. scalable groups and semi-explicit placement. The scalable groups (s_groups) enable scaling the network of Erlang nodes by eliminating transitive connections, i.e. a node may belong to multiple s_groups where each s_group node has transitive connections with the nodes from the same s_groups and non-transitive connections with other nodes. The semi-explicit placement enables to spawn processes on nodes either in a particular s_group, or with particular attributes (e.g. available hardware or software), or with certain parameters (e.g. least load). We’ll also cover the results of the preliminary validation, and SD Erlang operational semantics and its verification. We conclude the talk by providing a brief overview of the ongoing work and future plans.

Friday 4th June 1976, the Sex Pistols kicked off their first gig, a gig that's considered to change western music culture forever, pioneering the genesis of punk rock.

Wednesday 19th July 2000 had a similar impact on internet scale companies as the Sex Pistols did on music, with the keynote speech by Eric Brewer at the ACM symposium on the [Principles of Distributed Computing] (PODC). Eric Brewer claimed that as applications become more web-based we should stop worrying about data consistency, because if we want high availability in those new distributed applications, then we cannot have data consistency. Two years later, in 2002, Seth Gilbert and Nancy Lynch [formally proved] Brewer's claim as what is known today as the Brewer's Theorem or CAP.

The CAP theorem mandates that a distributed system cannot satisfy both Consistency, Availability and Partition tolerance. In the database ecosystem, many tools claim to solve our data persistence problems while scaling out, offering different capabilities (document stores, key/values, SQL, graph, etc).

In this talk we will explore the CAP theorem

+ We will define what are Consistency, Availability and Partition Tolerance
+ We will explore what CAP means for our applications (ACID vs BASE)
+ We will explore practical applications on MySQL with read slave, MongoDB and Riak based on the work by [Aphyr - Kyle Kingsbury].