re: How Netflix Is Ruining Cloud Computing
I would put it this way (and I plan on writing in more detail on this): I think from an IaaS provider, you need the equivalent of EC2 (at least a few machine types), S3, EBS (snapshot/restore/detach/attach), and Security Groups from AWS. Then you should interact with that IaaS through an abstraction layer, not directly with their API, so you can switch to other, similar providers. (Netflix does argue that since they're at least--in some cases--using boto/similar libraries, it will be easier to add support for other clouds, but I am skeptical).
So, on that abstraction layer. Ultimately, from a software architecture standpoint, you can either start from a "I'm going to build a library that interfaces with X API", or you can start from, "I'm going to build a generalized set of interfaces to a certain type of service, and I'm going to translate from my generalized interfaces to specific implementations". I believe--and I think the evidence shows--if you start with the former, it's very hard to connect to other APIs that do similar things, just not in the exact same way. However, if you do the latter, and you are explicit from the beginning that you'll be translating from your outward-facing interfaces to other APIs, you'll have set the project up properly. These are really quite different software engineering projects, and I think that's why you hear the Netflix engineers really focus on the idea that other platforms will have to adopt the AWS API for things to work properly with their tools.
I view the multi-cloud issue as (a) critical, and (b) very hard, for the reasons you mentioned above. So much software uses RDBMS, and multi-master replication is generally speaking not the most reliable thing. Netflix has a good article about how they treat different types of data to handle their multi-region deployment, and I think we're going to have to move to architectures like that to take full advantage of the cloud.
In the end, I think the debate in these comments has more to do with us talking past each other than anything else (as Charles Babcock points out here somewhere). I think we all agree that Netflix has solved their particular use case in a great way, and that open sourcing their tools and giving speeches on their architecture is very useful. Netflix appears to view their AWS API lock-in as a necessary thing to keep for the future (and I don't see why they shouldn't work toward abstracting all of it) and Netflix appears to think that everyone approaching their tools will be capable of making an informed decision about the potential issues of using them (and I think that's a really bad assumption).
2020 State of DevOps Report
Download this report today to learn more about the key tools and technologies being utilized, and how organizations deal with the cultural and process changes that DevOps brings. The report also examines the barriers organizations face, as well as the rewards from DevOps including faster application delivery, higher quality products, and quicker recovery from errors in production.
Special Report: Why Performance Testing is Crucial TodayThis special report will help enterprises determine what they should expect from performance testing solutions and how to put them to work most efficiently. Get it today!
Tweet This
User Rank: Apprentice
4/30/2013 | 3:54:40 PM