Amazon Disruption Produces Cloud Outage Spiral - InformationWeek

InformationWeek is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

IoT
IoT
Comments
Amazon Disruption Produces Cloud Outage Spiral
Newest First  |  Oldest First  |  Threaded View
Charlie Babcock
50%
50%
Charlie Babcock,
User Rank: Author
10/2/2015 | 6:14:11 PM
Amazon's public statement on the disruption
Amazon produced a public statement on the outage but I did not receive it in time to include in this story. It was: Between 2:13 AM and 7:10 AM PDT on September 20, 2015, Amazon Web Services (AWS) experienced significant error rates with read and write operations for the Amazon DynamoDB service in the US-East Region, which impacted some other AWS services in that region, and caused some AWS customers to experience elevated error rates.  

 
Charlie Babcock
50%
50%
Charlie Babcock,
User Rank: Author
9/22/2015 | 6:05:54 PM
Cloud processes designed to protect your system may eat it alive
"Many of us are customers of Amazon without knowing it." Good point, Tom. The cloud providers still don't know how to get a circuit breaker into an automated process gone awry. Amazon's Easter weekend shutdown four years ago was triggered by human error, when an operations person unplugged a trunk network, then replugged it into a backup network. That made all the data sets visible on that trunk line disappear, which triggered a "remirroring storm" as the cloud software tried to make up for the lost data by creating new sets. While that was going on, everything else pretty much ground to a halt--for 2-3 days.
Thomas Claburn
50%
50%
Thomas Claburn,
User Rank: Author
9/22/2015 | 4:55:22 PM
Re: Cloud shutdown as an automated process
One company affected by the AWS issues was Scout Alarm. This apparently limited the ability of customers to arm and disarm their alarm systems. As we become more dependent on cloud services, let hope Amazon improves reliability. Many of us are AWS customers without knowing it.
Charlie Babcock
50%
50%
Charlie Babcock,
User Rank: Author
9/22/2015 | 4:34:44 PM
Cloud shutdown as an automated process
On Feb. 29, 2012, an expiration date for security certificates that didn't recognize a a leap year caused one virtual machine after another to fail upon its attempted start. Three failed starts in a row caused a host in Microsoft's Azure cloud to conclude that hardware was failing, when it wasn't, and move the faulty virtual machine to another server, where the error could repeat itself. In this manner does cloud software bring down the cloud as an unexpected automated process.   


2020 State of DevOps Report
2020 State of DevOps Report
Download this report today to learn more about the key tools and technologies being utilized, and how organizations deal with the cultural and process changes that DevOps brings. The report also examines the barriers organizations face, as well as the rewards from DevOps including faster application delivery, higher quality products, and quicker recovery from errors in production.
News
The State of Chatbots: Pandemic Edition
Jessica Davis, Senior Editor, Enterprise Apps,  9/10/2020
Commentary
Deloitte on Cloud, the Edge, and Enterprise Expectations
Joao-Pierre S. Ruth, Senior Writer,  9/14/2020
Slideshows
Data Science: How the Pandemic Has Affected 10 Popular Jobs
Cynthia Harvey, Freelance Journalist, InformationWeek,  9/9/2020
Register for InformationWeek Newsletters
Video
Current Issue
IT Automation Transforms Network Management
In this special report we will examine the layers of automation and orchestration in IT operations, and how they can provide high availability and greater scale for modern applications and business demands.
White Papers
Slideshows
Twitter Feed
Sponsored Live Streaming Video
Everything You've Been Told About Mobility Is Wrong
Attend this video symposium with Sean Wisdom, Global Director of Mobility Solutions, and learn about how you can harness powerful new products to mobilize your business potential.
Sponsored Video
Flash Poll