AWS 重大故障！

这下只有两个 9 了。。。

都挂了十几个小时了吧

是啊昨天晚饭后就开始围观... 到现在都好久了

好像算是结束了

aws 依旧是不错的选择，中心化的力量

为什么连故障分析都没有

这次影响非常大，北美很多券商今天交易系统都登录不进去。

越来越多的企业会下云了, 用云服务平时不出事, 一出事全部玩完.

@stinkytofux 平时不出事，一出事全玩完。。这句话适用于所有的服务，不管你用不用云。

我记得昨天就看到消息了，不过以为只是一次日常的崩溃，没想到这次居然玩这么大

@Dragonish3600 一些地方政府都是非工作时间点断电

@stinkytofux 难道自己就不会炸吗

感觉没上次微软杀毒软件蓝屏影响大。

@stinkytofux #8 下云是不是因为不出事，而是因为出事没什么可做的。自建机房可靠性很难达到云的效果。更多大型企业是在调整方案多云并行

昨天下午 3 点多开始崩，说是美东内部的 dns 配置错误

这要是阿里腾讯不得 15 天啊

:狗头:

AWS 为什么连两地三活都不会建？

别说它有。这时候不起作用，那还什么时候起作用？

看上去一开始是 dynamo dns endpoint 解析出问题 修好以后变成经典 circular dependency

aws 你这 system design 做的也不行啊
Between 11:49 PM PDT on October 19 and 2:24 AM PDT on October 20, we experienced increased error rates and latencies for AWS Services in the US-EAST-1 Region. Additionally, services or features that rely on US-EAST-1 endpoints such as IAM and DynamoDB Global Tables also experienced issues during this time. At 12:26 AM on October 20, we identified the trigger of the event as DNS resolution issues for the regional DynamoDB service endpoints. After resolving the DynamoDB DNS issue at 2:24 AM, services began recovering but we had a subsequent impairment in the internal subsystem of EC2 that is responsible for launching EC2 instances due to its dependency on DynamoDB. As we continued to work through EC2 instance launch impairments, Network Load Balancer health checks also became impaired, resulting in network connectivity issues in multiple services such as Lambda, DynamoDB, and CloudWatch. We recovered the Network Load Balancer health checks at 9:38 AM. As part of the recovery effort, we temporarily throttled some operations such as EC2 instance launches, processing of SQS queues via Lambda Event Source Mappings, and asynchronous Lambda invocations. Over time we reduced throttling of operations and worked in parallel to resolve network connectivity issues until the services fully recovered. By 3:01 PM, all AWS services returned to normal operations.

@stinkytofux #8 整体稳定性远高于自建，所以还是会上云的

AWS 这个故障目前已经结束了，表示稍后会分享详细的事故总结。

@stinkytofux #8 ，下不下云都一样。昨天才经历某一个自建机房停电。很多服务都没必要做多活或者容灾。对比上云，自建机房成本明显高太多了。

好几年前去听一个中小公司的分享，他们老早就阿里云和 AWS 一起用了