[已解決] API 錯誤率增加 | [已解决] API 错误率增加 | Increased API Error Rates

10:43 PM PDT 我们正在调查AP-EAST-1区域中EC2 API错误率上升的原因 | 我們正在調查AP-EAST-1區域中EC2 API錯誤率上升的原因 | We are investigating increased API error rates for the EC2 APIs in the AP-EAST-1 Region.

10:51 PM PDT 我们确认在 AP-EAST-1 区域内的单个可用区中 EC2 API 的错误率上升和 某些EBS卷的性能下降 | 我們確認在 AP-EAST-1 區域內的單個可用區中 EC2 API 的錯誤率上升和 某些EBS卷的性能下降 | We can confirm increased error rates for the EC2 APIs and degraded performance for some EBS volumes within a single Availability Zone within the AP-EAST-1 Region

11:08 PM PDT 我们持续调查 EC2 API 的错误率增加、部分 EBS 卷的性能下降以及 AP-EAST-1 区域中单个可用区 (ape1-az2) 内的 EC2 实例连接问题。 AP-EAST-1 区域内的其他可用区不受此事件的影响。|我們持續調查 EC2 API 的錯誤率增加、部分 EBS 卷的性能下降以及 AP-EAST-1 區域中單個可用區 (ape1-az2) 內的 EC2 實例連接問題。 AP-EAST-1 區域內的其他可用區不受此事件的影響。| We continue to investigate increased error rates for the EC2 APIs, degraded performance for some EBS volumes and EC2 instance connectivity issues within a single Availability Zone (ape1-az2) in the AP-EAST-1 Region. Other Availability Zones within the AP-EAST-1 Region are not affected by this event.

11:27 PM PDT 我们持续调查 EC2 API 的错误率增加、部分 EBS 卷的性能下降以及 AP-EAST-1 区域中单个可用区 (ape1-az2) 内的 EC2 实例连接问题。 我们正在确认此事件的根本原因，但现阶段还无法确定。由于 AP-EAST-1 区域内的其他可用区不受此事件的影响，我们建议您暂时不要使用受影响的可用区。| 我們持續調查 EC2 API 的錯誤率增加、部分 EBS 卷的性能下降以及 AP-EAST-1 區域中單個可用區 (ape1-az2) 內的 EC2 實例連接問題。 我們正在確認此事件的根本原因，但現階段還無法確定。由於 AP-EAST-1 區域內的其他可用區不受此事件的影響，我們建議您暫時不要使用受影響的可用區。| We continue to investigate increased error rates for the EC2 APIs, degraded performance for some EBS volumes and EC2 instance connectivity issues within a single Availability Zone (ape1-az2) in the AP-EAST-1 Region. We are making progress towards determining root cause for this event, but have not been able to determine it at this stage. Since other Availability Zones within the AP-EAST-1 Region are not affected by this event, we recommend that you fail away from the affected Availability Zone.

Jun 18, 12:03 AM PDT 我们持续调查 EC2 API 的错误率增加、部分 EBS 卷的性能下降以及 AP-EAST-1 区域中单个可用区 (ape1-az2) 内的 EC2 实例连接问题。 我们正在确认此事件的根本原因，我们认为这与 EC2 实例和附加 EBS 卷之间的通信有关。这会导致可用区内受影响 EBS 卷的性能下降，进而发生操作系统内的 IO 卡住而导致 EC2 实例受损。在我们继续努力解决问题的同时，我们建议您从受影响的可用区进行故障转移。 | 我們持續調查 EC2 API 的錯誤率增加、部分 EBS 卷的性能下降以及 AP-EAST-1 區域中單個可用區 (ape1-az2) 內的 EC2 實例連接問題。 我們正在確認此事件的根本原因，我們認為這與 EC2 實例和附加 EBS 卷之間的通信有關。這會導致可用區內受影響 EBS 卷的性能下降，進而發生操作系統內的 IO 卡住而導致 EC2 實例受損。在我們繼續努力解決問題的同時，我們建議您從受影響的可用區進行故障轉移。 | We continue to investigate increased error rates for the EC2 APIs, degraded performance for some EBS volumes and EC2 instance connectivity issues within a single Availability Zone (ape1-az2) in the AP-EAST-1 Region. We are making progress in determining the root cause and believe it is related to communication between EC2 instances and attached EBS volumes. This leads to degraded performance for affected EBS volumes within the Availability Zone and can also lead to impaired EC2 instances due to stuck IO within the operating system. While we continue to work towards resolving the issue, we recommend that you fail away from the affected Availability Zone if you have not already done so.

Jun 18, 12:47 AM PDT 我們已確定導致 EC2 API 的錯誤率增加、AP-EAST-1 區域中某些 EBS 磁碟區和 EC2 實例連線問題的效能降低問題的根本原因。在受影響的可用區內的某些 EBS 儲存伺服器受損，導致受影響的 EBS 磁碟區效能降低。我們正在採取措施解決 EBS 儲存伺服器損害，這應該能開始解決受影響的 EC2 實例和 EBS 磁碟區的問題。在我們繼續努力解決問題的同時，我們建議您暫時不要使用受影響的可用區。| We have determined the root cause of the issue causing increased error rates for the EC2 APIs, degraded performance for some EBS volumes and EC2 instance connectivity issues within a single Availability Zone (ape1-az2) in the AP-EAST-1 Region. Some EBS storage servers within the affected Availability Zone are impaired, which is causing the degraded performance for the affected EBS volumes. We are taking steps to address the EBS storage server impairments, which should begin to resolve the issue for affected EC2 instances and EBS volumes. We continue to recommend that you fail away from the affected Availability Zone if you have not already done so.

Jun 18, 12:57 AM PDT 我們持續看到受影響的可用性區域內 EBS 磁碟區的效能降低。絕大多數受影響的 EBS 磁碟區現在已經復原，我們正在處理剩餘的 EBS 磁碟區。雖然大部分的服務現在都在受影響的可用性區域內看到復原，但我們還是不建議在完成完整復原之前，還是不建議回復至可用區域。我們將繼續為您提供更新。 我們已經開始採取措施來解決 EBS 儲存伺服器問題，並且正在看到某些受影響的 EBS 磁碟區的復原。我們將繼續處理剩餘的受損 EBS 儲存伺服器，以完全解決問題. | We have begun taking steps to address the EBS storage server impairments, and are seeing recovery for some of the affected EBS volumes. We will continue to work on remaining impaired EBS storage servers to fully resolve the issue.

Jun 18, 1:27 AM PDT 我们继续看到受影响的可用区内 EBS 卷的性能下降。大多数受影响的 EBS 卷现已恢复，我们将继续努力恢复其余受影响的卷。我们继续努力争取全面解决这个问题 | 我們持續看到受影響的可用性區域內 EBS 磁碟區的效能降低。大部分受影響的 EBS 磁碟區現在都已經復原，我們會繼續修復其餘受影響的磁碟區。我们继续努力全面解决这一问题 | We continue to see an improvement in degraded performance for EBS volumes within the affected Availability Zone. The majority of the affected EBS volumes have now been recovered and we continue to work on recovering the remaining affected volumes. We continue to work toward full resolution of the issue.

Jun 18, 1:53 AM PDT 我们继续看到受影响的可用区内 EBS 卷的性能下降。绝大多数受影响的 EBS 卷现已恢复，我们正在处理剩余的 EBS 卷。虽然大多数服务现在都在受影响的可用区内看到恢复，但我们建议在完成完全恢复之前，不要回到可用区。我们将继续为您提供最新信息. 我们已开始采取措施来解决 EBS 存储服务器损坏问题，并且看到一些受影响的 EBS 卷正在恢复。我们将继续处理剩余受损的 EBS 存储服务器，以充分解决此问题。| We continue to see an improvement in degraded performance for EBS volumes within the affected Availability Zone. The vast majority of affected EBS volumes have now been recovered and we are working on the remaining EBS volumes. While most services are now seeing recovery within the affected Availability Zone, we do not yet recommend failing back to the Availability Zone until we have completed full recovery. We will continue to provide you with updates.

Jun 18, 2:36 AM PDT 从 12:58 PM UTC+8 开始，在 AP-EAST-1 地区的一个可用区（ape1-az2）内，我们遇到了 EC2 APIs 的错误率增加、一些 EBS 卷的性能下降和 EC2 实例连接问题。该问题的根本原因是受影响的可用性区域内底层 EBS 存储服务器的性能下降。工程师采取了行动，缓解和解决 EBS 存储服务器性能下降的问题，从而解决了这个问题。在 03:45 PM UTC+8 性能下降的 EBS 卷开始恢复，到 03:58 PM UTC+8，绝大部分受影响的 EBS 卷已经恢复。我们继续在少数仍在经历性能下降的 EBS 卷上工作，并将通过个人健康仪表板或这些卷提供进一步的更新。所有的服务现在都在受影响的可用性区域内正常运行。这个问题已经得到解决，服务运行正常。| 從 12:58 PM UTC+8 開始，在 AP-EAST-1 地區的一個可用區（ape1-az2）內，我們遇到了 EC2 APIs 的錯誤率增加、一些 EBS 卷的性能下降和 EC2 實例連接問題。該問題的根本原因是受影響的可用性區域內底層 EBS 存儲服務器的性能下降。工程師採取了行動，緩解和解決 EBS 存儲服務器性能下降的問題，從而解決了這個問題。在 03:45 PM UTC+8 性能下降的 EBS 捲開始恢復，到 03:58 PM UTC+8，絕大部分受影響的 EBS 卷已經恢復。我們繼續在少數仍在經歷性能下降的 EBS 捲上工作，並將通過個人健康儀表板或這些卷提供進一步的更新。所有的服務現在都在受影響的可用性區域內正常運行。這個問題已經得到解決，服務運行正常。| Starting at 9:58 PM PDT we experienced increased error rates for the EC2 APIs, degraded performance for some EBS volumes and EC2 instance connectivity issues within a single Availability Zone (ape1-az2) in the AP-EAST-1 Region. The root cause of the issue was degraded performance for underlying EBS storage servers within the affected Availability Zone. Engineers took action to mitigate and resolve the degraded EBS storage server performance, which resolved the issue. At 12:41 AM PDT, EBS volumes with degraded performance began to recover and by 12:58 AM PDT, the vast majority of affected EBS volumes had recovered. We continue to work on a small number of EBS volumes that are still experiencing degraded performance, and will provide further updates via the Personal Health Dashboard for those volumes. All services are now operating normally within the affected Availability Zone. The issue has been resolved and the service is operating normally.