Percentage of builds on older build image versions failing due to SSL errors
Incident Report for Netlify
Resolved
We've resolved the issue in our Xenial version build image, and the fix for the few affected builds using our Trusty build image will be to update to Xenial or Focal.
The root cause for this incident was a missing root certificate within our OS certificate chain for Xenial and Trusty, which resulted in build failures for users whose yarn versions weren't cached.
If you're still having trouble with builds, please follow up via the helpdesk by emailing support@netlify.com if you are a paying customer, or this forum thread: https://answers.netlify.com/t/ways-to-work-around-ssl-caused-build-failures-server-certificate-verification-failed/44945
Posted Sep 30, 2021 - 18:43 UTC
Monitoring
A fix has been implemented. The mitigation is automatic on Xenial, but Trusty sites will have to update to Xenial or Focal in order to mitigate. We'll continue to monitor for a while before resolving the incident.
Posted Sep 30, 2021 - 18:25 UTC
Update
We identified the root cause to be an outdated root certificate. We’re in the process of releasing an updated build image.
Posted Sep 30, 2021 - 17:44 UTC
Update
We are continuing to work on a fix for this issue.
Posted Sep 30, 2021 - 17:44 UTC
Update
We're still working on a fix for the problem, and in the meantime we have detailed mitigation advice in this forums post: https://answers.netlify.com/t/ways-to-work-around-ssl-caused-build-failures/44945 . Our Support team will be monitoring that thread and our helpdesk to help you mitigate the problems quickly, while we continue to work on the fix.
Posted Sep 30, 2021 - 16:31 UTC
Identified
Builds using the Xenial and Trusty versions of our build image (configured in each site's Deploy Settings page) that require a download of yarn are failing. This will impact only builds without an already-cached copy of yarn. We're working on a fix but in the meantime, builds on our newer (default) Focal build image are not affected; you can update to the new image to work around this immediately.
Posted Sep 30, 2021 - 16:12 UTC
This incident affected: Build Pipeline.