Railway - Builds are queuing up due to a bad rollout on the workers. We're reverting. Builds triggered in the last 10 minutes will start to go out without needing to be retriggered – Incident details

All systems operational

Builds are queuing up due to a bad rollout on the workers. We're reverting. Builds triggered in the last 10 minutes will start to go out without needing to be retriggered

Resolved
Degraded performance
Started 10 months agoLasted 10 minutes

Affected

Builds (Legacy)

Degraded performance from 2:49 AM to 2:56 AM, Operational from 2:56 AM to 2:58 AM

Build Machines (GCP)

Degraded performance from 2:49 AM to 2:56 AM, Operational from 2:56 AM to 2:58 AM

Updates
  • Resolved
    Resolved
    Builds are going out nominally.
  • Monitoring
    Monitoring
    We have restored the build queue and are now monitoring the health of remaining builds.
  • Investigating
    Investigating
    Builds are queuing up due to a bad rollout on the workers. We're reverting. Builds triggered in the last 10 minutes will start to go out without needing to be retriggered