Railway - Notice history

A single host in the us-west1 region has encountered a hardware failure. We're investigating and recovering the host.

We have identified and implemented a fix for this issue, and are continuing to monitor the results. Our builders in EU West was under-provisioned leading to slower builds. We have scaled it up, and builds should now be going through normally.

New builds are taking longer to process than usual

This is a retrospective incident: Healthchecks on user workloads were failing for the period between 7:10 and 7:40am UTC due to the implementation of an overrestrictive firewall rule. This caused new deployments on services with a healthcheck configured to fail due to a failed healthcheck. We reverted the rule, and healthchecks began operating normally.

This is a retrospective incident: A single host in the asia-southeast1 region was unavailable between 11:40 and 11:59am UTC due to host failure. We automatically failed-over to a new host and all workloads were brought back online.

Between 12:09 and 12:19 UTC our edge network processed a large amount of traffic. Requests for your application may have not completed.

At 7:45am UTC, a single host in asian-southeast1 encountered a hardware failure. An automatic failover was begun and all affected workloads were restored by 8am UTC. We're running final checks to ensure the everything is operational.

A failure in our internal incident tooling meant this incident wasn't posted to instatus until now.