Railway - Notice history

All systems operational

Dashboard - Operational

100% - uptime
Nov 2024 · 100.0%Dec · 100.0%Jan 2025 · 100.0%

Previous page

Next page

Nov 2024
Dec 2024
Jan 2025
100% - uptime

Builders - Operational

100% - uptime
Nov 2024 · 100.0%Dec · 100.0%Jan 2025 · 100.0%

Previous page

Next page

Nov 2024
Dec 2024
Jan 2025

Image Registry - Operational

100% - uptime
Nov 2024 · 100.0%Dec · 100.0%Jan 2025 · 100.0%

Previous page

Next page

Nov 2024
Dec 2024
Jan 2025

Third-Party: npm Registry - Operational

Third-Party: Github Integrations - Operational

Third Party: Docker Hub - Operational

Third-Party: GitHub Container Registry - Operational

100% - uptime

EU West (europe-west4 / Amsterdam, Netherlands) - Operational

100% - uptime
Nov 2024 · 99.99%Dec · 100.0%Jan 2025 · 100.0%

Previous page

Next page

Nov 2024
Dec 2024
Jan 2025

US East (us-east4 / Virginia, USA) - Operational

100% - uptime
Nov 2024 · 100.0%Dec · 99.95%Jan 2025 · 100.0%

Previous page

Next page

Nov 2024
Dec 2024
Jan 2025

US West (us-west1 / Oregon, USA) - Operational

100% - uptime
Nov 2024 · 99.93%Dec · 100.0%Jan 2025 · 100.0%

Previous page

Next page

Nov 2024
Dec 2024
Jan 2025

Southeast Asia (asia-southeast1 / Singapore) - Operational

100% - uptime
Nov 2024 · 100.0%Dec · 100.0%Jan 2025 · 100.0%

Previous page

Next page

Nov 2024
Dec 2024
Jan 2025
100% - uptime

US West (us-west1 / Oregon, USA) - Operational

100% - uptime
Nov 2024 · 100.0%Dec · 99.95%Jan 2025 · 100.0%

Previous page

Next page

Nov 2024
Dec 2024
Jan 2025

US East (us-east4 / Virginia, USA) - Operational

100% - uptime
Nov 2024 · 100.0%Dec · 99.95%Jan 2025 · 100.0%

Previous page

Next page

Nov 2024
Dec 2024
Jan 2025

EU West (europe-west4 / Amsterdam, Netherlands) - Operational

100% - uptime
Nov 2024 · 100.0%Dec · 99.95%Jan 2025 · 100.0%

Previous page

Next page

Nov 2024
Dec 2024
Jan 2025

Southeast Asia (asia-southeast1 / Singapore) - Operational

100% - uptime
Nov 2024 · 100.0%Dec · 99.95%Jan 2025 · 100.0%

Previous page

Next page

Nov 2024
Dec 2024
Jan 2025

TCP Proxy - Operational

100% - uptime
Nov 2024 · 100.0%Dec · 100.0%Jan 2025 · 100.0%

Previous page

Next page

Nov 2024
Dec 2024
Jan 2025
100% - uptime

DNS - Operational

100% - uptime
Nov 2024 · 100.0%Dec · 100.0%Jan 2025 · 100.0%

Previous page

Next page

Nov 2024
Dec 2024
Jan 2025

Traffic - Operational

100% - uptime
Nov 2024 · 100.0%Dec · 100.0%Jan 2025 · 100.0%

Previous page

Next page

Nov 2024
Dec 2024
Jan 2025
100% - uptime

Logs - Operational

100% - uptime
Nov 2024 · 100.0%Dec · 100.0%Jan 2025 · 100.0%

Previous page

Next page

Nov 2024
Dec 2024
Jan 2025

Metrics - Operational

100% - uptime
Nov 2024 · 100.0%Dec · 100.0%Jan 2025 · 100.0%

Previous page

Next page

Nov 2024
Dec 2024
Jan 2025
100% - uptime

Payments - Operational

100% - uptime
Nov 2024 · 100.0%Dec · 100.0%Jan 2025 · 100.0%

Previous page

Next page

Nov 2024
Dec 2024
Jan 2025

Notice history

Jan 2025

railway.com is currently offline. We are investigating this incident.
  • Resolved
    Resolved
    This incident has been resolved. The Railway dashboard at https://railway.com was inaccessible for ~10 minutes from roughly 2120UTC to 2130UTC. No user workloads are impacted.
  • Monitoring
    Monitoring
    We have pushed a fix for this issue. This incident only impacts user dashboard access at https://railway.com. There is no impact to workloads hosted on Railway.
  • Investigating
    Investigating
    railway.com is currently offline. We are investigating this incident.
Some builds may fail due to certificate error on pushing images to a registry.
  • Resolved
    Resolved
    A change that improved load balancing of builds accross the builder fleet led to a small proportion of builds being delivered to improperly initialised builders instances. We have now resolved this issue.
  • Identified
    Identified
    We've identified the change that caused this and are rolling it back.
  • Investigating
    Investigating
    Some builds may fail due to certificate error on pushing images to a registry.
We are seeing some deploys failing to go through
  • Resolved
    Resolved
    Deploys are now back to a healthy state
  • Identified
    Identified
    We have identified the issue and are rolling out a fix now
  • Investigating
    Investigating
    We are seeing some deploys failing to go through
A single host in US West has restarted
  • Resolved
    Resolved
    All workloads on the host are now back online
  • Monitoring
    Monitoring
    We are actively monitoring workloads on this host as they come back online
  • Investigating
    Investigating
    A single host in US West has restarted

Dec 2024

A single host in US East is unresponsive due to load on the box.
  • Resolved
    Resolved

    All applications on the affected host have been restored.

  • Identified
    Identified

    Host is being recovered by the team.

  • Investigating
    Investigating
    We are currently investigating this incident.
Services Outage
  • Resolved
    Resolved

    This incident has been resolved. Railway was down for a total of ~10 minutes. We will publish a full retroactive on this incident shortly.

  • Monitoring
    Monitoring

    Services should be fully restored and we'll continue to monitor.

  • Identified
    Identified

    Issue has been identified.

  • Investigating
    Investigating
    We are currently investigating this incident.
Dashboard Maintenance
  • Completed
    December 16, 2024 at 5:30 AM
    Completed
    December 16, 2024 at 5:30 AM
    Maintenance has completed successfully
  • In progress
    December 16, 2024 at 5:00 AM
    In progress
    December 16, 2024 at 5:00 AM
    Maintenance is now in progress
  • Planned
    December 16, 2024 at 5:00 AM
    Planned
    December 16, 2024 at 5:00 AM

    We will be upgrading our high availability database on Sunday at midnight EST. We do not expect any interruption of service. While we do not expect it, during this time there may be dashboard "blips" and potentially delayed builds.

Some builds relying on GitHub fail with connection error
  • Resolved
    Resolved
    This incident has been resolved.
  • Identified
    Identified
    We are continuing to work on a fix for this incident.
Builds using NPM will fail during NPM’s scheduled maintenance.
  • Resolved
    Resolved
    NPM's maintenence window is completed.
  • Investigating
    Investigating
    Builds using NPM will fail during NPM’s scheduled maintenance.

Nov 2024

Singular Google Cloud host failing over. Users scheduled on this machine may notice request timeout/retries
  • Resolved
    Resolved
    Resolved
  • Monitoring
    Monitoring
    Host returned to nominal. Monitoring
  • Investigating
    Investigating
    Singular Google Cloud host failing over. Users scheduled on this machine may notice request timeout/retries
US-East Metal builds have stalled. We’re rolling out a fix
  • Resolved
    Resolved
    We've resolved the incident. All builds back to nominal
  • Investigating
    Investigating
    US-East Metal builds have stalled. We’re rolling out a fix
Users on project pages may see inverted 'dark mode'/'light mode' colors for service tiles. We've rolled out a patch which should be live in 5 minutes
  • Resolved
    Resolved
    Resolved
  • Investigating
    Investigating
    Users on project pages may see inverted 'dark mode'/'light mode' colors for service tiles. We've rolled out a patch which should be live in 5 minutes
DDOS attack being mitigated currently. A small subset of traffic may see a degredation
  • Resolved
    Resolved

    Resolved. A degraded serving instance failed to remove itself from the edge rotation. We're deploying a patch to prevent this from occuring again

  • Monitoring
    Monitoring
    We're stabilized the latency and are moving to monitoring
  • Investigating
    Investigating
    DDOS attack being mitigated currently. A small subset of traffic may see a degredation
We're experiencing elevated error rates on our builders. Deploys may fail to go live during this period
  • Resolved
    Resolved

    This incident has been resolved.

  • Monitoring
    Monitoring
    We implemented a fix and are currently monitoring the result.
  • Investigating
    Investigating
    We're experiencing elevated error rates on our builders. Deploys may fail to go live during this period

Nov 2024 to Jan 2025

Next