Railway - Notice history

All systems operational

Dashboard - Operational

100% - uptime
Oct 2024 · 100.0%Nov · 100.0%Dec · 100.0%

Previous page

Next page

Oct 2024
Nov 2024
Dec 2024
100% - uptime

Builders - Operational

100% - uptime
Oct 2024 · 100.0%Nov · 100.0%Dec · 100.0%

Previous page

Next page

Oct 2024
Nov 2024
Dec 2024

Image Registry - Operational

100% - uptime
Oct 2024 · 100.0%Nov · 100.0%Dec · 100.0%

Previous page

Next page

Oct 2024
Nov 2024
Dec 2024

Third-Party: npm Registry - Operational

Third-Party: Github Integrations - Operational

Third Party: Docker Hub - Operational

Third-Party: GitHub Container Registry - Operational

100% - uptime

EU West (europe-west4 / Amsterdam, Netherlands) - Operational

100% - uptime
Oct 2024 · 99.98%Nov · 99.99%Dec · 100.0%

Previous page

Next page

Oct 2024
Nov 2024
Dec 2024

US East (us-east4 / Virginia, USA) - Operational

100% - uptime
Oct 2024 · 100.0%Nov · 100.0%Dec · 100.0%

Previous page

Next page

Oct 2024
Nov 2024
Dec 2024

US West (us-west1 / Oregon, USA) - Operational

100% - uptime
Oct 2024 · 99.99%Nov · 99.93%Dec · 100.0%

Previous page

Next page

Oct 2024
Nov 2024
Dec 2024

Southeast Asia (asia-southeast1 / Singapore) - Operational

100% - uptime
Oct 2024 · 100.0%Nov · 100.0%Dec · 100.0%

Previous page

Next page

Oct 2024
Nov 2024
Dec 2024
100% - uptime

US West (us-west1 / Oregon, USA) - Operational

100% - uptime
Oct 2024 · 100.0%Nov · 100.0%Dec · 100.0%

Previous page

Next page

Oct 2024
Nov 2024
Dec 2024

US East (us-east4 / Virginia, USA) - Operational

100% - uptime
Oct 2024 · 100.0%Nov · 100.0%Dec · 100.0%

Previous page

Next page

Oct 2024
Nov 2024
Dec 2024

EU West (europe-west4 / Amsterdam, Netherlands) - Operational

100% - uptime
Oct 2024 · 100.0%Nov · 100.0%Dec · 100.0%

Previous page

Next page

Oct 2024
Nov 2024
Dec 2024

Southeast Asia (asia-southeast1 / Singapore) - Operational

100% - uptime
Oct 2024 · 100.0%Nov · 100.0%Dec · 100.0%

Previous page

Next page

Oct 2024
Nov 2024
Dec 2024

TCP Proxy - Operational

100% - uptime
Oct 2024 · 100.0%Nov · 100.0%Dec · 100.0%

Previous page

Next page

Oct 2024
Nov 2024
Dec 2024
100% - uptime

US West (us-west1 / Oregon, USA) - Operational

100% - uptime
Oct 2024 · 100.0%Nov · 100.0%Dec · 100.0%

Previous page

Next page

Oct 2024
Nov 2024
Dec 2024

US East (us-east4 / Virginia, USA) - Operational

100% - uptime
Oct 2024 · 100.0%Nov · 100.0%Dec · 100.0%

Previous page

Next page

Oct 2024
Nov 2024
Dec 2024

EU West (europe-west4 / Amsterdam, Netherlands) - Operational

100% - uptime
Oct 2024 · 100.0%Nov · 100.0%Dec · 100.0%

Previous page

Next page

Oct 2024
Nov 2024
Dec 2024

Southeast Asia (asia-southeast1 / Singapore) - Operational

100% - uptime
Oct 2024 · 100.0%Nov · 100.0%Dec · 100.0%

Previous page

Next page

Oct 2024
Nov 2024
Dec 2024
100% - uptime

DNS - Operational

100% - uptime
Oct 2024 · 100.0%Nov · 100.0%Dec · 100.0%

Previous page

Next page

Oct 2024
Nov 2024
Dec 2024

Traffic - Operational

100% - uptime
Oct 2024 · 100.0%Nov · 100.0%Dec · 100.0%

Previous page

Next page

Oct 2024
Nov 2024
Dec 2024
100% - uptime

Logs - Operational

100% - uptime
Oct 2024 · 100.0%Nov · 100.0%Dec · 100.0%

Previous page

Next page

Oct 2024
Nov 2024
Dec 2024

Metrics - Operational

100% - uptime
Oct 2024 · 100.0%Nov · 100.0%Dec · 100.0%

Previous page

Next page

Oct 2024
Nov 2024
Dec 2024
100% - uptime

Payments - Operational

100% - uptime
Oct 2024 · 100.0%Nov · 100.0%Dec · 100.0%

Previous page

Next page

Oct 2024
Nov 2024
Dec 2024

Notice history

Dec 2024

Builds using NPM will fail during NPM’s scheduled maintenance.
  • Resolved
    Resolved
    NPM's maintenence window is completed.
  • Investigating
    Investigating
    Builds using NPM will fail during NPM’s scheduled maintenance.
Users may see intermittent 404s or request failures on the dashboard
  • Resolved
    Resolved
    We have found the issue to be limited to multiple sessions from a single IP. We have now reconfigured those limits to be more permissive.
  • Investigating
    Investigating
    Users may see intermittent 404s or request failures on the dashboard. We are investigating.
A single host in US West is experiencing degraded performance
  • Resolved
    Resolved

    The host has fully recovered.

  • Monitoring
    Monitoring

    All applications are back online.

  • Update
    Update

    Applications on this host are gradually coming back online.

  • Identified
    Identified

    Host is being restarted to recover workloads.

  • Investigating
    Investigating
    We are currently investigating this incident.

Nov 2024

Singular Google Cloud host failing over. Users scheduled on this machine may notice request timeout/retries
  • Resolved
    Resolved
    Resolved
  • Monitoring
    Monitoring
    Host returned to nominal. Monitoring
  • Investigating
    Investigating
    Singular Google Cloud host failing over. Users scheduled on this machine may notice request timeout/retries
US-East Metal builds have stalled. We’re rolling out a fix
  • Resolved
    Resolved
    We've resolved the incident. All builds back to nominal
  • Investigating
    Investigating
    US-East Metal builds have stalled. We’re rolling out a fix
Users on project pages may see inverted 'dark mode'/'light mode' colors for service tiles. We've rolled out a patch which should be live in 5 minutes
  • Resolved
    Resolved
    Resolved
  • Investigating
    Investigating
    Users on project pages may see inverted 'dark mode'/'light mode' colors for service tiles. We've rolled out a patch which should be live in 5 minutes
DDOS attack being mitigated currently. A small subset of traffic may see a degredation
  • Resolved
    Resolved

    Resolved. A degraded serving instance failed to remove itself from the edge rotation. We're deploying a patch to prevent this from occuring again

  • Monitoring
    Monitoring
    We're stabilized the latency and are moving to monitoring
  • Investigating
    Investigating
    DDOS attack being mitigated currently. A small subset of traffic may see a degredation
We're experiencing elevated error rates on our builders. Deploys may fail to go live during this period
  • Resolved
    Resolved

    This incident has been resolved.

  • Monitoring
    Monitoring
    We implemented a fix and are currently monitoring the result.
  • Investigating
    Investigating
    We're experiencing elevated error rates on our builders. Deploys may fail to go live during this period

Oct 2024

We are investigating an issue with the edge network proxy
  • Resolved
    Resolved

    We have resolved this issue.

  • Monitoring
    Monitoring

    A fix has been implemented and we're currently monitoring the edge network proxy.

  • Identified
    Identified

    We have identified the issue as a DDoS attack. We're in the process of applying additional mitigations

  • Investigating
    Investigating

    We are currently investigating this incident.

A host in the us-west region experienced an unexpected restarts. We're recovering the last workloads on this host that failed to restore automatically.
  • Resolved
    Resolved
    The host has fully recovered and all impacted workloads are back online.
  • Investigating
    Investigating
    A host in the us-west region experienced an unexpected restarts. We're recovering the last workloads on this host that failed to restore automatically.
[Retroactive] A single host in US East has restarted
  • Resolved
    Resolved

    (This is a retroactive incident) During this time we noticed a single host in the US East 1 region had restarted and we quickly worked to bring containers back online, users may have not been able to dial their deployments or connect to databases during the restart efforts. This incident has been resolved.

[Retroactive] Two hosts in EU West experienced degraded performance
  • Resolved
    Resolved

    This is a retroactive incident which is resolved. For the period of the incident (11:45-12:35 UTC), some workloads may have exhibited connection problems. The incident was due to resource exhaustion on the two hosts due to unaccounted resource thresholds. We have improved our monitoring to better respond to this type of incident and are working on preventative measures in our orchestration.

We have observed degraded performance of the proxy layer. Users may experience dropped connections. We are investigating.
  • Resolved
    Resolved
    This incident has been resolved.
  • Monitoring
    Monitoring
    We have taken action to mitigate the issue and are monitoring.
  • Investigating
    Investigating

    We have observed degraded performance of the edge proxy layer. Users may experience dropped connections. We are investigating.

Oct 2024 to Dec 2024

Next