Monitoring - The 404 and other odd routing issues should now be resolved. Since this was a difficult issue to isolate and fix we won't declare victory immediately, but will continue to monitor to make sure there's nothing remaining to address.

Thank you for your considerable patience while we worked on this, and as always, do email tracker@pivotal.io if there's anything we can help with.
Feb 21, 22:00 UTC
Update - As part of our ongoing efforts to correct the existing production issues in Tracker, we’ll be doing an emergency update within the next hour. This update has the potential to cause downtime in Tracker, though we’ll do our best to avoid that. Thank you for your continued patience and understanding.
Feb 21, 19:46 UTC
Update - We've significantly reduced the rate at which you might see messages forcing you to click Reload. Having done as much as we can to work around the underlying issue, to prevent you from seeing that and losing data, we're continuing to work with the Google Cloud Platform team to get the root cause fixed. We'll keep updating this with any news, as we continue to give this our best attention.

Do email tracker@pivotal.io if we can help or answer any questions meanwhile.
Feb 20, 21:42 UTC
Update - We have deployed a fix that should at least significantly reduce the chances of you seeing a reload message. We're continuing work to identify any remaining instances of forced reloads.

You may continue to see 404's in browser console, even if you don't see the reload message) but those should be harmless.

Apologies for any remaining problems while we continue work. We'll continue until we're confident the issue is fully addressed.
Feb 20, 18:05 UTC
Update - Apologies for the issue escalating and getting worse, including constant 404's for some customers, while we continue to work on a fix. We're fixing and testing simultaneously to try and address this as quickly and safely as possible.
Feb 20, 17:50 UTC
Identified - We've identified a Google Cloud Platform issue that Tracker is not handling correctly, which is resulting in the 404 errors for a small percentage of activities in Tracker. However, unfortunately it does seem that data loss is possible if you see this error before a story is saved. We're working on it urgently and will provide further updates as soon as we have them.
Feb 20, 16:12 UTC
Investigating - We are seeing infrequent and intermittent 404 server errors on actions such as saving a story in Tracker. If you are in a Workspace, you might see the error caused by someone else saving a story in a project in your Workspace. Data doesn't appear to be being lost, so clicking reload on the error message should have no ill effect. However we are actively investigating and will address the issue as soon as possible. Sorry for any inconvenience meanwhile.
Feb 19, 23:55 UTC
Tracker   Operational
SendGrid   Operational
GitHub   Operational
Google Cloud Platform Google Cloud SQL   Operational
Google Cloud Platform Google Compute Engine   Operational
Google Cloud Platform Google Cloud Storage   Operational
AWS s3-us-standard   Operational
Operational
Degraded Performance
Partial Outage
Major Outage
Maintenance
System Metrics Month Week Day
Tracker API V5 Uptime
Fetching
Tracker Web application uptime
Fetching
Server Response Time
Fetching
Past Incidents
Feb 22, 2018

No incidents reported today.

Feb 18, 2018

No incidents reported.

Feb 17, 2018

No incidents reported.

Feb 16, 2018

No incidents reported.

Feb 15, 2018

No incidents reported.

Feb 14, 2018

No incidents reported.

Feb 13, 2018
Resolved - All delayed jobs have since caught up/dequeued. If you're still experiencing any issues, please don't hesitate to email us at tracker@pivotal.io. Thank you for your patience while we worked to resolve this, and apologies again for the inconvenience.
Feb 13, 19:23 UTC
Monitoring - We've identified and resolved the issues that were causing some functions to become backed up/delayed. As they dequeue, you will begin to see any delayed emails, webhook activities, etc, come through. If you were waiting on an attachment to upload, it might be better to cancel the in-progress upload and re-upload it. Once the queue is completely caught up, we'll provide another update. Our apologies again for the inconvenience this has caused.
Feb 13, 18:39 UTC
Investigating - We are urgently investigating an issue that is causing some functions to back-up. This means that attachment uploads, emails, iOS notifications as well as webhooks/integrations may be delayed. We are urgently working to correct this issue and will update this page as soon as there is more information. Our sincere apologies for the inconvenience meanwhile.
Feb 13, 17:23 UTC
Feb 12, 2018

No incidents reported.

Feb 11, 2018

No incidents reported.

Feb 10, 2018

No incidents reported.

Feb 9, 2018

No incidents reported.

Feb 8, 2018

No incidents reported.