Network outage impacting judging backend.
Incident Report for Kattis
Postmortem

The outage was caused by a failure of a Power Distribution Unit, or PDU, in the datacenter, causing one of our racks to partially lose power. We have rerouted the power to an alternative PDU in the rack, and escalated the issue to our Colocation provider.

Posted Dec 06, 2023 - 23:10 CET

Resolved
This incident has been resolved.
Posted Dec 06, 2023 - 23:05 CET
Monitoring
Full judging capacity has been restored.
Posted Dec 06, 2023 - 23:03 CET
Update
Partial restoration of the judging backend has been completed. However, we are running with reduced capacity.
Posted Dec 06, 2023 - 22:34 CET
Update
Staff is on site and troubleshooting the issue.
Posted Dec 06, 2023 - 22:33 CET
Update
Pending submissions will be stored, and judged as soon as the judging backend is back online.
Posted Dec 06, 2023 - 21:29 CET
Update
We are continuing to work on a fix for this issue.
Posted Dec 06, 2023 - 21:26 CET
Identified
We have lost contact with a switch in our core infrastructure. We are sending staff to the datacenter to investigate.
Posted Dec 06, 2023 - 21:25 CET
Investigating
We are currently investigating this issue.
Posted Dec 06, 2023 - 21:22 CET
This incident affected: Kattis Education (Web) and Kattis Recruiting (Web).