ap2002.chtc.wisc.edu shut down unexpectedly last night just before 10 PM and required manual intervention to restart.

Because execution points lost communication with ap2002 for more than 2 hours, running jobs submitted from ap2002 were abandoned. In practice this means when ap2002 restarted, the jobs returned to the “Idle” state in the queue.

We are still investigating the root cause of the shut down, but since we have not yet identified it there is the possibility of future recurrence. We thank you for your patience as we work to address the underlying issue.