Monitoring - We restarted the server and it appears to be functional, however, we have not yet identified the cause and it may recur.
Please remember: do not run intensive commands on the access point - this extends to any commands your AI agent may be running on your behalf!
Apr 20, 2026 - 16:23 CDT
Investigating - Confirmed user reports of being unable to login to ap2002. We're investigating the cause.
Apr 20, 2026 - 15:08 CDT
Resolved -
A fix has been implemented and confirmed to work. Users with idle GPU jobs should remove their jobs (`condor_rm`) and resubmit the jobs, due to an incorrect expression in the jobs' attributes. Newly submitted jobs should match normally.
Apr 15, 10:48 CDT
Monitoring -
A fix has been implemented. Users with idle GPU jobs should remove their jobs (`condor_rm`) and resubmit the jobs, due to an incorrect expression in the jobs' attributes. We are monitoring the situation.
Apr 14, 15:24 CDT
Investigating -
We've received reports of GPU jobs failing to match and start up, staying stuck in the IDLE state. We're investigating the cause and will update this statuspage as more information or a solution is implemented.
Apr 13, 16:50 CDT
Resolved -
We identified the cause of the issue. When all licenses are checked out, any new jobs requesting licenses will fail with the "Failed to connect to token server" message. We have contact users who are using a majority of the licenses.
All Gurobi users must use `concurrency_limits = GUROBI:1` in their Gurobi jobs' submit files. This ensures that when all licenses are checked out, jobs will remain in idle instead of failing.
Apr 15, 10:47 CDT
Investigating -
Some users are reporting that their Gurobi jobs are failing with the message, "Failed to connect to token server". We are currently investigating. We encourage users using Gurobi to double-check that they are using `concurrency_limits = GUROBI:1` in their submit file.
Apr 13, 14:29 CDT
Resolved -
We've fixed the issue and are working through the backlog of requests.
Apr 10, 13:30 CDT
Identified -
An issue with our accounting system is preventing us from creating new CHTC accounts or modifying existing CHTC accounts. This does not affect anyone with an existing account nor their ability to login, but will delay us from creating accounts for new users or giving existing users access to new resources.
Apr 9, 16:56 CDT
Resolved -
The Gurobi license has been renewed. Users may now submit Gurobi jobs.
Apr 8, 13:45 CDT
Identified -
The Gurobi license for CHTC has expired. We are working with campus IT to renew the license. In the meantime, user jobs attempting to use the Gurobi license will likely fail due to a "license expired" error.
Apr 2, 15:13 CDT