Connect: Voice call service issue

Summary

Two issues with voice calls (Sep 30-Oct 1, 2021) are resolved. We are working to mitigate against the possibility of similar problems in the future.

What happened?

CatapultConnect uses Let's Encrypt for secure communication over the web. A change to their security certificate had consequences for millions of websites including CatapultConnect. In our case, it halted communicate with our telecom provider. During this time, voice calls announced an application error rather than the normal message. After communication resumed, queued up messages released late in the night.

What now?

We are exploring options to mitigate against similar problems in the future. This includes adding additional monitoring for the CatapultConnect sending service, and possibly limiting the hours that CatapultConnect is allowed to send non-emergency messages. Some of these changes may take time since they involve research, development, and testing.

We apologize for the series of events. We know many of you were unable to release messages during the outage or fielded complaints from parents and student contacts who received messages at unreasonable hours of the night.