Monitoring has been completed, and the issue has been resolved. The root cause was a rare and temporary AWS hardware failure when the search indexing subsystem scaled up based on increased traffic. We worked with AWS to resolve the issue and as a preventive measure tripled the capacity dedicated to the subsystem. We further monitored the situation for another 15 hours to confirm indexed documents returned to full completion and that this had resolved the resulting search and licensing errors.
Posted Feb 06, 2024 - 18:26 UTC
Monitoring
A fix has been fully implemented and we are monitoring the results. Please contact support@cloudtheapp.com if you are still seeing any errors on search or licensing. Thanks
Posted Feb 06, 2024 - 02:18 UTC
Update
We have identified a solution to the issue and continue to actively work on applying it.
Posted Feb 06, 2024 - 00:39 UTC
Identified
we have identified the root cause of the issue and are working closely with Amazon (AWS) to resolve it.
Posted Feb 05, 2024 - 23:37 UTC
Investigating
As of 8PM UTC Mon 5-Feb, the searched document index is itemporarily returning inconsistent search results, and invalid team/app subscription errors are occuring. We are currently investigating the root cause and will update asap