M32 Milestone Updates

Here are the updates for Milestone M32 (Mar 10 - Mar 23)

• The monthly Neeto Audit suggested vulnerability fixes and performance and security improvements. We completed the work in NeetoCI, and it's in progress on NeetoDeploy.

• NeetoDeploy Autoscale gem was revamped. It will not affect application performance anymore. Autos-scaling was temporarily disabled for all applications. It is being re-introduced in phases and will be completed soon.

• Traefik upgraded to V3. All the applications except the ones with custom domains have been moved to Traefik V3. The ones with custom domains will be migrated soon.

• We had an issue with Postgresql memory usage. The bigger and more active databases like that of NeetoCal have been steadily consuming more and more RAM and have been running out of resources. We had to increase the RAM to keep them running continuously. We finally figured out the root cause - the PgBackRest backup process that ran in scheduled intervals caused this memory spike. We disabled the PgBackRest backups for now (we have other redundant backup policies in place). We will figure out the fix to keep the memory usage in control and resume the PgBackRest backups soon.

• Switching back and forth between maintenance mode took a significantly long time (minutes) for applications with many custom domains. This was not acceptable, and we have fixed it. The maintenance mode switches no longer depend on the app's number of custom domains.

• NeetoDeploy only sent the deployment status to the review app branches in Github. We have now extended it to even the non-review app branches. This helps track the deployment in staging and production environments.

• Bug fixes in NeetoCI and NeetoDeploy.