Create a cut-down version of Lagoon suitable for local build/startup/testing

[tobybellwood]

Currently a full build of Lagoon locally will start all the services in the docker-compose file.

This starts a stack of 35(!) services, which are then used to deploy local/test sites into either minishift or k3s depending on your setup.

In order for all 35 services to start, you also need to build the full repertoire of 110(!!) images.

We should look at providing a cut-down version available via the makefile to spin-up only what's needed to run the Lagoon stack locally. This would also mean that the images used to build them would also not be required to be built.

From a quick look, the following could be considered as unnecessary for local build:

• logging (ELK stack and forwarders),
• notifications (Slack, email, Teams and RocketChat)
• Container Registry (Harbor) and image scanning (Clair and Trivy)
• Other utilities (backup-handler, auto-idler, storage-calculator)

We should provide the documentation needed to add them into a local build should you wish to, and of course, the full docker-compose option will still be there.

From my rough estimates, a third of our images are "core" Lagoon, a third are application (php, nginx, python) images, and a third are production necessary images.

Being able to run the leaner, meaner version would certainly speed up local development, reduce the barrier to entry, and lead to smarter, faster testing.

Thoughts?

[smlx]

I've got a docker-compose.yaml for this on a local branch. it contains:

api-db
broker
api
ui
ssh
auth-server
keycloak
keycloak-db
local-api-data-watcher-pusher
logs-db
logs-db-ui

The reason that it contains logs-db and logs-db-ui is that I'm working on features relating to those components. So I don't know if you could exactly specify a single "minimal lagoon" profile because it would depend on what work you're doing.

Maybe we could just move the components that make up "core lagoon" (everything in the list above except logs-db*) to the top of docker-compose.yaml, and separate them from the the other components with a comment explaining that you can comment out the services below as desired?

Going forward it would be great if each non-core "subsystem" (logging, harbor, etc.) had its own test suite as per #1905

[dan2k3k4]

There's the extends option for docker-compose, Perhaps it could be structured like:

docker-compose.yaml // extend following 3 docker-compose files
├── docker-compose.main.yaml
│   └── services for production
├── docker-compose.application.yaml
│   └── services for application
└── docker-compose.core.yaml
    └── services for core

[xantrix]

Being able to run the leaner, meaner version would certainly speed up local development, reduce the barrier to entry, and lead to smarter, faster testing.

Thoughts?

Building Lagoon locally is definitely a pain point at the moment and could significantly slow-down new contributions (I faced this issue personally 😅 ).
As suggested by @dan2k3k4 a combination of extends and multiple compose files could help in making a leaner version.
Also, pass args options during the build of service images could help to reduce memory requirements.

e.g. for a local Keycloak 1024m could be enough
actual value: JAVA_OPTS="-server -Xms2048m -Xmx4096m
leaner: JAVA_OPTS="-server -Xms1024m -Xmx2048m