I apologize in advance for the length - but this is interesting, especially for architects and traefik wizards.
The Challange:
The bottom line is, I'm hoping that there's a setup in which we can leverage traefik
s dynamic config capability to:
detect automatically when a container is stopped, and know to fallback to external services on developer machine / external host.
Alternatively - when a local-db container is started, know to prefer it over db service from external source.
The full picture
I need to optimize a workflow for a couple of teams.
The constraints:
- developers work on a multi-layer complex micro-service mesh
- number of services in the mesh is ~50, some of which are memory-intensive
- local demos & acceptance tests requires to show the whole mesh works from end-user point of view
- developers are now asked to run all the mesh locally, except for some shared databases with live data replicas - which may occasionally also be run locally instead.
This requires to run on developer machine the absolute minimum required.
Current solution relays on docker-compose and traefik@1.x
, and I'm asked to preserve this, however, I believe upgrading to traefik@2.x
would be accepted.
Pursued Use Cases
I'm actually pursuing with you here two use-cases, which I believe have the same solution - at least in theory...
Use-Case one - local dev of multiple micro-services
Dev-Mode = running a process in debug mode, with IDE, file-watch live reloads, etc.
AS A - developer of services in mid layers of the service mesh
I WANT - to be able to direct traffic to services under development to
native processes on my machine, and keep the rest in the compose
network
SO THAT - I can run in Dev-Mode only services under development,
while all the rest in the mesh run using production-level docker
images in docker-compose.
Use-Case two - local dev and external database
AS A - developer of data brokers in the mesh
I WANT - to control if containers get to the shared db or to local
db container using traefik
SO THAT - I can keep applications in the containers completely agnostic to
the topography in which they run
So what's missing?
As a start - Since Traefik is the service-buss through which all inter-container communications pass - I'm looking for a way to configure traefik
to consume services from outside the docker-compose network.
The first use-case means to use traefik
as the service buss - the sole entity that knows where each service runs, which could be a container, or a service external to the docker-compose network.
The second use-case means to setup traefik
to direct traffic to a process runs outside the docker-compose network, but specifically on the developer's machine, which in turn, consumes services from the compose network (for this, the dev compose, unlike production compose - exposes an entrypoint for each node in the mesh, including for services that are consumed only internally on production).
Two for the price of one
I know there are two questions here, but I could not think of another way to ask each question separately and get answers relevant to the full picture...
I'm hoping that there's a setup in which we can leverage traefik
s dynamic config capability to:
detect automatically when a container is stopped, and know to fallback to external services on developer machine or any other host for the matter. Alternatively - when a db container is started, knows to prefer it over external db service.
We can ask developers to add records to their hosts
file, but a DNS-based solution would be much preferred, especially if we can use the DNS built-in to docker :)
Help ?
Any help or point to the right direction would be very appreciated!