Ability to override resolv.conf DNS servers

[andrew-edgar]

Is your feature request related to a problem? Please describe.
When we are working on the hybrid migration we have placed a second DNS server in front of the bosh-dns so that we can manage dual DNS lookups.
Quarks always sets the spec.dnsConfig like this ...

{
  "nameservers": [
    "172.21.190.24"
  ],
  "options": [
    {
      "name": "ndots",
      "value": "5"
    }
  ],
  "searches": [
    "kubecf.svc.cluster.local",
    "svc.cluster.local",
    "cluster.local"
  ]
}

Where 172.21.190.24 is the ip of the bosh-dns server. we would like to be able to override that and set it to a different static IP.

Describe the solution you'd like
Have the ability to set a parameter in the manifest that quarks will accept to use as an override of the server values. if not set it should default to the standard configuration.

Describe alternatives you've considered
We are currently forced to manually update the /etc/resolv.conf and we also need customer configuration on the dns server so we cannot add new config into the current bosh-dns to support the use case.

Additional context
See initial discussion in slack here https://cloudfoundry.slack.com/archives/C1BQKKNP4/p1602782923025400

[cf-gitbot]

We have created an issue in Pivotal Tracker to manage this:

https://www.pivotaltracker.com/story/show/175292035

The labels on this github issue will be updated when the story is started.

[manno]

@andrew-edgar we've been discussing this. If we allow to override the server, clients won't be able to resolve domains that are configured in bosh-dns anymore, won't that break things?
Unless the server you put in front of bosh-dns is doing conditional forwarding to the bosh-dns?

[manno]

#1229

[andrew-edgar]

@manno yes that's exactly right you need to have the dns in front of the bosh-dns. which is what we are doing but a warning about this override is clear that yes you must also somehow front the existing dns.

[univ0298]

Confirming it works. For example:

  - type: replace
    path: /instance_groups/name=diego-api/env?/bosh/agent/settings/dns
    value: 172.21.1.1