Custom DABs Template Documentation Additions

[bmswatli]

Enhancement Request:

When creating Custom Asset Bundle Templates, I would like to see additional documentation. I have to look at existing template code and try to copy that functionality.

For example, when defining template variables that the user must select, there isn't any documentation about how to use enums for the user to select from a list. That would be very helpful for users to know.

Also

I'd also like an easy way to export custom bundle templates for other members of my team. Any ideas here (besides using git) would be appreciated!

[shreyas-goenka]

@bmswatli Thanks for your feedback! We'll look into adding more documentation on authoring custom templates.

I'd also like an easy way to export custom bundle templates for other members of my team. Any ideas here (besides using git) would be appreciated!

The only other option is to use the --template-dir flag to point to a local path where your template definition lives. We recommend this flag normally for developing the template. Could you please elaborate on why hosting your template as a Git repository does not work for you?

[bmswatli]

@shreyas-goenka git works - it just takes more time. Imagine a feature where the custom template shows up as part of the list with the default templates. There would still likely be exporting and importing - but once it got imported, it would be a lot easier to select than referencing a directory.

[shreyas-goenka]

@bmswatli Could you please elaborate on what you mean by git taking more time? Is it that you have to setup a git repository in the first place to share the template or is it that you have to type the full URL when you do databricks bundle init ....

[bmswatli]

@shreyas-goenka

On the first one: If there was a clean way to export a custom template without git - that would be awesome. Then I could share that export (via email, teams, slack, etc.) and have other team members easily bring it in.

I actually didn't know we could reference the custom template by URL. I was using a local file path (another idea for documentation here).

A potential workflow:

1. Create a custom template bundle
2. Run something like databricks bundle export {bundle_path} to export the custom bundle.
3. Share exported file with team
4. Team members run something like databricks bundle import {file_path}.
5. Custom bundle template shows up databricks bundle init

Some challenges to this approach could be:

1. What happens when the custom bundle gets an update? Would a re-export overwrite the previous custom template? (probably)
2. How to export custom templates, as there are multiple files and directories involved.

[shreyas-goenka]

Thanks for outlining the suggesting in detail @bmswatli. I still don't follow why you would like to share the template as an exported file instead of creating a private repository with the template and giving the relevant users access. Could you please elaborate?

As for the Git URL functionality, while it is not documented in our public docs, it already exists in the CLI docs, ie when you run databricks bundle init -h. Was that in your experience not easy to find or understand?

[bmswatli]

It's for ease of use. A URL or local file directory is typically a long string.
Idea: The custom template could be mapped into the databricks bundle template selection.
Ex: custom-python-tmpl maps to https://github.com/repo-name.
The goal here is to make it easier to import and use other's generated custom templates. Knowing that we can reference a direct URL is probably sufficient, but figured I'd share some ideas anyway.

Public docs are the easiest way for me to find supported features. I don't typically use command line docs to search for features.

[shreyas-goenka]

Thanks for elaborating @bmswatli. The ROI for this feature does not sound convincing to me when stacked against other features we are working on, since we'll need to add additional state management to the CLI for what seems like a relatively minor UX improvement. I'll keep this issue open though incase this other users also echo your concerns.

Please correct me if you it's actually blocking/slowing down your workflows and is not an ease of use enhancement.

[bmswatli]

All good. Most of this issue was just about additional documentation for custom asset bundles templates - so as long as that improves, we're all happy.