Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Placeholders for catalog.yml not working properly when value contains underscore #4223

Open
Acomar0 opened this issue Oct 11, 2024 · 1 comment
Labels
Issue: Bug Report 🐞 Bug that needs to be fixed

Comments

@Acomar0
Copy link

Acomar0 commented Oct 11, 2024

Description

Placeholders for catalog.yml not working properly when value contains underscore

Context

I am encountering some issues regarding the use of placeholders for the data catalog and I was hoping you can shed some light on this .
I have the following pipeline:

load_date = settings.LOAD_DATE_COMPARISON.get("current")
previous_load_date = settings.LOAD_DATE_COMPARISON.get("previous")

def create_pipeline(**kwargs) -> Pipeline:


    format_data_quality = pipeline(
                [   node(
                        func= compare_id,
                        inputs=[f"maestro_indicadores_{load_date}",
                                f"maestro_indicadores_{previous_load_date}"],
                        outputs=f"compare_id_{load_date}_{previous_load_date}",
                        name="compare_id_node",
                        tags = "compare_id"
    ),]
    )
    return format_data_quality

With the corresponding catalog entry for the output:

compare_id_{load_date}_{previous_load_date}:
  type: json.JSONDataset
  filepath: reports/{load_date}/id_comparison/id_comparison_{load_date}_{previous_load_date}.json

The issue here is that whenever the value of load date is something like 2024_07_01, it will generate a path like:
reports/2024/id_comparison/id_comparison_ 2024_07_01_2024_05_01.json

Note that the first placeholder is not being substituted with the intended value, while the others are.
This will only happen when the value of load_date contains underscores, not happening with dots or hyphens.

  • Kedro version used : 0.19.8
  • Python version used : 3.11.4
  • Operating system and version: Win11 Pro 10.0.22.631
@Acomar0 Acomar0 changed the title <Title> Placeholders for catalog.yml not working properly when value contains underscore Oct 11, 2024
@noklam noklam added the Issue: Bug Report 🐞 Bug that needs to be fixed label Oct 16, 2024
@noklam
Copy link
Contributor

noklam commented Oct 16, 2024

Thanks for creating the issue @Acomar0, we will look into it soon.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Issue: Bug Report 🐞 Bug that needs to be fixed
Projects
Status: No status
Development

No branches or pull requests

2 participants