Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Linting on save throws error but still appears to work(?) #63

Open
dschneiderch opened this issue Sep 29, 2022 · 7 comments
Open

Linting on save throws error but still appears to work(?) #63

dschneiderch opened this issue Sep 29, 2022 · 7 comments

Comments

@dschneiderch
Copy link

I don't think its recognizing my .sqlfluff config file since its complaining about the dialect but then linting shows up anyway. the config is at /Users/dschneider/bombora/repos/topic-interest-score-af2-dags/bombora/bairflow/topic_interest_score/include/sql
my config:

dialect = bigquery
templater = jinja
sql_file_exts = .sql,.sql.j2,.dml,.ddl,.bqsql
# disable reorder select statements so aggregations are last
# disable complaint of USING() for joins
exclude_rules = L034, L032

[sqlfluff:templater]
unwrap_wrapped_queries = true

[sqlfluff:templater:jinja]
# Make dbt default macros like available like `ref`
apply_dbt_builtins = true
# Allow mocking dbt_utils and other packages through
# python files placed in this dir
library_path = ./sqlfluff_libs
load_macros_from_path = ../sql

[sqlfluff:rules]
max_line_length = 88

[sqlfluff:templater:jinja:context]
ds = "YYYY-MM-DD"
ds_nodash = "YYYYMMDD"

the error:

Reading from stdin, not file, input may be dirty/partial

--------------------Executing Command--------------------

/usr/local/bin/sqlfluff fix --force --exclude-rules L003,L054 --ignore parsing -

------------------------------------------------------------

Received close event, code 1 signal null
Raw stdout output:

------------------------------------------------------------



------------------------------------------------------------

Raw stderr output:

------------------------------------------------------------

Traceback (most recent call last):
  File "/usr/local/bin/sqlfluff", line 33, in <module>

    sys.exit(load_entry_point('sqlfluff==1.3.1', 'console_scripts', 'sqlfluff')())

  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/click/core.py", line 1130, in __call__

    return self.main(*args, **kwargs)

  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/click/core.py", line 1055, in main

    rv = self.invoke(ctx)

  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/click/core.py", line 1657, in invoke

    return _process_result(sub_ctx.command.invoke(sub_ctx))

  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/click/core.py", line 1404, in invoke

    return ctx.invoke(self.callback, **ctx.params)

  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/click/core.py", line 760, in invoke

    return __callback(*args, **kwargs)

  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/sqlfluff/cli/commands.py", line 781, in fix

    result = lnt.lint_string_wrapped(stdin, fname="stdin", fix=True)

  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/sqlfluff/core/linter/linter.py", line 1075, in lint_string_wrapped

    linted_path.add(self.lint_string(string, fname=fname, fix=fix))

  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/sqlfluff/core/linter/linter.py", line 920, in lint_string

    parsed = self.parse_string(

  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/sqlfluff/core/linter/linter.py", line 852, in parse_string

    rendered = self.render_string(in_str, fname, config, encoding)
  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/sqlfluff/core/linter/linter.py", line 788, in render_string

    config.verify_dialect_specified()

  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/sqlfluff/core/config.py", line 530, in verify_dialect_specified

    
raise SQLFluffUserError(
sqlfluff.core.errors.SQLFluffUserError: No dialect was specified. You must configure a dialect or specify one on the command line using --dialect after the command. Available dialects:
ansi, athena, bigquery, clickhouse, databricks, db2, exasol, hive, mysql, oracle, postgres, redshift, snowflake, soql, sparksql, sqlite, teradata, tsql


------------------------------------------------------------

Reading from file, not stdin

--------------------Executing Command--------------------

/usr/local/bin/sqlfluff lint --format json --exclude-rules L003,L054 --ignore parsing bombora/bairflow/topic_interest_score/include/sql/import_event_profile_tables.sql

------------------------------------------------------------

Received close event, code 1 signal null
Raw stdout output:

------------------------------------------------------------



------------------------------------------------------------

Raw stderr output:

------------------------------------------------------------

Traceback (most recent call last):
  File "/usr/local/bin/sqlfluff", line 33, in <module>

    sys.exit(load_entry_point('sqlfluff==1.3.1', 'console_scripts', 'sqlfluff')())

  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/click/core.py", line 1130, in __call__

    return self.main(*args, **kwargs)

  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/click/core.py", line 1055, in main

    
rv = self.invoke(ctx)

  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/click/core.py", line 1657, in invoke

    return _process_result(sub_ctx.command.invoke(sub_ctx))

  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/click/core.py", line 1404, in invoke

    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/click/core.py", line 760, in invoke

    return __callback(*args, **kwargs)
  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/sqlfluff/cli/commands.py", line 579, in lint

    
result = lnt.lint_paths(
  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/sqlfluff/core/linter/linter.py", line 1173, in lint_paths

    self.lint_path(

  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/sqlfluff/core/linter/linter.py", line 1125, in lint_path

    for i, linted_file in enumerate(runner.run(fnames, fix), start=1):

  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/sqlfluff/core/linter/runner.py", line 106, in run

    for fname, partial in self.iter_partials(fnames, fix=fix):

  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/sqlfluff/core/linter/runner.py", line 59, in iter_partials

    for fname, rendered in self.iter_rendered(fnames):

  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/sqlfluff/core/linter/runner.py", line 46, in iter_rendered

    yield fname, self.linter.render_file(fname, self.config)

  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/sqlfluff/core/linter/linter.py", line 830, in render_file

    return self.render_string(raw_file, fname, config, encoding)

  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/sqlfluff/core/linter/linter.py", line 801, in render_string

    
templated_file, templater_violations = self.templater.process(
  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/sqlfluff/core/templaters/base.py", line 53, in _wrapped

    return func(self, in_str=in_str, fname=fname, config=config, **kwargs)
  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/sqlfluff/core/templaters/jinja.py", line 376, in process

    env, live_context, make_template = self.template_builder(


  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/sqlfluff/core/templaters/jinja.py", line 331, in template_builder

    live_context = self.get_context(fname=fname, config=config, env=env)

  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/sqlfluff/core/templaters/jinja.py", line 311, in get_context

    self._extract_macros_from_path(
  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/sqlfluff/core/templaters/jinja.py", line 106, in _extract_macros_from_path

    cls._extract_macros_from_path(

  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/sqlfluff/core/templaters/jinja.py", line 91, in _extract_macros_from_path

    cls._extract_macros_from_template(template, env=env, ctx=ctx)
  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/sqlfluff/core/templaters/jinja.py", line 62, in _extract_macros_from_template

    for k in macro_template.module.__dict__:

  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/jinja2/environment.py", line 1478, in module

    return self._get_default_module()

  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/jinja2/environment.py", line 1446, in _get_default_module

    self._module = self.make_module()

  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/jinja2/environment.py", line 1405, in make_module

    return TemplateModule(self, ctx)

  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/jinja2/environment.py", line 1535, in __init__

    body_stream = list(template.root_render_func(context))  # type: ignore

  File "<template>", line 12, in root

  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/jinja2/environment.py", line 1010, in get_template

    return self._load_template(name, globals)

  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/jinja2/environment.py", line 969, in _load_template

    template = self.loader.load(self, name, self.make_globals(globals))


  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/jinja2/loaders.py", line 126, in load

    source, filename, uptodate = self.get_source(environment, name)

  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/jinja2/loaders.py", line 218, in get_source

    raise TemplateNotFound(template)

jinja2.exceptions.TemplateNotFound
: 
sql/weekly__calc_tis__1_filter_features_for_bots_and_history.sql


------------------------------------------------------------```
@RobertOstermann
Copy link
Contributor

@dschneiderch Do you have any extension specific settings? I'm not quite sure why it wouldn't be picking up your config file, but sometimes the easiest solution is just to manually set the config setting specifically for your workspace.

@dschneiderch
Copy link
Author

These are my extension specific settings

    "sqlfluff.executablePath": "/usr/local/bin/sqlfluff",
    "sqlfluff.ignoreParsing": true,
    "sqlfluff.linter.run": "onSave",
    "sqlfluff.excludeRules": [
        "L034",
        "L032"
    ],
    "[sql]": {
        "editor.defaultFormatter": "dorzey.vscode-sqlfluff"
    },

i added these and now it gets the config file at least. can there be multiple config locations? i'm wondering how to handle nested sql files

    "sqlfluff.config": "${fileDirname}/.sqlfluff",
    "sqlfluff.ignoreLocalConfig": false,

I still see this error in the log but it happens with i used the integrated terminal too. i guess this might be a sqlfluff issue?:

Format triggered for /Users/dschneider/bombora/repos/topic-interest-score-af2-dags/bombora/bairflow/topic_interest_score/include/sql/import_event_profile_tables.sql
Reading from stdin, not file, input may be dirty/partial

--------------------Executing Command--------------------

/usr/local/bin/sqlfluff fix --force --config /Users/dschneider/bombora/repos/topic-interest-score-af2-dags/bombora/bairflow/topic_interest_score/include/sql/.sqlfluff --exclude-rules L034,L032 --ignore parsing -

------------------------------------------------------------

Received close event, code 1 signal null
Raw stdout output:

------------------------------------------------------------



------------------------------------------------------------

Raw stderr output:

------------------------------------------------------------

Traceback (most recent call last):

  File "/usr/local/bin/sqlfluff", line 33, in <module>

    
sys.exit(load_entry_point('sqlfluff==1.3.1', 'console_scripts', 'sqlfluff')())
  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/click/core.py", line 1130, in __call__

    return self.main(*args, **kwargs)

  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/click/core.py", line 1055, in main

    rv = self.invoke(ctx)

  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/click/core.py", line 1657, in invoke

    return _process_result(sub_ctx.command.invoke(sub_ctx))

  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/click/core.py", line 1404, in invoke

    
return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/click/core.py", line 760, in invoke

    return __callback(*args, **kwargs)

  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/sqlfluff/cli/commands.py", line 781, in fix

    result = lnt.lint_string_wrapped(stdin, fname="stdin", fix=True)
  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/sqlfluff/core/linter/linter.py", line 1075, in lint_string_wrapped

    linted_path.add(self.lint_string(string, fname=fname, fix=fix))

  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/sqlfluff/core/linter/linter.py", line 920, in lint_string

    parsed = self.parse_string(
  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/sqlfluff/core/linter/linter.py", line 852, in parse_string

    rendered = self.render_string(in_str, fname, config, encoding)
  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/sqlfluff/core/linter/linter.py", line 801, in render_string

    templated_file, templater_violations = self.templater.process(
  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/sqlfluff/core/templaters/base.py", line 53, in _wrapped

    return func(self, in_str=in_str, fname=fname, config=config, **kwargs)

  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/sqlfluff/core/templaters/jinja.py", line 376, in process

    env, live_context, make_template = self.template_builder(

  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/sqlfluff/core/templaters/jinja.py", line 331, in template_builder

    live_context = self.get_context(fname=fname, config=config, env=env)

  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/sqlfluff/core/templaters/jinja.py", line 311, in get_context

    self._extract_macros_from_path(
  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/sqlfluff/core/templaters/jinja.py", line 106, in _extract_macros_from_path

    cls._extract_macros_from_path(
  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/sqlfluff/core/templaters/jinja.py", line 91, in _extract_macros_from_path

    cls._extract_macros_from_template(template, env=env, ctx=ctx)
  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/sqlfluff/core/templaters/jinja.py", line 62, in _extract_macros_from_template

    for k in macro_template.module.__dict__:

  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/jinja2/environment.py", line 1478, in module

    return self._get_default_module()
  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/jinja2/environment.py", line 1446, in _get_default_module
    self._module = self.make_module()
  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/jinja2/environment.py", line 1405, in make_module
    return TemplateModule(self, ctx)
  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/jinja2/environment.py", line 1535, in __init__
    body_stream = list(template.root_render_func(context))  # type: ignore
  File "<template>", line 12, in root
  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/jinja2/environment.py", line 1010, in get_template
    return self._load_template(name, globals)
  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/jinja2/environment.py", line 969, in _load_template
    template = self.loader.load(self, name, self.make_globals(globals))
  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/jinja2/loaders.py", line 126, in load

    source, filename, uptodate = self.get_source(environment, name)
  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/jinja2/loaders.py", line 218, in get_source

    raise TemplateNotFound(template)

jinja2.exceptions.TemplateNotFound: sql/weekly__calc_tis__1_filter_features_for_bots_and_history.sql


------------------------------------------------------------

Reading from file, not stdin

--------------------Executing Command--------------------

/usr/local/bin/sqlfluff lint --format json --config /Users/dschneider/bombora/repos/topic-interest-score-af2-dags/bombora/bairflow/topic_interest_score/include/sql/.sqlfluff --exclude-rules L034,L032 --ignore parsing bombora/bairflow/topic_interest_score/include/sql/import_event_profile_tables.sql

------------------------------------------------------------

Received close event, code 1 signal null
Raw stdout output:

------------------------------------------------------------



------------------------------------------------------------

Raw stderr output:

------------------------------------------------------------

Traceback (most recent call last):
  File "/usr/local/bin/sqlfluff", line 33, in <module>

    sys.exit(load_entry_point('sqlfluff==1.3.1', 'console_scripts', 'sqlfluff')())
  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/click/core.py", line 1130, in __call__

    return self.main(*args, **kwargs)
  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/click/core.py", line 1055, in main

    rv = self.invoke(ctx)
  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/click/core.py", line 1657, in invoke

    return _process_result(sub_ctx.command.invoke(sub_ctx))

  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/click/core.py", line 1404, in invoke

    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/click/core.py", line 760, in invoke

    return __callback(*args, **kwargs)

  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/sqlfluff/cli/commands.py", line 579, in lint

    
result = lnt.lint_paths(
  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/sqlfluff/core/linter/linter.py", line 1173, in lint_paths

    self.lint_path(
  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/sqlfluff/core/linter/linter.py", line 1125, in lint_path

    for i, linted_file in enumerate(runner.run(fnames, fix), start=1):
  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/sqlfluff/core/linter/runner.py", line 106, in run

    for fname, partial in self.iter_partials(fnames, fix=fix):
  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/sqlfluff/core/linter/runner.py", line 59, in iter_partials
    for fname, rendered in self.iter_rendered(fnames):
  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/sqlfluff/core/linter/runner.py", line 46, in iter_rendered

    yield fname, self.linter.render_file(fname, self.config)

  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/sqlfluff/core/linter/linter.py", line 830, in render_file

    return self.render_string(raw_file, fname, config, encoding)

  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/sqlfluff/core/linter/linter.py", line 801, in render_string

    templated_file, templater_violations = self.templater.process(

  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/sqlfluff/core/templaters/base.py", line 53, in _wrapped

    return func(self, in_str=in_str, fname=fname, config=config, **kwargs)

  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/sqlfluff/core/templaters/jinja.py", line 376, in process

    env, live_context, make_template = self.template_builder(

  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/sqlfluff/core/templaters/jinja.py", line 331, in template_builder

    live_context = self.get_context(fname=fname, config=config, env=env)
  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/sqlfluff/core/templaters/jinja.py", line 311, in get_context

    self._extract_macros_from_path(
  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/sqlfluff/core/templaters/jinja.py", line 106, in _extract_macros_from_path

    cls._extract_macros_from_path(

  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/sqlfluff/core/templaters/jinja.py", line 91, in _extract_macros_from_path

    cls._extract_macros_from_template(template, env=env, ctx=ctx)
  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/sqlfluff/core/templaters/jinja.py", line 62, in _extract_macros_from_template

    
for k in macro_template.module.__dict__:
  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/jinja2/environment.py", line 1478, in module

    return self._get_default_module()

  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/jinja2/environment.py", line 1446, in _get_default_module

    self._module = self.make_module()

  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/jinja2/environment.py", line 1405, in make_module

    return TemplateModule(self, ctx)
  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/jinja2/environment.py", line 1535, in __init__

    body_stream = list(template.root_render_func(context))  # type: ignore

  File "<template>", line 12, in root

  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/jinja2/environment.py", line 1010, in get_template

    return self._load_template(name, globals)

  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/jinja2/environment.py", line 969, in _load_template

    template = self.loader.load(self, name, self.make_globals(globals))
  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/jinja2/loaders.py", line 126, in load

    source, filename, uptodate = self.get_source(environment, name)
  File "/usr/local/Cellar/sqlfluff/1.3.1/libexec/lib/python3.10/site-packages/jinja2/loaders.py", line 218, in get_source

    raise TemplateNotFound(template)
jinja2.exceptions
.TemplateNotFound: sql/weekly__calc_tis__1_filter_features_for_bots_and_history.sql



------------------------------------------------------------```

@dschneiderch
Copy link
Author

Looking at that last error, that is coming from a different file that i'm not trying to lint. And that file used to work fine once I added
load_macros_from_path = ../sql to my .sqlfluff

@RobertOstermann
Copy link
Contributor

Using the sqlfluff.config does not allow for multiple configuration files to be used at once. That simply allows you to choose a single configuration file. If you only need 1 config it might be better to use the workspace folder and point the path directly to that .sqlfluff path instead of using the file directory which changes when the files are nested.

"sqlfluff.config": "${workspaceFolder}/path/to/sqlfluff"

But if you want to use multiple configuration files you could try something like this, which is basically telling the extension to run the commands from the file directory. Basically what would happen if you cd into that directory and run the sqlfluff command. If you try this you should comment out or remove the config setting.

"sqlfluff.workingDirectory": "${fileDirname}"

As for that last error, it looks like a sqlfluff or setup issue so not something I could fix with this extension.

@dschneiderch
Copy link
Author

when i added the workingDirectory entry and removed the config entry it
tells me it can't find the sqlfluff executable (even though it's explicitly set)

Format triggered for /Users/dschneider/bombora/repos/topic-interest-score-
af2-dags/bombora/bairflow/topic_interest_score/include/sql/
import_event_profile_tables.sql
Reading from stdin, not file, input may be dirty/partial

--------------------Executing Command--------------------

/usr/local/bin/sqlfluff fix --force --exclude-rules L034,L032 -

------------------------------------------------------------

Child process threw error
Error: spawn /usr/local/bin/sqlfluff ENOENT
Reading from file, not stdin

--------------------Executing Command--------------------

/usr/local/bin/sqlfluff lint --format json --exclude-rules L034,L032
../Users/dschneider/bombora/repos/topic-interest-score-af2
-dags/bombora/bairflow/topic_interest_score/include/sql/
import_event_profile_tables.sql

------------------------------------------------------------

Child process threw error
Error: spawn /usr/local/bin/sqlfluff ENOENT
Linting command failed to execute

@RobertOstermann
Copy link
Contributor

Hm ok, probably best not to use that setting for now then. Are you able to get everything working just using the config setting? Or do you need to use multiple .sqlfluff configuration files at once?

@dschneiderch
Copy link
Author

In that repo I was testing on its just a flat hierarchy (for now). In newer repos we've organized into a nested structure though. And in one folder export_surge_for_shipper the dialect would actually be mysql instead of bigquery so I was hoping I could override the config for that one directory with a local config file.

~/b/r/s/b/b/s/i/sql ❯❯❯ tree -L 2 -a                
.
├── .sqlfluff
├── __init__.py
├── __pycache__
│   └── __init__.cpython-310.pyc
├── decorate
│   ├── __init__.py
│   ├── gen_decorated_tables.sql
│   ├── gen_decorated_tables__cte__merge_dt_dtc_dtcs_dimensions.sql
│   ├── gen_decorated_tables__cte__merge_dt_dtc_dtm_dimensions.sql
│   ├── gen_decorated_tables__cte__merge_full_dt_dtc_dtm_dimensions.sql
│   ├── gen_flat_company_data.sql
│   ├── map_top_n_countries_per_domain.sql
│   └── map_top_n_zipcodes_per_metro.sql
├── export_surge_for_freemium
│   ├── __init__.py
│   ├── freemium.sql
│   ├── freemium_arctic.sql
│   ├── freemium_waist_high.sql
│   └── freemium_waist_high_count.sql
├── export_surge_for_sales
│   ├── __init__.py
│   └── surge_for_sales.sql
├── export_surge_for_shipper
│   ├── .sqlfluff
│   ├── __init__.py
│   ├── surge_export_config__shipper_sb_s3_destination.sql
│   ├── surge_export_config__shipper_sb_sftp_destination.sql
│   └── surge_export_config__surge_export_output.sql
└── sqlfluff_libs
    ├── DT.py
    ├── __init__.py
    ├── __pycache__
    ├── params.py
    └── posixpath.py

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@dschneiderch @RobertOstermann