Update dependency sqlparse to v0.5.4 [SECURITY]

[renovate]

This PR contains the following updates:

Package	Change	Age	Confidence
sqlparse (changelog)	==0.4.4 → ==0.5.4

---

sqlparse parsing heavily nested list leads to Denial of Service

CVE-2024-4340 / GHSA-2m57-hf25-phgg

More information

Details

Summary

Passing a heavily nested list to sqlparse.parse() leads to a Denial of Service due to RecursionError.

Details + PoC

Running the following code will raise Maximum recursion limit exceeded exception:

import sqlparse
sqlparse.parse('[' * 10000 + ']' * 10000)

We expect a traceback of RecursionError:

Traceback (most recent call last):
  File "trigger_sqlparse_nested_list.py", line 3, in <module>
    sqlparse.parse('[' * 10000 + ']' * 10000)
  File "/home/uriya/.local/lib/python3.10/site-packages/sqlparse/__init__.py", line 30, in parse
    return tuple(parsestream(sql, encoding))
  File "/home/uriya/.local/lib/python3.10/site-packages/sqlparse/engine/filter_stack.py", line 36, in run
    stmt = grouping.group(stmt)
  File "/home/uriya/.local/lib/python3.10/site-packages/sqlparse/engine/grouping.py", line 428, in group
    func(stmt)
  File "/home/uriya/.local/lib/python3.10/site-packages/sqlparse/engine/grouping.py", line 53, in group_brackets
    _group_matching(tlist, sql.SquareBrackets)
  File "/home/uriya/.local/lib/python3.10/site-packages/sqlparse/engine/grouping.py", line 48, in _group_matching
    tlist.group_tokens(cls, open_idx, close_idx)
  File "/home/uriya/.local/lib/python3.10/site-packages/sqlparse/sql.py", line 328, in group_tokens
    grp = grp_cls(subtokens)
  File "/home/uriya/.local/lib/python3.10/site-packages/sqlparse/sql.py", line 161, in __init__
    super().__init__(None, str(self))
  File "/home/uriya/.local/lib/python3.10/site-packages/sqlparse/sql.py", line 165, in __str__
    return ''.join(token.value for token in self.flatten())
  File "/home/uriya/.local/lib/python3.10/site-packages/sqlparse/sql.py", line 165, in <genexpr>
    return ''.join(token.value for token in self.flatten())
  File "/home/uriya/.local/lib/python3.10/site-packages/sqlparse/sql.py", line 214, in flatten
    yield from token.flatten()
  File "/home/uriya/.local/lib/python3.10/site-packages/sqlparse/sql.py", line 214, in flatten
    yield from token.flatten()
  File "/home/uriya/.local/lib/python3.10/site-packages/sqlparse/sql.py", line 214, in flatten
    yield from token.flatten()
  [Previous line repeated 983 more times]
RecursionError: maximum recursion depth exceeded

Fix suggestion

The flatten() function of TokenList class should limit the recursion to a maximal depth:

from sqlparse.exceptions import SQLParseError

MAX_DEPTH = 100

    def flatten(self, depth=1):
        """Generator yielding ungrouped tokens.

        This method is recursively called for all child tokens.
        """
    if depth >= MAX_DEPTH:
        raise SQLParseError('Maximal depth reached')
        for token in self.tokens:
            if token.is_group:
                yield from token.flatten(depth + 1)
            else:
                yield token

Impact

Denial of Service (the impact depends on the use).
Anyone parsing a user input with sqlparse.parse() is affected.

Severity

• CVSS Score: 7.5 / 10 (High)
• Vector String: CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H

References

• https://github.com/andialbrecht/sqlparse/security/advisories/GHSA-2m57-hf25-phgg
• https://github.com/andialbrecht/sqlparse/commit/b4a39d9850969b4e1d6940d32094ee0b42a2cf03
• https://nvd.nist.gov/vuln/detail/CVE-2024-4340
• https://research.jfrog.com/vulnerabilities/sqlparse-stack-exhaustion-dos-jfsa-2024-001031292
• https://github.com/advisories/GHSA-2m57-hf25-phgg

This data is provided by the GitHub Advisory Database (CC-BY 4.0).

---

sqlparse: formatting list of tuples leads to denial of service

GHSA-27jp-wm6q-gp25

More information

Details

Summary

The below gist hangs while attempting to format a long list of tuples.

This was found while drafting a regression test for Dja
ngo 5.2's composite primary key feature, which allows querying composite fields with tuples.

Severity

• CVSS Score: 6.9 / 10 (Medium)
• Vector String: CVSS:4.0/AV:N/AC:L/AT:N/PR:N/UI:N/VC:N/VI:L/VA:N/SC:N/SI:N/SA:N

References

• https://github.com/andialbrecht/sqlparse/security/advisories/GHSA-27jp-wm6q-gp25
• https://github.com/andialbrecht/sqlparse/commit/40ed3aa958657fa4a82055927fa9de70ab903360
• https://github.com/andialbrecht/sqlparse/releases/tag/0.5.4
• https://github.com/advisories/GHSA-27jp-wm6q-gp25

This data is provided by the GitHub Advisory Database (CC-BY 4.0).

---

Release Notes

andialbrecht/sqlparse (sqlparse)

v0.5.4

Compare Source

Enhancements

• Add support for Python 3.14.
• Add type annotations to top-level API functions and include py.typed marker
  for PEP 561 compliance, enabling type checking with mypy and other tools
  (issue756).
• Add pre-commit hook support. sqlparse can now be used as a pre-commit hook
  to automatically format SQL files. The CLI now supports multiple files and
  an --in-place flag for in-place editing (issue537).
• Add ATTACH and DETACH to PostgreSQL keywords (pr808).
• Add INTERSECT to close keywords in WHERE clause (pr820).
• Support REGEXP BINARY comparison operator (pr817).

Bug Fixes

• Add additional protection against denial of service attacks when parsing
  very large lists of tuples. This enhances the existing recursion protections
  with configurable limits for token processing to prevent DoS through
  algorithmic complexity attacks. The new limits (MAX_GROUPING_DEPTH=100,
  MAX_GROUPING_TOKENS=10000) can be adjusted or disabled (by setting to None)
  if needed for legitimate large SQL statements.
• Remove shebang from cli.py and remove executable flag (pr818).
• Fix strip_comments not removing all comments when input contains only
  comments (issue801, pr803 by stropysh).
• Fix splitting statements with IF EXISTS/IF NOT EXISTS inside BEGIN...END
  blocks (issue812).
• Fix splitting on semicolons inside BEGIN...END blocks (issue809).

v0.5.3

Compare Source

Bug Fixes

• This version introduces a more generalized handling of potential denial of
  service attack (DOS) due to recursion errors for deeply nested statements.
  Brought up and fixed by @​living180. Thanks a lot!

v0.5.2

Compare Source

Bug Fixes

• EXTENSION is now recognized as a keyword (issue785).
• SQL hints are not removed when removing comments (issue262, by skryzh).

v0.5.1

Compare Source

Enhancements

• New "compact" option for formatter. If set, the formatter tries to produce
  a more compact output by avoiding some line breaks (issue783).

Bug Fixes

• The strip comments filter was a bit greedy and removed too much
  whitespace (issue772).
  Note: In some cases you might want to add strip_whitespace=True where you
  previously used just strip_comments=True. strip_comments did some of the
  work that strip_whitespace should do.
• Fix error when splitting statements that contain multiple CASE clauses
  within a BEGIN block (issue784).
• Fix whitespace removal with nested expressions (issue782).
• Fix parsing and formatting of ORDER clauses containing NULLS FIRST or
  NULLS LAST (issue532).

v0.5.0

Compare Source

Notable Changes

• Drop support for Python 3.5, 3.6, and 3.7.
• Python 3.12 is now supported (pr725, by hugovk).
• IMPORTANT: Fixes a potential denial of service attack (DOS) due to recursion
  error for deeply nested statements. Instead of recursion error a generic
  SQLParseError is raised. See the security advisory for details:
  GHSA-2m57-hf25-phgg
  The vulnerability was discovered by @​uriyay-jfrog. Thanks for reporting!

Enhancements

• Splitting statements now allows to remove the semicolon at the end.
  Some database backends love statements without semicolon (issue742).
• Support TypedLiterals in get_parameters (pr749, by Khrol).
• Improve splitting of Transact SQL when using GO keyword (issue762).
• Support for some JSON operators (issue682).
• Improve formatting of statements containing JSON operators (issue542).
• Support for BigQuery and Snowflake keywords (pr699, by griffatrasgo).
• Support parsing of OVER clause (issue701, pr768 by r33s3n6).

Bug Fixes

• Ignore dunder attributes when creating Tokens (issue672).
• Allow operators to precede dollar-quoted strings (issue763).
• Fix parsing of nested order clauses (issue745, pr746 by john-bodley).
• Thread-safe initialization of Lexer class (issue730).
• Classify TRUNCATE as DDL and GRANT/REVOKE as DCL keywords (based on pr719
  by josuc1, thanks for bringing this up!).
• Fix parsing of PRIMARY KEY (issue740).

Other

• Optimize performance of matching function (pr799, by admachainz).

---

Configuration

📅 Schedule: (UTC)

• Branch creation
  • At any time (no schedule defined)
• Automerge
  • At any time (no schedule defined)

🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about this update again.

---

• If you want to rebase/retry this PR, check this box

---

This PR was generated by Mend Renovate. View the repository job log.

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
No data about Duplication

See analysis details on SonarCloud

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud