Skip to content

[ty] Infer constant membership in inline list and set literals - #28676

Merged
carljm merged 6 commits into
mainfrom
cjm/inline-literal-membership
Sep 21, 2026
Merged

carljm merged 6 commits into
mainfrom
cjm/inline-literal-membership

Conversation

@carljm

@carljm carljm commented Sep 17, 2026 •

Copy link
Copy Markdown
Contributor

Inline list and set membership tests now preserve the values of their individual elements, allowing ty to determine when a condition is always true or false. With the target platform set to Linux, both sys.platform in ["linux", "windows"] and sys.platform in {"linux", "windows"} select only the matching branch, so an assignment there is inferred as Literal[1].

Reuse the precise element extraction used for iteration and membership narrowing without changing the inferred container type. Lists and sets use the existing tuple membership equality evaluator, including its identity-or-equality behavior. For sets, assume equality is an equivalence relation and equal objects have equal hashes. Checking reflexive equality across union alternatives also improves existing tuple membership inference.

Context is that in python/typing#2173 people are wanting to specify that sys.platform in ["linux", "windows"] should work. This makes it work in ty so that we can be fine with any outcome of that specification change.

Test plan

Mdtests cover platform-dependent branch reachability; positive and negative membership in inline lists and sets; inferred literal values and unions; fixed-length unpacking; stored mutable containers; custom equality and identity; comparison chains; and preservation of enum-set narrowing.

Ecosystem report

The comparison for head 90111f10da9359722abb903351b89aaae0f9bcba adds four redundant-condition-strict warnings. The paasta and SciPy warnings correctly identify disjoint membership checks. The dd-trace-py warning exposes existing limitations in modeling dynamic setattr; the Scrapy warning exposes existing attribute narrowing across method calls and await. Equivalent equality checks already exhibit both limitations on the base revision. These existing limitations are accepted for this change. See the full ecosystem report for minimized examples and exact comparison details.

@carljm carljm added the ty Multi-file analysis & type inference label Sep 17, 2026
@astral-sh-bot

astral-sh-bot Bot commented Sep 17, 2026 •

Copy link
Copy Markdown

Typing conformance results

No changes detected ✅

Current numbers
The percentage of diagnostics emitted that were expected errors held steady at 98.15%. The percentage of expected errors that received a diagnostic held steady at 98.07%. The number of fully passing files held steady at 133/146.

@astral-sh-bot

astral-sh-bot Bot commented Sep 17, 2026 •

Copy link
Copy Markdown

Memory usage report

Summary

Project Old New Diff Outcome
sphinx 165.86MB 165.94MB +0.04% (75.07kB) ⏫
prefect 460.78MB 460.81MB +0.01% (27.93kB) ⏫
flake8 38.12MB 38.12MB +0.01% (3.41kB) ⏫
trio 91.29MB 91.29MB -0.00% (920.00B) ⬇️

Significant changes

Click to expand detailed breakdown

sphinx

Name Old New Diff Outcome
BoundMethodType 1.19MB 1.21MB +1.35% (16.52kB) ⏫
infer_expression_types_impl 11.42MB 11.43MB +0.14% (15.89kB) ⏫
MemberLookupKey 3.01MB 3.03MB +0.51% (15.74kB) ⏫
Type<'db>::class_member_with_policy_inner_ 3.28MB 3.30MB +0.46% (15.55kB) ⏫
member_lookup_with_policy_inner 3.65MB 3.66MB +0.36% (13.42kB) ⏫
infer_definition_types 11.51MB 11.51MB +0.05% (6.03kB) ⏫
FunctionType 1.83MB 1.83MB -0.18% (3.36kB) ⬇️
all_narrowing_constraints_for_expression 1.64MB 1.64MB +0.15% (2.44kB) ⏫
GenericContext 895.56kB 893.33kB -0.25% (2.23kB) ⬇️
Type<'db>::apply_specialization_inner_::interned_arguments 1.16MB 1.16MB -0.13% (1.48kB) ⬇️
TypeVarSetInner 405.09kB 403.79kB -0.32% (1.30kB) ⬇️
analyze_condition 417.45kB 418.75kB +0.31% (1.30kB) ⏫
infer_statement_types_impl 513.29kB 514.41kB +0.22% (1.12kB) ⏫
Type<'db>::apply_specialization_inner_ 774.34kB 773.38kB -0.12% (976.00B) ⬇️
TypeVarInference 456.08kB 455.14kB -0.21% (960.00B) ⬇️
... 21 more

prefect

Name Old New Diff Outcome
BoundMethodType 3.03MB 3.04MB +0.29% (8.97kB) ⏫
MemberLookupKey 7.10MB 7.11MB +0.11% (8.33kB) ⏫
Type<'db>::class_member_with_policy_inner_ 9.93MB 9.94MB +0.08% (8.32kB) ⏫
member_lookup_with_policy_inner 9.16MB 9.16MB +0.08% (7.45kB) ⏫
infer_expression_types_impl 23.81MB 23.81MB +0.01% (3.20kB) ⏫
FunctionType 5.75MB 5.75MB -0.04% (2.35kB) ⬇️
GenericContext 2.18MB 2.18MB -0.07% (1.56kB) ⬇️
Type<'db>::apply_specialization_inner_::interned_arguments 3.95MB 3.95MB -0.03% (1.17kB) ⬇️
TypeVarSetInner 1.06MB 1.06MB -0.09% (1.01kB) ⬇️
Specialization 4.80MB 4.80MB -0.02% (896.00B) ⬇️
TypeVarInference 1.40MB 1.40MB -0.06% (840.00B) ⬇️
Type<'db>::apply_specialization_inner_ 2.83MB 2.83MB -0.02% (672.00B) ⬇️
BoundTypeVarInstance 1.60MB 1.60MB -0.04% (616.00B) ⬇️
TypeVarInstance 646.97kB 646.50kB -0.07% (480.00B) ⬇️
StaticClassLiteral<'db>::implicit_attribute_inner_ 1.16MB 1.16MB +0.04% (448.00B) ⏫
... 21 more

flake8

Name Old New Diff Outcome
BoundMethodType 95.48kB 96.36kB +0.92% (896.00B) ⏫
MemberLookupKey 277.25kB 278.06kB +0.29% (832.00B) ⏫
Type<'db>::class_member_with_policy_inner_ 281.05kB 281.87kB +0.29% (832.00B) ⏫
member_lookup_with_policy_inner 287.74kB 288.43kB +0.24% (704.00B) ⏫
infer_expression_types_impl 777.18kB 777.52kB +0.04% (344.00B) ⏫
Type<'db>::apply_specialization_inner_::interned_arguments 147.58kB 147.50kB -0.05% (80.00B) ⬇️
Type<'db>::apply_specialization_inner_ 96.70kB 96.66kB -0.05% (48.00B) ⬇️
infer_definition_types 1.16MB 1.16MB +0.00% (16.00B) ⏫

trio

Name Old New Diff Outcome
FunctionType 1.23MB 1.23MB -0.05% (688.00B) ⬇️
BoundMethodType 413.44kB 414.09kB +0.16% (672.00B) ⏫
MemberLookupKey 1.15MB 1.15MB +0.05% (624.00B) ⏫
Type<'db>::class_member_with_policy_inner_ 1.30MB 1.30MB +0.04% (576.00B) ⏫
member_lookup_with_policy_inner 1.24MB 1.24MB +0.04% (520.00B) ⏫
GenericContext 568.97kB 568.52kB -0.08% (456.00B) ⬇️
Type<'db>::apply_specialization_inner_::interned_arguments 925.08kB 924.77kB -0.03% (320.00B) ⬇️
TypeVarSetInner 254.44kB 254.15kB -0.11% (296.00B) ⬇️
TypeVarInference 359.19kB 358.95kB -0.07% (240.00B) ⬇️
Specialization 1.07MB 1.07MB -0.02% (224.00B) ⬇️
Type<'db>::apply_specialization_inner_ 651.64kB 651.44kB -0.03% (208.00B) ⬇️
TypeVarInstance 147.56kB 147.38kB -0.13% (192.00B) ⬇️
TypePair 960.75kB 960.56kB -0.02% (192.00B) ⬇️
BoundTypeVarInstance 328.02kB 327.85kB -0.05% (176.00B) ⬇️
infer_expression_types_impl 5.84MB 5.84MB +0.00% (168.00B) ⏫
... 8 more

@astral-sh-bot

astral-sh-bot Bot commented Sep 17, 2026 •

Copy link
Copy Markdown

ecosystem-analyzer results

Lint rule Added Removed Changed
redundant-condition-strict 4 0 0
Total 4 0 0

Flaky changes detected. This PR summary excludes flaky changes; see the HTML report for details.

Raw diff:

dd-trace-py (https://github.com/DataDog/dd-trace-py)
+ ddtrace/internal/settings/errortracking.py:43:48 warning[redundant-condition-strict] Condition `config._report_handled_errors in ["all", "user", "third_party"]` is always false

paasta (https://github.com/yelp/paasta)
+ paasta_tools/paastaapi/api_client.py:211:12 warning[redundant-condition-strict] Condition `response_type not in ["file", "bytes"]` is always true

scipy (https://github.com/scipy/scipy)
+ scipy/optimize/tests/test__root.py:25:16 warning[redundant-condition-strict] Condition `method in ('linearmixing', 'excitingmixing')` is always false

scrapy (https://github.com/scrapy/scrapy)
+ scrapy/core/engine.py:267:12 warning[redundant-condition-strict] Condition `self._state in (_EngineState.STOPPING, _EngineState.STOPPED)` is always false

Full report with detailed diff (timing results)

@carljm

carljm commented Sep 17, 2026 •

Copy link
Copy Markdown
Contributor Author
Ecosystem report

PR #28676 ecosystem summary

The comparison for head 90111f10da9359722abb903351b89aaae0f9bcba adds four redundant-condition-strict warnings. Two correctly identify disjoint membership checks in paasta and SciPy. Two expose existing limitations: dynamically replaced configuration attributes in dd-trace-py, and attribute narrowing retained across mutations in Scrapy. Equivalent equality checks already exhibit both limitations on the base revision. These existing limitations are accepted for this change.

Affected projects

1. dd-trace-py: dynamically replaced configuration attributes

Report entry: ddtrace/internal/settings/errortracking.py:43

The PR adds warning[redundant-condition-strict] Condition config._report_handled_errors in ["all", "user", "third_party"] is always false. At runtime, envier replaces the configuration marker with a parsed string through setattr, so the condition can be true. Both revisions infer the attribute as EnvVariable[str], following the factory's return annotation. The new warning extends existing comparison reasoning to inline list membership; the base already reports the following equality checks against these strings as always false.

Existing ty issue: ty#2016 discusses the underlying limitation in modeling dynamic setattr effects.

class Config:
    value = 0

config = Config()
setattr(config, 'value', 'all')
# Merge base: no diagnostic
# PR: warning[redundant-condition-strict] Condition `config.value in ['all']` is always false
if config.value in ['all']:
    pass

The condition is true at runtime; ty retains the declared attribute type after setattr.

2. paasta: a tuple parameter checked against strings

Report entry: paasta_tools/paastaapi/api_client.py:211

The parameter is annotated as Optional[Tuple[Any]], but the condition tests response_type not in ["file", "bytes"]. The base reports no diagnostic at this condition; the PR correctly reports that it is always true under the existing non-strict equality policy. An ordinary tuple or None cannot match either string.

# Merge base: no diagnostic
# PR: warning[redundant-condition-strict] Condition `() not in ['']` is always true
if () not in ['']:
    pass

3. SciPy: a literal union excludes both tested method names

Report entry: scipy/optimize/tests/test__root.py:25

The loop iterates over seven literal method names, including neither linearmixing nor excitingmixing. The PR correctly reports method in ('linearmixing', 'excitingmixing') as always false. This also improves existing tuple membership: each alternative of a union of builtin literal values now retains reflexive equality semantics, allowing ty to establish absence.

for method in ['hybr', 'lm']:
    # Merge base: no diagnostic
    # PR: warning[redundant-condition-strict] Condition `method in ('linearmixing',)` is always false
    if method in ('linearmixing',):
        pass

4. Scrapy: attribute narrowing survives state changes

Report entry: scrapy/core/engine.py:267

The PR adds warning[redundant-condition-strict] Condition self._state in (_EngineState.STOPPING, _EngineState.STOPPED) is always false. An earlier guard narrows the state to CREATED, SPIDER_OPENING, or SPIDER_OPEN. Both revisions retain exactly that union after _transition_to changes the state and an awaited signal handler may stop the engine. Consequently, the new warning is a runtime false positive.

The PR does not change the narrowing. Its union handling for reflexive equality lets existing tuple membership establish that the inferred union is disjoint from the tested states. Replacing only the final membership condition with the equivalent two == comparisons already produces the same false positive on the base revision. The example below preserves the mutation responsible for the discrepancy between runtime behavior and static narrowing; Engine().start() returns True.

Existing ty issues: ty#2792 tracks attribute narrowing across mutating method calls; ty#885 discusses narrowing in unsound contexts, including await.

class Engine:
    _state: bool | None = False

    def stop(self):
        self._state = True

    def start(self):
        if self._state not in (False, None):
            return
        self.stop()
        # Merge base: no diagnostic
        # PR: warning[redundant-condition-strict] Condition `self._state in (True,)` is always false
        if self._state in (True,):
            return True

Reproduction

  • Detailed report: ecosystem-analyzer report
  • Actions run: run 35658236839, attempt 1
  • Ruff comparison: e86121d0d1 to 3fa80d1734, the merge of PR head 90111f10da.
  • ecosystem-analyzer: a31ef389f3
  • mypy-primer: 3058720299
  • Dependency cutoff: 2026-09-21T21:37:15Z.
  • Project Python: dd-trace-py, paasta, SciPy, and Scrapy use Python 3.11.
  • Project analysis mode: Scrapy uses strict settings; dd-trace-py, paasta, and SciPy use non-strict settings.
  • Comparison method: run both exact-revision profiling binaries with the PR ecosystem configuration installed as user-level configuration and each pinned project environment. Use ty check for SciPy; ty check . for Scrapy; ty check paasta_tools for paasta; and ty check --exclude tests/appsec/iast/fixtures/aspects/str/non_utf8_content.py --exclude tests/appsec/iast/fixtures/ast/str/non_utf8_content.py for dd-trace-py. Each command includes --python <project-venv> --output-format concise, with TY_CONFIG_FILE unset. For Scrapy, also include --config analysis.strict-equality-semantics=true --config analysis.strict-generic-narrowing=true on both binaries. The standalone minimized examples need only ty check <file> --python-version 3.11 with the same user-level configuration and their respective strictness settings.

@carljm
carljm marked this pull request as ready for review September 17, 2026 22:43
@carljm
carljm requested a review from a team as a code owner September 17, 2026 22:43
@astral-sh-bot
astral-sh-bot Bot requested a review from dcreager September 17, 2026 22:43
@carljm
carljm requested review from AlexWaygood and removed request for dcreager September 17, 2026 22:44

@AlexWaygood AlexWaygood left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Some of this feels a bit overcomplicated right now

Comment thread crates/ty_python_semantic/src/types/equality.rs
Comment thread crates/ty_python_semantic/src/types/infer/builder.rs Outdated
Comment thread crates/ty_python_semantic/src/types/infer/comparisons.rs Outdated
Comment thread crates/ty_python_semantic/src/types/infer/comparisons.rs Outdated
Comment thread crates/ty_python_semantic/src/types/iteration.rs Outdated
Comment thread crates/ty_python_semantic/src/types/iteration.rs Outdated
@carljm
carljm merged commit 660350b into main Sep 21, 2026
72 checks passed
@carljm
carljm deleted the cjm/inline-literal-membership branch September 21, 2026 21:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ty Multi-file analysis & type inference

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants