Fix TypedDict get, pop, and setdefault type evaluation with union keys - #11613
Fix TypedDict get, pop, and setdefault type evaluation with union keys#11613Henry Su (hsusul) wants to merge 3 commits into
Conversation
|
🔒 Automated review in progress — Stella Huang (@StellaHuang95) is auto-reviewing this PR. |
| let isBound = false; | ||
| let boundType = overload.priv.strippedFirstParamType; | ||
| if (boundType) { | ||
| isBound = true; |
There was a problem hiding this comment.
Issue · Please address or respond
Please gate this transform on FunctionType.isSynthesizedMethod(overload). Matching only the method name and a TypedDict receiver can intercept a user-defined overloaded get, pop, or setdefault function and bypass its actual overload resolution when passed a union key.
| const keyTypeResult = keyArg.typeResult ?? evaluator.getTypeOfExpression(keyNode); | ||
| const keyType = keyTypeResult.type; | ||
|
|
||
| if (!isUnion(keyType)) { |
There was a problem hiding this comment.
Issue · Please address or respond
This early return bypasses validateOverloadedArgTypes, so union-key calls no longer receive normal argument validation. For example, extra arguments are ignored, keyword arguments do not match the positional indexes, and an unbound receiver is never validated. Preserve normal overload validation or fall back to it for anything other than the supported positional call shape.
|
GitHub cannot anchor PR review comments to unchanged lines in the diff. Falling back to a general PR comment for packages/pyright-internal/src/analyzer/typedDicts.ts:L1810.
This implements the behavior only in the synchronous evaluator. The async Pylance evaluator has a separate |
|
GitHub cannot anchor PR review comments to unchanged lines in the diff. Falling back to a general PR comment for packages/pyright-internal/src/analyzer/typedDicts.ts:L1810.
This reimplements TypedDict key-access behavior already encoded by the synthesized methods and |
…er and call shape
|
Thank you Stella Huang (@StellaHuang95) for the thorough review! I have updated the PR to address all your points:
All tests ( |
| return { classType: boundType, methodName: name, isBound }; | ||
| } | ||
|
|
||
| return undefined; |
There was a problem hiding this comment.
Issue · Please address or respond
Use isMethodType(overload) rather than inferring bound status solely from strippedFirstParamType. The shared helper also handles pre-bound constructor methods, so duplicating only part of its logic can misclassify synthesized methods if their binding form changes.
| return undefined; | ||
| } | ||
|
|
||
| // Require all arguments to be simple positional (no keyword names, no *args/**kwargs) |
There was a problem hiding this comment.
Issue · Please address or respond
setdefault requires its default argument, but the bound-call minimum here is one argument. A union-key call like p.setdefault(k) is intercepted and returns a field type without an error, whereas normal overload validation rejects it. Require the default argument for setdefault, or fall through to normal validation when it is absent.
| } | ||
| } | ||
| return entry.valueType; | ||
| } |
There was a problem hiding this comment.
Issue · Please address or respond
A union key containing a non-string subtype reaches this fallback and returns Unknown without setting argumentErrors or emitting a diagnostic. For example, Literal["name"] | int should still reject the int component as the normal str key overload does. Preserve normal argument validation for unsupported key subtypes.
| evaluator.addDiagnostic( | ||
| DiagnosticRule.reportGeneralTypeIssues, | ||
| LocAddendum.keyUndefined().format({ | ||
| name: entryName, |
There was a problem hiding this comment.
Info · Optional note
Propagate isTypeIncomplete from the evaluated key, default, and unbound receiver into this CallResult. Returning only the type and argument-error flag can cause an incomplete call result to be treated as final.
…tdefault, validate non-string key subtypes, propagate isTypeIncomplete
|
Thank you Stella Huang (@StellaHuang95)! I have updated the PR to address your latest comments:
All unit tests ( |
|
Diff from mypy_primer, showing the effect of this PR on open source code: sympy (https://github.com/sympy/sympy)
- .../projects/sympy/sympy/solvers/ode/lie_group.py:619:61 - error: Operator "-" not supported for type "Unknown | Basic" (reportOperatorIssue)
- .../projects/sympy/sympy/solvers/ode/nonhomogeneous.py:225:45 - error: Cannot access attribute "has" for class "tuple[Expr, int]"
- Attribute "has" is unknown (reportAttributeAccessIssue)
+ .../projects/sympy/sympy/solvers/ode/nonhomogeneous.py:223:22 - error: No overloads for "__new__" match the provided arguments (reportCallIssue)
+ .../projects/sympy/sympy/solvers/ode/nonhomogeneous.py:223:25 - error: Argument of type "CRootOf | tuple[Expr, int]" cannot be assigned to parameter "arg" of type "Expr" in function "__new__"
+ Type "CRootOf | tuple[Expr, int]" is not assignable to type "Expr"
+ "tuple[Expr, int]" is not assignable to "Expr" (reportArgumentType)
+ .../projects/sympy/sympy/solvers/ode/nonhomogeneous.py:238:40 - error: No overloads for "__new__" match the provided arguments (reportCallIssue)
+ .../projects/sympy/sympy/solvers/ode/nonhomogeneous.py:238:50 - error: Argument of type "CRootOf | tuple[Expr, int]" cannot be assigned to parameter "arg" of type "Expr" in function "__new__"
+ Type "CRootOf | tuple[Expr, int]" is not assignable to type "Expr"
+ "tuple[Expr, int]" is not assignable to "Expr" (reportArgumentType)
- .../projects/sympy/sympy/solvers/ode/ode.py:1433:17 - error: Argument of type "One | NegativeOne | Zero | Integer | NaN | ComplexInfinity | Rational | Unknown | Expr" cannot be assigned to parameter "value" of type "Zero" in function "__setitem__"
+ .../projects/sympy/sympy/solvers/ode/ode.py:1433:17 - error: Argument of type "One | NegativeOne | Zero | Integer | NaN | ComplexInfinity | Rational | Infinity | NegativeInfinity | Float | Number | Expr" cannot be assigned to parameter "value" of type "Zero" in function "__setitem__"
- Type "One | NegativeOne | Zero | Integer | NaN | ComplexInfinity | Rational | Unknown | Expr" is not assignable to type "Zero"
+ Type "One | NegativeOne | Zero | Integer | NaN | ComplexInfinity | Rational | Infinity | NegativeInfinity | Float | Number | Expr" is not assignable to type "Zero"
- .../projects/sympy/sympy/solvers/ode/ode.py:2961:17 - error: Argument of type "One | NegativeOne | Zero | Integer | NaN | ComplexInfinity | Rational | Unknown | Expr" cannot be assigned to parameter "value" of type "Zero" in function "__setitem__"
+ .../projects/sympy/sympy/solvers/ode/ode.py:2961:17 - error: Argument of type "One | NegativeOne | Zero | Integer | NaN | ComplexInfinity | Rational | Infinity | NegativeInfinity | Float | Number | Expr" cannot be assigned to parameter "value" of type "Zero" in function "__setitem__"
- Type "One | NegativeOne | Zero | Integer | NaN | ComplexInfinity | Rational | Unknown | Expr" is not assignable to type "Zero"
+ Type "One | NegativeOne | Zero | Integer | NaN | ComplexInfinity | Rational | Infinity | NegativeInfinity | Float | Number | Expr" is not assignable to type "Zero"
- .../projects/sympy/sympy/solvers/ode/single.py:867:9 - error: Expression with type "tuple[ComplexInfinity | NaN | Rational | Zero | Infinity | NegativeInfinity | Float | NotImplementedType | Expr | Unknown | Any, list[tuple[Unknown, Unknown]] | list[Unknown]] | tuple[ComplexInfinity | NaN | Rational | Zero | Infinity | NegativeInfinity | Float | NotImplementedType | Expr | Unknown | Any, list[tuple[Unknown, Unknown]] | list[Unknown], list[tuple[Unknown, Unknown]] | list[Unknown]]" cannot be assigned to target tuple
+ .../projects/sympy/sympy/solvers/ode/single.py:867:9 - error: Expression with type "tuple[ComplexInfinity | Unknown | Any, list[tuple[Unknown, Unknown]] | list[Unknown]] | tuple[ComplexInfinity | Unknown | Any, list[tuple[Unknown, Unknown]] | list[Unknown], list[tuple[Unknown, Unknown]] | list[Unknown]]" cannot be assigned to target tuple
- Type "tuple[ComplexInfinity | NaN | Rational | Zero | Infinity | NegativeInfinity | Float | NotImplementedType | Expr | Unknown | Any, list[tuple[Unknown, Unknown]] | list[Unknown], list[tuple[Unknown, Unknown]] | list[Unknown]]" is incompatible with target tuple
+ Type "tuple[ComplexInfinity | Unknown | Any, list[tuple[Unknown, Unknown]] | list[Unknown], list[tuple[Unknown, Unknown]] | list[Unknown]]" is incompatible with target tuple
- .../projects/sympy/sympy/solvers/ode/single.py:2646:27 - error: Argument of type "One | NegativeOne | Zero | Integer | NaN | ComplexInfinity | Rational | Unknown | Expr | NegativeInfinity | Number | int" cannot be assigned to parameter "stop" of type "SupportsIndex" in function "__new__"
- Type "One | NegativeOne | Zero | Integer | NaN | ComplexInfinity | Rational | Unknown | Expr | NegativeInfinity | Number | int" is not assignable to type "SupportsIndex"
- "Expr" is incompatible with protocol "SupportsIndex"
- "__index__" is not present (reportArgumentType)
- .../projects/sympy/sympy/solvers/ode/single.py:2652:9 - error: Operator "+=" not supported for types "One | NegativeOne | Zero | Integer | NaN | ComplexInfinity | Rational | Unknown | Expr | Any" and "Basic | Any | Unknown"
+ .../projects/sympy/sympy/solvers/ode/single.py:2652:9 - error: Operator "+=" not supported for types "Unknown | Any | Zero | One | NegativeOne | Integer | NaN | ComplexInfinity | Rational | Infinity | NegativeInfinity | Float | Number | Expr" and "Basic | Any | Unknown"
+ Operator "+" not supported for types "Number" and "Basic"
- .../projects/sympy/sympy/solvers/solveset.py:776:18 - error: Argument of type "Expr | Unknown | None" cannot be assigned to parameter "expr" of type "Expr" in function "together"
+ .../projects/sympy/sympy/solvers/solveset.py:776:18 - error: Argument of type "Unknown | None" cannot be assigned to parameter "expr" of type "Expr" in function "together"
- Type "Expr | Unknown | None" is not assignable to type "Expr"
+ Type "Unknown | None" is not assignable to type "Expr"
+ .../projects/sympy/sympy/stats/crv_types.py:2544:9 - error: Method "_cdf" overrides class "SingleContinuousDistribution" in an incompatible manner
+ Return type mismatch: base method returns type "None", override returns type "ComplexInfinity | Unknown"
+ Type "ComplexInfinity | Unknown" is not assignable to type "None"
+ "ComplexInfinity" is not assignable to "None" (reportIncompatibleMethodOverride)
+ .../projects/sympy/sympy/stats/crv_types.py:2723:9 - error: Method "_cdf" overrides class "SingleContinuousDistribution" in an incompatible manner
+ Return type mismatch: base method returns type "None", override returns type "Expr | Unknown"
+ Type "Expr | Unknown" is not assignable to type "None"
+ "Expr" is not assignable to "None" (reportIncompatibleMethodOverride)
- .../projects/sympy/sympy/stats/drv.py:269:22 - error: Argument of type "Generator[tuple[Unknown, ...] | Unknown | Sum | Expr | ZeroMatrix | Add | Zero | NaN | Piecewise | Basic | int | None, None, None]" cannot be assigned to parameter "iterable" of type "Iterable[_SupportsSumNoDefaultT@sum]" in function "sum"
+ .../projects/sympy/sympy/stats/drv.py:269:22 - error: Argument of type "Generator[tuple[Unknown, ...] | Unknown | Sum | Expr | ZeroMatrix | Zero | NaN | Piecewise | Basic | int | None, None, None]" cannot be assigned to parameter "iterable" of type "Iterable[_SupportsSumNoDefaultT@sum]" in function "sum"
- "Generator[tuple[Unknown, ...] | Unknown | Sum | Expr | ZeroMatrix | Add | Zero | NaN | Piecewise | Basic | int | None, None, None]" is not assignable to "Iterable[_SupportsSumNoDefaultT@sum]"
+ "Generator[tuple[Unknown, ...] | Unknown | Sum | Expr | ZeroMatrix | Zero | NaN | Piecewise | Basic | int | None, None, None]" is not assignable to "Iterable[_SupportsSumNoDefaultT@sum]"
- Type parameter "_T_co@Iterable" is covariant, but "tuple[Unknown, ...] | Unknown | Sum | Expr | ZeroMatrix | Add | Zero | NaN | Piecewise | Basic | int | None" is not a subtype of "_SupportsSumNoDefaultT@sum"
+ Type parameter "_T_co@Iterable" is covariant, but "tuple[Unknown, ...] | Unknown | Sum | Expr | ZeroMatrix | Zero | NaN | Piecewise | Basic | int | None" is not a subtype of "_SupportsSumNoDefaultT@sum"
- Type "tuple[Unknown, ...] | Unknown | Sum | Expr | ZeroMatrix | Add | Zero | NaN | Piecewise | Basic | int | None" is not assignable to type "_SupportsSumWithNoDefaultGiven"
+ Type "tuple[Unknown, ...] | Unknown | Sum | Expr | ZeroMatrix | Zero | NaN | Piecewise | Basic | int | None" is not assignable to type "_SupportsSumWithNoDefaultGiven"
- Type "tuple[Unknown, ...] | Unknown | Sum | Expr | ZeroMatrix | Add | Zero | NaN | Piecewise | Basic | int | None" is not assignable to type "_SupportsSumWithNoDefaultGiven"
+ Type "tuple[Unknown, ...] | Unknown | Sum | Expr | ZeroMatrix | Zero | NaN | Piecewise | Basic | int | None" is not assignable to type "_SupportsSumWithNoDefaultGiven"
- .../projects/sympy/sympy/stats/drv_types.py:293:16 - error: Operator "*" not supported for types "Expr" and "tuple[Unknown, ...] | Unknown | Sum | Expr | ZeroMatrix | Add | Zero | NaN | Piecewise | Basic"
+ .../projects/sympy/sympy/stats/drv_types.py:293:16 - error: Operator "*" not supported for types "Expr" and "tuple[Unknown, ...] | Unknown | Sum | Expr | ZeroMatrix | Zero | NaN | Piecewise | Basic"
- .../projects/sympy/sympy/stats/joint_rv_types.py:576:27 - error: Argument of type "One | NegativeOne | Zero | Integer | NaN | ComplexInfinity | Rational | Unknown | Expr" cannot be assigned to parameter "stop" of type "SupportsIndex" in function "__new__"
+ .../projects/sympy/sympy/stats/joint_rv_types.py:576:27 - error: Argument of type "One | NegativeOne | Zero | Integer | NaN | ComplexInfinity | Rational | Infinity | NegativeInfinity | Float | Number | Expr" cannot be assigned to parameter "stop" of type "SupportsIndex" in function "__new__"
... (truncated 1415 lines) ...
prefect (https://github.com/PrefectHQ/prefect)
- .../projects/prefect/src/prefect/_internal/launchers.py:81:12 - error: Type "BundleLauncher | None" is not assignable to return type "list[str] | None"
- Type "BundleLauncher | None" is not assignable to type "list[str] | None"
- Type "BundleLauncherOverride" is not assignable to type "list[str] | None"
- "BundleLauncherOverride" is not assignable to "list[str]"
- "BundleLauncherOverride" is not assignable to "None" (reportReturnType)
- 6406 errors, 201 warnings, 0 informations
+ 6405 errors, 201 warnings, 0 informations
|
|
This mypy_primer difference seems odd? |
| if (defaultType) { | ||
| return defaultType; | ||
| } | ||
| evaluator.addDiagnostic( |
There was a problem hiding this comment.
Warning · Non-blocking recommendation
This gives required and ReadOnly keys precise pop return types, but the synthesized single-key overloads intentionally specialize pop only for non-required, non-ReadOnly keys; other keys use the pop(str) -> object fallback. Make the union transform follow that same rule so union-key calls do not diverge from single-key behavior.
| return UnknownType.create(); | ||
| } | ||
| if (defaultType && defaultArg) { | ||
| const diag = new DiagnosticAddendum(); |
There was a problem hiding this comment.
Warning · Non-blocking recommendation
The ReadOnly setdefault diagnostic uses reportGeneralTypeIssues, unlike the corresponding pop and indexed-assignment paths, which use reportTypedDictNotRequiredAccess. This prevents users from consistently suppressing the ReadOnly TypedDict-access diagnostic; align this branch with the established rule.
Stella Huang (StellaHuang95)
left a comment
There was a problem hiding this comment.
Approved via Review Center.
Rich Chiodo (rchiodo)
left a comment
There was a problem hiding this comment.
Approved via Review Center.
Rich Chiodo (rchiodo)
left a comment
There was a problem hiding this comment.
Approved via Review Center.
Summary
Fixes a bug where Pyright fails to properly evaluate
TypedDictsynthesized methods (get,pop, andsetdefault) when the key argument is a union of literal string types (such asLiteral["a", "b"]).Reproduction
Current vs. Corrected Behavior
Current Behavior:
p.get(k)returnedAny | NonewhenkwasLiteral["name", "age"].p.pop(k)returnedobjectwhenkwasLiteral["name", "age"].p.setdefault(k, default)reported a false-positive diagnostic errorNo overloads for "setdefault" match the provided arguments.Corrected Behavior:
p.get(k)correctly infersstr | int.p.pop(k)correctly infersstr | int.p.setdefault(k, default)correctly infersstr | intand validatesdefaulttypes against each key's value type.Typing Rule & Root Cause
When invoking
get,pop, orsetdefaulton aTypedDictinstance, Pyright evaluates the call against the synthesizedOverloadedTypecreated for thatTypedDict. The synthesized overloads each accept a single literal string key (e.g.,get(k: Literal["name"])andget(k: Literal["age"])).When the argument
kis a union of literal key types (Literal["name", "age"]), overload evaluation (validateOverloadedArgTypes) checks whetherLiteral["name", "age"]as a whole is assignable toLiteral["name"]orLiteral["age"]. Because it is not a subtype of any single key overload:get(k)fell through to the fallback overloadget(k: str), returningAny | None.pop(k)fell through to the fallback overloadpop(k: str), returningobject.setdefault(k)has no generic fallback overload, resulting in no matching overloads.In contrast, indexing
p[k](__getitem__) usesgetTypeOfIndexedTypedDictwhich maps overmapSubtypes(keyType, ...)to evaluate each subtype key and combine the resulting types.Implementation Details
getTypedDictClassFromMethodto detect calls to synthesizedTypedDictmethods (get,pop,setdefault) on both bound and unbound method calls.applyTypedDictMethodTransforminpackages/pyright-internal/src/analyzer/typedDicts.tswhich maps over union key subtypes (mapSubtypes) and evaluates member access, defaults, and diagnostic rules (e.g.ReadOnlykeys anddefaultparameter type mismatches) consistently with__getitem__.validateCallForOverloadedinpackages/pyright-internal/src/analyzer/typeEvaluator.ts.Regression Coverage
Added
packages/pyright-internal/src/tests/samples/typedDict28.pyand registeredTypedDict28inpackages/pyright-internal/src/tests/typeEvaluator7.test.tstestingget,pop,setdefault, default values,ReadOnlydiagnostics, and unbound method calls with union literal keys.Validation Results
npx jest typeEvaluator7.test.ts: PASS (168 tests passed, including newTypedDict28)npm run check(syncpack, eslint, prettier): PASSnpm run typecheck: PASSgit diff --check: PASS (clean diff)Compatibility Considerations
This change is additive and specifically targets calls to
TypedDictmethods when key arguments are union types. Normal overload resolution for single keys, standarddictoperations, and non-TypedDict methods remain completely unaffected.