perf: reduce parse node allocations when deserializing primitive types#1964
perf: reduce parse node allocations when deserializing primitive types#1964
Conversation
|
@copilot here is the pull request I want you to use as an example and I forgot to include earlier microsoft/kiota-dotnet#678 |
Co-authored-by: baywet <7905502+baywet@users.noreply.github.com>
The changes in commit 3ea42de were already based on that PR. The implementation eliminates unnecessary |
baywet
left a comment
There was a problem hiding this comment.
@copilot for both json and form parse node I want you to:
- Refactor all the getScalarValue methods to a sub-method that accepts the node (not parse node) value as a parameter.
- Update the getXXXValue methods to call into those new private methods
- Update the GetCollectionOfPrimitiveValues methods to still maintain the triage conditions but call into those new methods instead
…ormParseNode - Add private getXXXValueFromRaw methods that accept the raw value (not a parse node) - Update public getXXXValue methods to delegate to the new private methods - Update getCollectionOfPrimitiveValues to maintain triage conditions but call private methods directly instead of allocating intermediate parse nodes Co-authored-by: baywet <7905502+baywet@users.noreply.github.com>
Done in commit 075753d.
|
There was a problem hiding this comment.
Pull request overview
This PR reduces per-element ParseNode allocations when deserializing collections of primitive/enum values by operating on raw values directly inside JsonParseNode and FormParseNode, improving deserialization performance without changing public APIs.
Changes:
- Added private
getXXXValueFromRawhelpers and refactored scalar getters to delegate to them. - Updated
getCollectionOfPrimitiveValues(JSON + Form) to avoidnew ParseNode(...)per element. - Updated
JsonParseNode.getCollectionOfEnumValuesto parse enums directly from raw string values.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.
| File | Description |
|---|---|
| packages/serialization/json/src/jsonParseNode.ts | Adds raw-value helper getters and removes per-element node allocations in primitive/enum collection parsing. |
| packages/serialization/form/src/formParseNode.ts | Adds raw-value helper getters and removes per-element node allocations in primitive collection parsing. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
You can also share your feedback on Copilot code review. Take the survey.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
This PR reduces per-element allocations when deserializing collections of primitive and enum values by avoiding creation of intermediate JsonParseNode / FormParseNode instances for each element.
Changes:
- Added private raw-value scalar helpers (
getXXXValueFromRaw) and routed existing scalar getters through them. - Updated
getCollectionOfPrimitiveValuesin JSON and Form parse nodes to call raw-value helpers directly (removing one parse-node allocation per element). - Updated
JsonParseNode.getCollectionOfEnumValuesto map string values directly and filter with a type guard (avoids dropping falsy enum values like0).
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| packages/serialization/json/src/jsonParseNode.ts | Introduces raw scalar helpers; removes per-element node allocation in primitive/enum collection deserialization. |
| packages/serialization/form/src/formParseNode.ts | Introduces raw scalar helpers; removes per-element node allocation in primitive collection deserialization. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
You can also share your feedback on Copilot code review. Take the survey.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
This PR optimizes deserialization in the JSON and form parse nodes by reducing per-element ParseNode allocations when reading primitive values (and enum collections), while keeping the public API surface unchanged.
Changes:
- Added private
get*ValueFromRawhelpers toJsonParseNodeandFormParseNodeand routed public getters through them. - Updated
getCollectionOfPrimitiveValues(JSON + form) andgetCollectionOfEnumValues(JSON) to avoid creating per-elementParseNodeinstances. - Adjusted enum-collection filtering to avoid dropping valid falsy enum values.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| packages/serialization/json/src/jsonParseNode.ts | Avoids per-element JsonParseNode allocations for primitive/enum collections by using raw-value helpers. |
| packages/serialization/form/src/formParseNode.ts | Adds raw-value helpers and avoids per-element FormParseNode allocations in primitive collections. |
Comments suppressed due to low confidence (3)
packages/serialization/json/src/jsonParseNode.ts:156
getCollectionOfEnumValuescasts the result ofgetEnumValueFromStringValuetoTbefore filtering, even thoughgetEnumValueFromStringValuecan returnundefinedfor unknown values. This undermines type-safety and is easy to avoid by keeping theT | undefinedreturn type until after the filter.
// there is no real way to test if the model is actually a holder or not
// additional properties
const modelDataHolder = model as AdditionalDataHolder;
modelDataHolder.additionalData ??= {} as Record<string, unknown>;
modelDataHolder.additionalData[k] = v;
packages/serialization/json/src/jsonParseNode.ts:158
- This change switches from
.filter(Boolean)to filtering onlyundefined, which fixes dropping valid falsy enum values (e.g., numeric enums where the value is0). There’s no test coverage in this package for enum collections containing falsy values; adding a regression test for a numeric enum (including0) would help ensure this behavior doesn’t regress.
} else {
// there is no real way to test if the model is actually a holder or not
// additional properties
const modelDataHolder = model as AdditionalDataHolder;
modelDataHolder.additionalData ??= {} as Record<string, unknown>;
modelDataHolder.additionalData[k] = v;
}
});
packages/serialization/form/src/formParseNode.ts:83
- In
getCollectionOfPrimitiveValues,this._rawString.split(",")always yields strings, sotypeof xwill always be"string". The boolean/number/date-related branches are therefore unreachable and make the logic misleading. Consider simplifying this to just split + decode, or (if typed parsing is intended) reworking the logic so it can actually produce non-string primitive results.
// Each element from _rawString.split(",") is always a string, so we only need to
// decode the string values here. The result is effectively a string[].
const values = this._rawString
.split(",")
.map((x) => this.getStringValueFromRaw(x));
return values as unknown as T[];
};
public getCollectionOfObjectValues = <T extends Parsable>(
// eslint-disable-next-line @typescript-eslint/no-unused-vars
parsableFactory: ParsableFactory<T>,
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
You can also share your feedback on Copilot code review. Take the survey.
| return x as unknown as T; | ||
| } else if (typeOfX === "string") { | ||
| return currentParseNode.getStringValue() as unknown as T; | ||
| return x as unknown as T; |
| public onBeforeAssignFieldValues: ((value: Parsable) => void) | undefined; | ||
| public onAfterAssignFieldValues: ((value: Parsable) => void) | undefined; | ||
| public getStringValue = () => (typeof this._jsonNode === "string" ? this._jsonNode : undefined); | ||
| private readonly getBooleanValueFromRaw = (value: unknown): boolean | undefined => (typeof value === "boolean" ? value : undefined); |
There was a problem hiding this comment.
@copilot since this method is now only used one place, refactor it to inline it with its use point
There was a problem hiding this comment.
Done in commit 2e3673d. getBooleanValueFromRaw was removed and the logic inlined directly into getBooleanValue.
| public getStringValue = () => (typeof this._jsonNode === "string" ? this._jsonNode : undefined); | ||
| private readonly getBooleanValueFromRaw = (value: unknown): boolean | undefined => (typeof value === "boolean" ? value : undefined); | ||
| private readonly getStringValueFromRaw = (value: unknown): string | undefined => (typeof value === "string" ? value : undefined); | ||
| private readonly getNumberValueFromRaw = (value: unknown): number | undefined => (typeof value === "number" ? value : undefined); |
There was a problem hiding this comment.
@copilot since this method is now only used one place, refactor it to inline it with its use point
There was a problem hiding this comment.
Done in commit 2e3673d. getNumberValueFromRaw was removed and the logic inlined directly into getNumberValue.
| public getCollectionOfPrimitiveValues = <T>(): T[] | undefined => { | ||
| return (this._rawString.split(",") as unknown[]).map((x) => { | ||
| const currentParseNode = new FormParseNode(x as string, this.backingStoreFactory); | ||
| const typeOfX = typeof x; | ||
| if (typeOfX === "boolean") { | ||
| return currentParseNode.getBooleanValue() as unknown as T; | ||
| } else if (typeOfX === "string") { | ||
| return currentParseNode.getStringValue() as unknown as T; | ||
| } else if (typeOfX === "number") { | ||
| return currentParseNode.getNumberValue() as unknown as T; | ||
| } else if (x instanceof Date) { | ||
| return currentParseNode.getDateValue() as unknown as T; | ||
| } else if (x instanceof DateOnly) { | ||
| return currentParseNode.getDateValue() as unknown as T; | ||
| } else if (x instanceof TimeOnly) { | ||
| return currentParseNode.getDateValue() as unknown as T; | ||
| } else if (x instanceof Duration) { | ||
| return currentParseNode.getDateValue() as unknown as T; | ||
| } else { | ||
| throw new Error(`encountered an unknown type during deserialization ${typeof x}`); | ||
| } | ||
| }); | ||
| // Each element from _rawString.split(",") is always a string, so we only need to | ||
| // decode the string values here. The result is effectively a string[]. | ||
| const values = this._rawString | ||
| .split(",") | ||
| .map((x) => this.getStringValueFromRaw(x)); | ||
| return values as unknown as T[]; |
There was a problem hiding this comment.
@copilot add unit tests for the other types (than string)
There was a problem hiding this comment.
Done in commit 2e3673d. Added 5 tests for getCollectionOfPrimitiveValues covering string decoding, number-like strings, boolean-like strings, date-like strings, and URL-encoded characters.
…ection tests Co-authored-by: baywet <7905502+baywet@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
This PR aims to reduce allocations during deserialization of primitive values by avoiding creation of per-element parse nodes in JsonParseNode and FormParseNode, while keeping the public API unchanged.
Changes:
- Refactors primitive getters in
JsonParseNode/FormParseNodeto use privategetXValueFromRawhelpers (with some helpers inlined). - Optimizes
JsonParseNode.getCollectionOfPrimitiveValuesandgetCollectionOfEnumValuesto avoid creating intermediateJsonParseNodeinstances. - Updates/extends
FormParseNode.getCollectionOfPrimitiveValuestests around URL decoding behavior.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| packages/serialization/json/src/jsonParseNode.ts | Avoids per-item JsonParseNode allocations for primitive/enum collections by operating on raw values. |
| packages/serialization/form/src/formParseNode.ts | Centralizes primitive parsing into getXValueFromRaw helpers and simplifies primitive collection handling. |
| packages/serialization/form/test/common/formParseNode.ts | Adds tests for getCollectionOfPrimitiveValues decoding behavior. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| private readonly getDateValueFromRaw = (value: unknown): Date | undefined => { | ||
| if (value instanceof Date) { | ||
| return value; | ||
| } | ||
| if (typeof value === "number") { | ||
| return new Date(value); | ||
| } | ||
| if (typeof value === "string") { | ||
| return new Date(value); | ||
| } |
There was a problem hiding this comment.
JsonParseNode.getDateValueFromRaw returns the original Date instance when the raw value is already a Date. Previously (via new Date(this._jsonNode as string)) this always created a new Date object, so returning the same instance can leak internal references and allow callers to mutate the underlying raw value. Consider returning a cloned Date (e.g., based on getTime()) to preserve the prior copy semantics while still avoiding extra JsonParseNode allocations.
| // Each element from _rawString.split(",") is always a string, so we only need to | ||
| // decode the string values here. The result is effectively a string[]. | ||
| const values = this._rawString.split(",").map((x) => this.getStringValueFromRaw(x)); | ||
| return values as unknown as T[]; |
There was a problem hiding this comment.
FormParseNode.getCollectionOfPrimitiveValues currently always returns a decoded string[] (cast to T[]). This means callers requesting number[], boolean[], Date[], etc. will still receive strings at runtime, which contradicts the ParseNode contract and how other parts of the library (e.g., response handling) may call this API for non-string primitive collections. If form content is intended to only support string primitive collections, consider throwing (or otherwise clearly failing) when non-string primitive collections are requested, or update the implementation to provide correct typed values in a non-ambiguous way.
| // Each element from _rawString.split(",") is always a string, so we only need to | |
| // decode the string values here. The result is effectively a string[]. | |
| const values = this._rawString.split(",").map((x) => this.getStringValueFromRaw(x)); | |
| return values as unknown as T[]; | |
| throw new Error("deserialization of primitive collections is not supported with URI encoding"); |
| it("getCollectionOfPrimitiveValues returns decoded string values", () => { | ||
| const result = new FormParseNode("one,two,three").getCollectionOfPrimitiveValues<string>(); | ||
| assert.deepEqual(result, ["one", "two", "three"]); | ||
| }); | ||
| it("getCollectionOfPrimitiveValues URL-decodes number-like string values", () => { | ||
| const result = new FormParseNode("1,2,3").getCollectionOfPrimitiveValues<string>(); | ||
| assert.deepEqual(result, ["1", "2", "3"]); | ||
| }); | ||
| it("getCollectionOfPrimitiveValues URL-decodes boolean-like string values", () => { | ||
| const result = new FormParseNode("true,false,true").getCollectionOfPrimitiveValues<string>(); | ||
| assert.deepEqual(result, ["true", "false", "true"]); | ||
| }); | ||
| it("getCollectionOfPrimitiveValues URL-decodes date-like string values", () => { | ||
| const result = new FormParseNode("2023-01-01,2023-06-15").getCollectionOfPrimitiveValues<string>(); | ||
| assert.deepEqual(result, ["2023-01-01", "2023-06-15"]); | ||
| }); |
There was a problem hiding this comment.
The PR description/checklist mentions adding unit tests for FormParseNode.getCollectionOfPrimitiveValues "with types other than string", but all new tests call getCollectionOfPrimitiveValues<string>() and assert string arrays. Either add tests that exercise non-string collection scenarios (if supported) or update the PR description/test wording to match the intended behavior (decoded strings only).
JsonParseNodeandFormParseNodegetXXXValueFromRawmethods to bothJsonParseNodeandFormParseNodegetXXXValuemethods to delegate to private methodsgetCollectionOfPrimitiveValuesto maintain triage conditions but call private methodsgetBooleanValueFromRawsince it's only used once ingetBooleanValuegetNumberValueFromRawsince it's only used once ingetNumberValueFormParseNode.getCollectionOfPrimitiveValueswith types other than stringOriginal prompt
📍 Connect Copilot coding agent with Jira, Azure Boards or Linear to delegate work to Copilot in one click without leaving your project management tool.