-
Notifications
You must be signed in to change notification settings - Fork 321
[DRAFT] Azure Split - Step 1 - Prep Work #3902
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
Make all of the common changes that aren't directly related to the Azure Split work, but set the stage for it and will make subsequent PRs much easier to consume.
- Added a few more changes.
- Fixed compilation issues.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR represents "Step 1" of an Azure component split initiative, focusing on common infrastructure changes needed to support future modularization. The changes primarily refactor the build system, update pipeline configurations, and improve code quality without introducing functional changes to the driver itself.
Changes:
- Refactored NuGet package generation by splitting monolithic targets into separate files for MDS, AKV, and SqlServer packages
- Renamed and reorganized build properties (e.g.,
NugetPackageVersion→mdsPackageVersion,AssemblyFileVersion→mdsAssemblyFileVersion) to support multiple package versioning - Converted pipeline template paths from relative to absolute for better maintainability
- Improved code quality with nullable annotations, property encapsulation, and defensive null checks
- Updated test infrastructure to support both Project and Package reference types more cleanly
Reviewed changes
Copilot reviewed 97 out of 97 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| tools/targets/add-ons/GenerateAkvPackage.targets | New target file for generating AKV NuGet packages with commit ID and version properties |
| tools/targets/add-ons/GenerateAKVProviderNugetPackage.targets | Removed (replaced by GenerateAkvPackage.targets) |
| tools/targets/GenerateSqlServerPackage.targets | New target file for generating SqlServer NuGet packages |
| tools/targets/GenerateMdsPackage.targets | New target file for generating MDS NuGet packages with proper property passing |
| tools/targets/GenerateNugetPackage.targets | Removed (functionality split into separate package-specific targets) |
| tools/targets/CopySniDllsForNetFxProjectReferenceBuilds.targets | New helper target to copy SNI DLLs for .NET Framework project reference builds |
| tools/specs/add-ons/Microsoft.Data.SqlClient.AlwaysEncrypted.AzureKeyVaultProvider.nuspec | Updated to use |
| tools/specs/Microsoft.SqlServer.Server.nuspec | Updated to use |
| tools/specs/Microsoft.Data.SqlClient.nuspec | Updated to use |
| tools/props/Versions.props | Reorganized version properties into logical groups (Common, SqlServer, MDS, AKV) with better documentation |
| src/Microsoft.SqlServer.Server/StringsHelper.cs | Added braces to single-statement if blocks for consistency |
| src/Microsoft.SqlServer.Server/Microsoft.SqlServer.Server.csproj | Changed DocumentationFile to use $(AssemblyName) variable |
| src/Microsoft.Data.SqlClient/tests/tools/Microsoft.Data.SqlClient.TestUtilities/xunit.runner.json | Updated for xUnit v2/v3 compatibility with comments and new configuration options |
| src/Microsoft.Data.SqlClient/tests/tools/Microsoft.Data.SqlClient.TestUtilities/config.default.json | Added WorkloadIdentityFederationServiceConnectionId property |
| src/Microsoft.Data.SqlClient/tests/tools/Microsoft.Data.SqlClient.TestUtilities/Utils.cs | Removed unused using statement |
| src/Microsoft.Data.SqlClient/tests/tools/Microsoft.Data.SqlClient.TestUtilities/Microsoft.Data.SqlClient.TestUtilities.csproj | Simplified to target only netstandard2.0, removed unused package references |
| src/Microsoft.Data.SqlClient/tests/tools/Microsoft.Data.SqlClient.TestUtilities/Config.cs | Added comment about environment variable override |
| src/Microsoft.Data.SqlClient/tests/StressTests/Directory.Packages.props | Updated MDS version references and imported Versions.props |
| src/Microsoft.Data.SqlClient/tests/StressTests/Directory.Build.props | Reduced .NET Framework test targets to net462 only |
| src/Microsoft.Data.SqlClient/tests/PerformanceTests/Microsoft.Data.SqlClient.PerformanceTests.csproj | Added conditional Project/Package reference support for MDS |
| src/Microsoft.Data.SqlClient/tests/ManualTests/Microsoft.Data.SqlClient.ManualTesting.Tests.csproj | Reorganized references and imported CopySniDllsForNetFxProjectReferenceBuilds.targets |
| src/Microsoft.Data.SqlClient/tests/FunctionalTests/app.config | Added comment for custom authentication provider |
| src/Microsoft.Data.SqlClient/tests/FunctionalTests/DataCommon/TestUtility.cs | Added IsNet, IsNetCore, IsNetFramework properties to replace old naming |
| src/Microsoft.Data.SqlClient/tests/FunctionalTests/DataCommon/DummySqlAuthenticationProvider.cs | Added braces to single-statement method |
| src/Microsoft.Data.SqlClient/tests/FunctionalTests/AssertExtensions.cs | Updated to use new TestUtility property names |
| src/Microsoft.Data.SqlClient/tests/Directory.Packages.props | Removed MDS package version dependency (moved to root) |
| src/Microsoft.Data.SqlClient/tests/Directory.Build.props | Removed ReferenceType default (moved to root Directory.Build.props) |
| src/Microsoft.Data.SqlClient/tests/CustomConfigurableRetryLogic/CustomRetryLogicProvider.csproj | Reorganized conditional references for clarity |
| src/Microsoft.Data.SqlClient/tests/Common/Common.csproj | Imported CopySniDllsForNetFxProjectReferenceBuilds.targets |
| src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/TdsParserHelperClasses.cs | Refactored SqlFedAuthInfo and SqlFedAuthToken to use properties and constructors instead of public fields |
| src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/TdsParser.cs | Updated to use new SqlFedAuthInfo constructor and properties |
| src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/SqlUtil.cs | Added conditional compilation for Windows-specific imports |
| src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/SqlException.cs | Improved null safety with defensive checks |
| src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/SqlErrorCollection.cs | Changed Add method to return SqlErrorCollection for chaining |
| src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/SqlDependency.cs | Added conditional compilation for Windows-specific imports |
| src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/SqlAuthenticationToken.cs | Removed internal helper methods (moved to SqlFedAuthToken) |
| src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/Diagnostics/SqlClientMetrics.cs | Added conditional compilation for Windows-specific imports |
| src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/ConnectionPool/DbConnectionPoolIdentity.cs | Fixed conditional compilation nesting |
| src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/Connection/SqlConnectionInternal.cs | Updated to use new SqlFedAuthToken constructor and properties |
| src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/ActiveDirectoryAuthenticationTimeoutRetryHelper.cs | Updated to use SqlFedAuthToken.AccessToken property |
| src/Microsoft.Data.SqlClient/src/Microsoft/Data/Common/AdapterUtil.netfx.cs | Added conditional compilation for Windows-specific imports |
| src/Microsoft.Data.SqlClient/src/Microsoft.Data.SqlClient.csproj | Added AssemblyName and reorganized TargetFrameworks properties |
| src/Microsoft.Data.SqlClient/netfx/src/Microsoft.Data.SqlClient.csproj | Changed DocumentationFile to use $(AssemblyName) |
| src/Microsoft.Data.SqlClient/netfx/ref/Microsoft.Data.SqlClient.csproj | Added AssemblyName and changed DocumentationFile |
| src/Microsoft.Data.SqlClient/netcore/src/Microsoft.Data.SqlClient.csproj | Reordered properties for consistency |
| src/Microsoft.Data.SqlClient/netcore/ref/Microsoft.Data.SqlClient.csproj | Added AssemblyName and changed DocumentationFile |
| src/Microsoft.Data.SqlClient/add-ons/Directory.Packages.props | Removed (functionality moved to root) |
| src/Microsoft.Data.SqlClient/add-ons/Directory.Build.props | Removed ReferenceType and AllowedOutputExtensionsInPackageBuildOutputFolder properties |
| src/Microsoft.Data.SqlClient/add-ons/AzureKeyVaultProvider/Microsoft.Data.SqlClient.AlwaysEncrypted.AzureKeyVaultProvider.csproj | Reorganized conditional references for clarity |
| src/Directory.Build.props | Updated ReferenceType documentation and default value handling |
| eng/pipelines/variables/akv-official-variables.yml | Renamed assemblyFileVersion to akvAssemblyFileVersion and nugetPackageVersion to akvPackageVersion |
| eng/pipelines/steps/roslyn-analyzers-akv-step.yml | Added akvPackageVersion parameter and documentation |
| eng/pipelines/steps/compound-nuget-pack-step.yml | Added properties parameter for additional nuget properties |
| eng/pipelines/steps/compound-build-akv-step.yml | Updated to use akvAssemblyFileVersion and akvPackageVersion parameters |
| eng/pipelines/stages/stress-tests-ci-stage.yml | Reorganized parameters and improved documentation |
| eng/pipelines/sqlclient-pr-project-ref-pipeline.yml | Changed buildType to referenceType |
| eng/pipelines/sqlclient-pr-package-ref-pipeline.yml | Changed buildType to referenceType |
| eng/pipelines/libraries/variables.yml | Updated template path to absolute |
| eng/pipelines/libraries/mds-validation-variables.yml | Updated template paths and variable names |
| eng/pipelines/libraries/common-variables.yml | Added assemblyBuildNumber calculation and renamed version variables |
| eng/pipelines/libraries/ci-build-variables.yml | Updated variable names to akvPackageVersion and mdsPackageVersion |
| eng/pipelines/libraries/build-variables.yml | Updated template paths to absolute |
| eng/pipelines/jobs/stress-tests-ci-job.yml | Fixed comment (stage → job) |
| eng/pipelines/jobs/build-akv-official-job.yml | Updated parameter names and moved Roslyn analysis before build |
| eng/pipelines/dotnet-sqlclient-signing-pipeline.yml | Updated template paths and variable names |
| eng/pipelines/dotnet-sqlclient-ci-project-reference-pipeline.yml | Added dotnetVerbosity parameter and renamed buildType to referenceType |
| eng/pipelines/dotnet-sqlclient-ci-package-reference-pipeline.yml | Added dotnetVerbosity parameter and renamed buildType to referenceType |
| eng/pipelines/dotnet-sqlclient-ci-core.yml | Renamed buildType to referenceType, added dotnetVerbosity, reorganized stages |
| eng/pipelines/common/templates/steps/* | Multiple template files updated with parameter renames, absolute paths, and improved formatting |
| eng/pipelines/common/templates/stages/ci-run-tests-stage.yml | Reorganized parameters and updated template paths |
| eng/pipelines/common/templates/jobs/* | Multiple job files updated with parameter renames and mdsPackageVersion/mdsArtifactName support |
| eng/pipelines/akv-official-pipeline.yml | Updated template path and variable names |
| build.proj | Major refactoring: reorganized imports, properties, targets, and dependencies |
| NuGet.config.local | New file for package reference builds with local packages/ directory |
| NuGet.config | Updated with better documentation and renamed feed key |
| Directory.Packages.props | Added conditional MDS/AKV package references and updated Azure.Identity version |
| BUILDGUIDE.md | Comprehensive update explaining ReferenceType property and build workflows |
| .github/workflows/codeql.yml | Added feat/* branches and explicit project builds |
| .editorconfig | Added slnx and proj file extensions |
- Fixed missing referenceType parameter.
- Fixed another missing referenceType.
- Fixed buildConfiguration parameter name.
- Removed non-existent projects from CodeQL Workflow.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
Copilot reviewed 98 out of 98 changed files in this pull request and generated no new comments.
|
|
||
| variables: | ||
| - template: ../../../libraries/variables.yml@self | ||
| - template: /eng/pipelines/libraries/variables.yml@self |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Making these paths absolute makes moving them around easier, and it allows the VS Code Azure Pipelines extension to validate them pre-commit.
| # See the LICENSE file in the project root for more information. # | ||
| ################################################################################# | ||
| parameters: | ||
| - name: configProperties |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I started sorting parameter lists alphabetically, but decided that would just add a mess. This file and a few others remain re-sorted since backing it out would also be a pain.
| - ${{ if parameters.isPreview }}: | ||
| - name: NugetPackageVersion | ||
| value: $(PreviewNugetPackageVersion) | ||
| - name: mdsPackageVersion |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
NugetPackageVersion was being used for MDS and AKV throughout the pipelines, which was ambiguous. Now we have obvious names for these parameters and variables.
| # These variables are sourced from common-variables.yml. | ||
| mdsAssemblyFileVersion: $(mdsAssemblyFileVersion) | ||
| mdsPackageVersion: $(mdsPackageVersion) | ||
| referenceType: Package |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Our official MDS build used to use project-references; now it uses package references.
| - name: mdsArtifactName | ||
| type: string | ||
| default: Artifacts | ||
| default: MDS.Artifact |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We're going to have several groups of artifacts, some of which we will want to download individually. The new name here makes it clear what package the artifacts are for.
| { | ||
| get => _federatedAuthenticationInfoRequested && | ||
| DateTime.FromFileTimeUtc(_fedAuthToken.expirationFileTime) < DateTime.UtcNow.AddSeconds(accessTokenExpirationBufferTime); | ||
| DateTime.FromFileTimeUtc(_fedAuthToken.ExpirationFileTime) < DateTime.UtcNow.AddSeconds(accessTokenExpirationBufferTime); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I tidied up the FedAuthToken and FedAuthInfo classes, and this is the fallout.
| using System.Runtime.Versioning; | ||
| #endif | ||
|
|
||
| #if _WINDOWS |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This was unnecessarily restrictive.
|
|
||
| // Here we mimic how ADAL calculates hash for token. They use UTF8 instead of Unicode. | ||
| var originalTokenString = SqlAuthenticationToken.AccessTokenStringFromBytes(token.accessToken); | ||
| var originalTokenString = Encoding.Unicode.GetString(token.AccessToken); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
AccesTokenStringFromBytes() was a one-liner that did exactly this.
| </ItemGroup> | ||
|
|
||
| <!-- Ensure SNI DLLs are made available for .NET Framework project reference builds. --> | ||
| <Import Project="$(ToolsDir)/targets/CopySniDllsForNetFxProjectReferenceBuilds.targets" /> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I realize that @benrr101 has a different approach/solution for this problem. I think the root cause has something to do with the MDS projects' output directories being incompatible with the targets included in the SNI NuGet package. We will find a proper solution separate from this work.
| See the BUILDGUIDE.md for more details. | ||
| --> | ||
| <ReferenceType Condition="$(ReferenceType) == ''">Project</ReferenceType> | ||
| <AllowedOutputExtensionsInPackageBuildOutputFolder>$(AllowedOutputExtensionsInPackageBuildOutputFolder);.pdb</AllowedOutputExtensionsInPackageBuildOutputFolder> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We don't want PDBs included in our NuGet packages. Developers can use the symbols packages that are generated beside the NuGet packages.
- Tweaks after reviewing the PR.
- Fixed Windows .NET compilation issue.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
Copilot reviewed 98 out of 98 changed files in this pull request and generated no new comments.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We're no longer editing the NuGet.config file. We are copying the NuGet.config.local file in its place whenever we need to source packages from the packages/ directory.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This file wasn't really adding any value. Its content was moved into the calling templates where appropriate.
- Fixed packages/ dir creation.
- Fixed missing MDS package version during Package builds.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
Copilot reviewed 98 out of 98 changed files in this pull request and generated no new comments.
- Generating MDS NuGet package for subsequent AKV build in Package mode.
- Running Release build for Debug pipelines in Project mode to avoid NuGet packaging issues.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
Copilot reviewed 98 out of 98 changed files in this pull request and generated no new comments.
- Restoring default mdsPackageVersion for Project based runs.
- Restoring inhibition of NuGet package generation.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
Copilot reviewed 98 out of 98 changed files in this pull request and generated no new comments.
Description
This PR contains a variety of prep work that sets the stage for the addition of the Abstractions and Azure packages:
I have added commentary around the interesting changes. This is probably the largest and most complex PR of this series.
Testing