Skip to content

Merge | Ref Projects#3963

Open
benrr101 wants to merge 23 commits intomainfrom
dev/russellben/common/ref
Open

Merge | Ref Projects#3963
benrr101 wants to merge 23 commits intomainfrom
dev/russellben/common/ref

Conversation

@benrr101
Copy link
Contributor

Description

As much as I want to remove the ref projects, they are tightly coupled to the netstandard binaries and the AnyOS "not supported" binaries. Detangling those will be a headache that we don't need to do right now. So instead, let's at least merge the ref projects into a common ref project just like we did for MDS projects.

The files were broken up differently from netcore, netfx, and netstandard, so I opted to break them down into one file per namespace. This made merging easier, and will likely make maintenance easier in the future.

No changes to the builds were made. Projects were updated to point to the merged files, and a common ref project was added. The solution file was modified slightly to make it clearer the difference between common ref project and common mds project.

This should unblock packing the common project.

Testing

Local builds of the ref projects still works, I presume they will continue to work in CI builds.

@benrr101 benrr101 added this to the 7.0.0-preview4 milestone Feb 16, 2026
@benrr101 benrr101 added the Common Project 🚮 Things that relate to the common project project label Feb 16, 2026
Copilot AI review requested due to automatic review settings February 16, 2026 19:11
@benrr101 benrr101 requested a review from a team as a code owner February 16, 2026 19:11
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR consolidates the various reference-assembly source files into a shared src/Microsoft.Data.SqlClient/ref/ set (organized roughly by namespace), updates the existing netcore/ref and netfx/ref projects to compile those shared files, and adjusts the solution layout to include the new common ref project—aiming to unblock packing the common project without removing ref projects.

Changes:

  • Added a new common ref project (src/Microsoft.Data.SqlClient/ref/Microsoft.Data.SqlClient.csproj) and split ref sources into multiple shared files under src/Microsoft.Data.SqlClient/ref/.
  • Updated src/Microsoft.Data.SqlClient/netcore/ref and src/Microsoft.Data.SqlClient/netfx/ref csproj files to compile the shared ref sources from ../../ref/.
  • Updated the solution to add a ref solution folder and include the new common ref project (and re-add the common src project).

Reviewed changes

Copilot reviewed 13 out of 16 changed files in this pull request and generated 21 comments.

Show a summary per file
File Description
src/Microsoft.Data.SqlClient/ref/Microsoft.Data.cs Adds ref stubs for Microsoft.Data surface (currently has namespace mismatch issues).
src/Microsoft.Data.SqlClient/ref/Microsoft.Data.Sql.cs Adds ref stubs for Microsoft.Data.Sql surface (contains an auto-property stub issue).
src/Microsoft.Data.SqlClient/ref/Microsoft.Data.SqlClient.csproj New common multi-targeted ref project for shared ref sources.
src/Microsoft.Data.SqlClient/ref/Microsoft.Data.SqlClient.Server.cs Adds server namespace ref stubs (contains incorrect <include> pathing).
src/Microsoft.Data.SqlClient/ref/Microsoft.Data.SqlClient.Diagnostics.cs Adds diagnostics ref stubs (contains namespace mismatch + malformed XML doc comments + auto-property stub issue).
src/Microsoft.Data.SqlClient/ref/Microsoft.Data.SqlClient.DataClassification.cs Adds DataClassification ref stubs.
src/Microsoft.Data.SqlClient/ref/Microsoft.Data.SqlTypes.cs Adds SqlTypes ref stubs (SqlJson/SqlVector/SqlFileStream).
src/Microsoft.Data.SqlClient/ref/Microsoft.Data.SqlClient.Batch.cs Removes old batch ref file (now expected to be covered by merged ref sources).
src/Microsoft.Data.SqlClient/ref/Microsoft.Data.SqlClient.Batch.NetCoreApp.cs Removes netcoreapp-only batch ref file (now expected to be covered by merged ref sources).
src/Microsoft.Data.SqlClient/netfx/ref/Microsoft.Data.SqlClient.csproj Points netfx ref project at shared ../../ref/* files (currently missing Diagnostics ref file).
src/Microsoft.Data.SqlClient/netcore/ref/Microsoft.Data.SqlClient.csproj Points netcore ref project at shared ../../ref/* files (currently missing Diagnostics ref file).
src/Microsoft.Data.SqlClient/netcore/ref/Microsoft.Data.SqlClient.Manual.cs Removes old manual ref file (expected to be covered by merged ref sources).
src/Microsoft.Data.SqlClient.sln Adds a ref folder and the new common ref project; adjusts project entries (project type GUID inconsistency).

Copy link
Contributor

@paulmedynski paulmedynski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have questions about internal/private items being declared, and a lack of access modifiers.

I'm also wondering if it's worth adding a check to the pipelines to ensure that the APIs declared in the ref project are identical to those declared in the implementation. Or perhaps a build target to catch problems pre-commit. It's critical that the APIs are identical, and we've released packages with discrepancies before.

#endif
public sealed class OperationAbortedException : System.SystemException
{
internal OperationAbortedException() { }
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why would the ref APIs declare any internal or private items? Same question across other files here as well.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have no idea!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As per the magic matricies:

Yes, there is a valid reason in this case.

src/Microsoft.Data.SqlClient/ref/Microsoft.Data.cs:13 includes a non-public constructor so the compiler does not synthesize a public parameterless .ctor for OperationAbortedException. If that constructor were absent in the ref stub, the type would accidentally appear publicly constructible in the reference assembly, which would be an API contract bug.

So while ref files are for public API contract, they sometimes include non-public members to preserve correct type shape semantics. This pattern shows up in other ref types too (for example SqlException in src/Microsoft.Data.SqlClient/ref/Microsoft.Data.SqlClient.cs:1642).

Short version: those members are not there to expose internal API, they are there to prevent exposing the wrong public API.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds like a great code comment so we don't ask this question again in a few months!

@paulmedynski paulmedynski self-assigned this Feb 17, 2026
Copy link
Contributor

@mdaigle mdaigle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would also like to find a way to more easily compare before and after. Maybe we can leverage this? https://learn.microsoft.com/en-us/dotnet/standard/assembly/inspect-contents-using-metadataloadcontext

@edwardneal
Copy link
Contributor

I think there's already tooling to compare ref assemblies to implementation assemblies - the ApiCompat packages may be useful here.

There are also other similar tools located here (although netstandard2.0 ref assemblies make GenAPI very difficult.) An earlier version of GenAPI is already in the SqlClient source tree.

Copilot AI review requested due to automatic review settings February 18, 2026 23:19
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 17 out of 20 changed files in this pull request and generated 5 comments.

Copy link
Contributor

@paulmedynski paulmedynski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will be an interesting experiment to see how well the AI can be instructed to produce precisely the code that we want.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm going to review whatever the AI/agents create from this file, rather than this file itself.

<!-- ================================================================== -->
<!-- _SetApiCompatProperties -->
<!-- -->
<!-- All path properties are set inside a target so that they are -->
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Extra padding at the end of these 3 lines.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This target is specific to the MDS package, so maybe CompareMdsRefAssemblies.targets would be a better name.

Build target that compares locally-built ref assemblies against those published
in a specified NuGet package version of Microsoft.Data.SqlClient. Uses
Microsoft.DotNet.ApiCompat.Tool in strict mode to detect any API surface
differences (additions or removals).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does that include changes, like function arguments whose order has changed?

Copy link
Contributor

@mdaigle mdaigle Feb 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it does. But interestingly, it doesn't care about "source breaking" changes: https://learn.microsoft.com/en-us/dotnet/standard/library-guidance/breaking-changes#source-breaking-change

So, for example, it won't error if you change an optional parameter to required.

differences (additions or removals).
Usage:
dotnet msbuild build.proj /t:CompareRefAssemblies /p:BaselinePackageVersion=6.1.4
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dotnet build is the typical way to build. We're not using any special msbuild features here.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will make some of these changes, but don't want to invest too much time into polishing this. Once we have our pack process improved, we'll be able to use the EnablePackageValidation property in our csproj file to get this for free: https://learn.microsoft.com/en-us/dotnet/fundamentals/apicompat/package-validation/baseline-version-validator

Text="Baseline package $(BaselinePackageVersion) already extracted at $(BaselineExtractDir). Skipping download." />

<MakeDir
Condition="!Exists('$(BaselineExtractDir)ref\')"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we avoid repeating this condition by putting these actions in an <ItemGroup> ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would need to separate them out into another target. ItemGroups can't contain tasks.


<Message
Importance="high"
Text="Baseline ref assemblies available at $(BaselineExtractDir)ref\" />
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a duplicate message when the baseline package already exists.

<!-- _RestoreApiCompatTool -->
<!-- ================================================================== -->
<Target Name="_RestoreApiCompatTool" DependsOnTargets="_SetApiCompatProperties">
<Exec Command="dotnet tool restore" WorkingDirectory="$(RepoRoot)" />
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This restores all of the tools, which contradicts the name of the target.

<Message Importance="high" Text="Building legacy netfx ref project (net462)..." />
<MSBuild
Projects="$(LegacyNetFxRefProject)"
Targets="restore"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why separate out the Restore target? You can specify multiple targets here:

Targets="Restore;Build"

It's fine to pass more properties to the Restore target than it needs.

<!-- ================================================================== -->
<Target Name="_BuildNewRefProject" DependsOnTargets="_SetApiCompatProperties">
<Message Importance="high" Text="Building new consolidated ref project (all TFMs)..." />
<Exec Command="dotnet restore &quot;$(NewRefProjectPath)&quot;" />
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are we using <Exec> here instead of <MSBuild> ?

Copilot AI review requested due to automatic review settings February 19, 2026 22:03
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 17 out of 20 changed files in this pull request and generated 2 comments.

Comment on lines +12 to +114
dotnet msbuild build.proj /t:CompareRefAssemblies /p:BaselinePackageVersion=6.1.4
```

- `BaselinePackageVersion` is **required** (no default). The user must specify which published package to compare against.
- Both the **legacy** ref projects (`netcore/ref/`, `netfx/ref/`) and the **new consolidated** ref project (`ref/`) are built and compared independently, so you can distinguish whether a difference comes from source consolidation vs. project restructuring.
- All comparisons run to completion (errors don't short-circuit), so you see all differences at once.

## Implementation Steps

### 1. Register `Microsoft.DotNet.ApiCompat.Tool` in `dotnet-tools.json`

Add an entry for version `9.0.200` alongside the existing `dotnet-coverage` entry. This enables `dotnet apicompat` after `dotnet tool restore`.

### 2. Create `tools/targets/CompareRefAssemblies.targets`

A single new file containing all properties, items, and targets (steps 3–11 below). Follows the naming convention of existing files like `GenerateMdsPackage.targets`.

### 3. Define properties

| Property | Value | Notes |
|----------|-------|-------|
| `BaselinePackageVersion` | *(none)* | Required; validated by guard target |
| `BaselinePackageDir` | `$(Artifacts)apicompat\` | Working directory for downloads |
| `BaselineNupkgPath` | `$(BaselinePackageDir)microsoft.data.sqlclient.$(BaselinePackageVersion).nupkg` | Downloaded nupkg path |
| `BaselinePackageUrl` | `https://api.nuget.org/v3-flatcontainer/microsoft.data.sqlclient/$(BaselinePackageVersion)/microsoft.data.sqlclient.$(BaselinePackageVersion).nupkg` | NuGet flat container URL |
| `BaselineExtractDir` | `$(BaselinePackageDir)extracted\$(BaselinePackageVersion)\` | Extraction destination |
| `NewRefProjectPath` | `$(ManagedSourceCode)ref\Microsoft.Data.SqlClient.csproj` | New consolidated ref project |
| `NewRefOutputDir` | `$(ManagedSourceCode)ref\bin\$(Configuration)\` | SDK default output for new ref project |
| `LegacyNetFxRefDir` | `$(Artifacts)Project\bin\Windows_NT\$(Configuration)\Microsoft.Data.SqlClient\ref\` | Legacy netfx ref output |
| `LegacyNetCoreRefDir` | `$(Artifacts)Project\bin\AnyOS\$(Configuration)\Microsoft.Data.SqlClient\ref\` | Legacy netcore ref output |

### 4. `_ValidateBaselineVersion` target

Emits `<Error>` if `$(BaselinePackageVersion)` is empty, with a message instructing the user to pass `/p:BaselinePackageVersion=X.Y.Z`.

### 5. `_DownloadBaselinePackage` target

- Depends on `_ValidateBaselineVersion`
- Uses `<DownloadFile>` to fetch the nupkg from nuget.org
- Uses `<Unzip>` to extract it into `$(BaselineExtractDir)`
- Conditioned to skip if `$(BaselineExtractDir)` already exists

### 6. `_RestoreApiCompatTool` target

Runs `<Exec Command="dotnet tool restore" WorkingDirectory="$(RepoRoot)" />`.

### 7. `_BuildLegacyRefNetFx` target

- Depends on `RestoreNetFx`
- Builds `$(NetFxSource)ref\Microsoft.Data.SqlClient.csproj` for `net462`
- Conditioned on Windows (`$(IsEnabledWindows)`)
- Output: `$(LegacyNetFxRefDir)net462\Microsoft.Data.SqlClient.dll`

### 8. `_BuildLegacyRefNetCore` target

- Depends on `RestoreNetCore`
- Builds `$(NetCoreSource)ref\Microsoft.Data.SqlClient.csproj` with `/p:OSGroup=AnyOS`
- Output: `$(LegacyNetCoreRefDir){net8.0,net9.0,netstandard2.0}\Microsoft.Data.SqlClient.dll`

### 9. `_BuildNewRefProject` target

- Runs `dotnet build` on `$(NewRefProjectPath)`
- Builds all 4 TFMs (`net462`, `net8.0`, `net9.0`, `netstandard2.0`) in one shot
- Output: `$(NewRefOutputDir){tfm}\Microsoft.Data.SqlClient.dll`

### 10. `_RunRefApiCompat` target

- Depends on `_DownloadBaselinePackage;_RestoreApiCompatTool;_BuildLegacyRefNetFx;_BuildLegacyRefNetCore;_BuildNewRefProject`
- Iterates over 4 TFMs (`net462`, `net8.0`, `net9.0`, `netstandard2.0`) using item batching
- For each TFM, runs two `<Exec>` calls with `ContinueOnError="ErrorAndContinue"`:
- **Legacy vs baseline**: `dotnet apicompat -l {baseline-ref-dll} -r {legacy-ref-dll} --strict-mode`
- **New vs baseline**: `dotnet apicompat -l {baseline-ref-dll} -r {new-ref-dll} --strict-mode`
- Each `<Exec>` is conditioned on `Exists(...)` for both DLLs (gracefully skips missing TFMs)
- Preceded by `<Message Importance="high">` labelling each comparison
- Uses item metadata to map `net462` to `$(LegacyNetFxRefDir)` and others to `$(LegacyNetCoreRefDir)`

### 11. `CompareRefAssemblies` target (public entry point)

- Declared with `DependsOnTargets="_RunRefApiCompat"`
- Emits a final `<Message>` summarizing completion

### 12. Import into `build.proj`

Add one line after the existing `.targets` imports (after line 7):

```xml
<Import Project="$(ToolsDir)targets\CompareRefAssemblies.targets" />
```

## Design Decisions

- **Single new file** at `tools/targets/CompareRefAssemblies.targets` — only one `<Import>` line added to `build.proj`.
- **Internal targets prefixed with `_`** to signal they're not intended to be called directly.
- **Strict mode** ensures API additions are also flagged — important for detecting accidental public surface changes during file reorganization.
- **`ContinueOnError="ErrorAndContinue"`** on each apicompat `Exec` so all 8 comparisons run and all differences are reported together.
- **`_BuildLegacyRefNetFx`** is conditioned on Windows (net462 can't build on Unix), matching the existing `BuildNetFx` pattern.
- **No default baseline version** — the user must always explicitly specify which published package to compare against.
- **Download caching** — re-runs skip the download if the extracted directory already exists.

## Verification

```
dotnet msbuild build.proj /t:CompareRefAssemblies /p:BaselinePackageVersion=6.1.4
Copy link

Copilot AI Feb 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The documentation plan states the target name should be CompareRefAssemblies (lines 12, 25, 88, 98, 114), but the actual implementation uses CompareMdsRefAssemblies (line 206 in the targets file and line 8 in build.proj). This creates confusion for users following the documentation. Update the plan document to use CompareMdsRefAssemblies consistently, matching the actual implementation.

Copilot uses AI. Check for mistakes.
Comment on lines +25 to +103
### 2. Create `tools/targets/CompareRefAssemblies.targets`

A single new file containing all properties, items, and targets (steps 3–11 below). Follows the naming convention of existing files like `GenerateMdsPackage.targets`.

### 3. Define properties

| Property | Value | Notes |
|----------|-------|-------|
| `BaselinePackageVersion` | *(none)* | Required; validated by guard target |
| `BaselinePackageDir` | `$(Artifacts)apicompat\` | Working directory for downloads |
| `BaselineNupkgPath` | `$(BaselinePackageDir)microsoft.data.sqlclient.$(BaselinePackageVersion).nupkg` | Downloaded nupkg path |
| `BaselinePackageUrl` | `https://api.nuget.org/v3-flatcontainer/microsoft.data.sqlclient/$(BaselinePackageVersion)/microsoft.data.sqlclient.$(BaselinePackageVersion).nupkg` | NuGet flat container URL |
| `BaselineExtractDir` | `$(BaselinePackageDir)extracted\$(BaselinePackageVersion)\` | Extraction destination |
| `NewRefProjectPath` | `$(ManagedSourceCode)ref\Microsoft.Data.SqlClient.csproj` | New consolidated ref project |
| `NewRefOutputDir` | `$(ManagedSourceCode)ref\bin\$(Configuration)\` | SDK default output for new ref project |
| `LegacyNetFxRefDir` | `$(Artifacts)Project\bin\Windows_NT\$(Configuration)\Microsoft.Data.SqlClient\ref\` | Legacy netfx ref output |
| `LegacyNetCoreRefDir` | `$(Artifacts)Project\bin\AnyOS\$(Configuration)\Microsoft.Data.SqlClient\ref\` | Legacy netcore ref output |

### 4. `_ValidateBaselineVersion` target

Emits `<Error>` if `$(BaselinePackageVersion)` is empty, with a message instructing the user to pass `/p:BaselinePackageVersion=X.Y.Z`.

### 5. `_DownloadBaselinePackage` target

- Depends on `_ValidateBaselineVersion`
- Uses `<DownloadFile>` to fetch the nupkg from nuget.org
- Uses `<Unzip>` to extract it into `$(BaselineExtractDir)`
- Conditioned to skip if `$(BaselineExtractDir)` already exists

### 6. `_RestoreApiCompatTool` target

Runs `<Exec Command="dotnet tool restore" WorkingDirectory="$(RepoRoot)" />`.

### 7. `_BuildLegacyRefNetFx` target

- Depends on `RestoreNetFx`
- Builds `$(NetFxSource)ref\Microsoft.Data.SqlClient.csproj` for `net462`
- Conditioned on Windows (`$(IsEnabledWindows)`)
- Output: `$(LegacyNetFxRefDir)net462\Microsoft.Data.SqlClient.dll`

### 8. `_BuildLegacyRefNetCore` target

- Depends on `RestoreNetCore`
- Builds `$(NetCoreSource)ref\Microsoft.Data.SqlClient.csproj` with `/p:OSGroup=AnyOS`
- Output: `$(LegacyNetCoreRefDir){net8.0,net9.0,netstandard2.0}\Microsoft.Data.SqlClient.dll`

### 9. `_BuildNewRefProject` target

- Runs `dotnet build` on `$(NewRefProjectPath)`
- Builds all 4 TFMs (`net462`, `net8.0`, `net9.0`, `netstandard2.0`) in one shot
- Output: `$(NewRefOutputDir){tfm}\Microsoft.Data.SqlClient.dll`

### 10. `_RunRefApiCompat` target

- Depends on `_DownloadBaselinePackage;_RestoreApiCompatTool;_BuildLegacyRefNetFx;_BuildLegacyRefNetCore;_BuildNewRefProject`
- Iterates over 4 TFMs (`net462`, `net8.0`, `net9.0`, `netstandard2.0`) using item batching
- For each TFM, runs two `<Exec>` calls with `ContinueOnError="ErrorAndContinue"`:
- **Legacy vs baseline**: `dotnet apicompat -l {baseline-ref-dll} -r {legacy-ref-dll} --strict-mode`
- **New vs baseline**: `dotnet apicompat -l {baseline-ref-dll} -r {new-ref-dll} --strict-mode`
- Each `<Exec>` is conditioned on `Exists(...)` for both DLLs (gracefully skips missing TFMs)
- Preceded by `<Message Importance="high">` labelling each comparison
- Uses item metadata to map `net462` to `$(LegacyNetFxRefDir)` and others to `$(LegacyNetCoreRefDir)`

### 11. `CompareRefAssemblies` target (public entry point)

- Declared with `DependsOnTargets="_RunRefApiCompat"`
- Emits a final `<Message>` summarizing completion

### 12. Import into `build.proj`

Add one line after the existing `.targets` imports (after line 7):

```xml
<Import Project="$(ToolsDir)targets\CompareRefAssemblies.targets" />
```

## Design Decisions

- **Single new file** at `tools/targets/CompareRefAssemblies.targets` — only one `<Import>` line added to `build.proj`.
Copy link

Copilot AI Feb 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The documentation plan states the filename should be CompareRefAssemblies.targets (line 25, 98, 103), but the actual filename is CompareMdsRefAssemblies.targets. Update the plan to match the actual implementation.

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Common Project 🚮 Things that relate to the common project project

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

Comments