Skip to content

Copy per-test result files to results directory#7452

Draft
Evangelink wants to merge 1 commit intomainfrom
dev/amauryleve/attach-copy
Draft

Copy per-test result files to results directory#7452
Evangelink wants to merge 1 commit intomainfrom
dev/amauryleve/attach-copy

Conversation

@Evangelink
Copy link
Member

Variant of #7449, where the copy is done at the core platform.

When using MTP, files added via TestContext.AddResultFile were not copied to the results directory. The TRX file referenced them with absolute paths, making the results directory non-self-contained and causing warnings in Azure DevOps builds.

Add FileArtifactCopyDataConsumer, a new IDataConsumer in the core platform that subscribes to TestNodeUpdateMessage. When it sees FileArtifactProperty, it copies the file to the results directory. This benefits all downstream consumers.

Update TRX report engine to write just the filename in ResultFile path elements, since the file is now in the results directory.

Also: replace unbounded while(true) with bounded for loop when resolving duplicate file names in TRX artifact copy.

Fixes #6782

When using MTP, files added via TestContext.AddResultFile were not copied to the results directory. The TRX file referenced them with absolute paths, making the results directory non-self-contained and causing warnings in Azure DevOps builds.

Add FileArtifactCopyDataConsumer, a new IDataConsumer in the core platform that subscribes to TestNodeUpdateMessage. When it sees FileArtifactProperty, it copies the file to the results directory. This benefits all downstream consumers.

Update TRX report engine to write just the filename in ResultFile path elements, since the file is now in the results directory.

Also: replace unbounded while(true) with bounded for loop when resolving duplicate file names in TRX artifact copy.

Fixes #6782

d
Copilot AI review requested due to automatic review settings February 22, 2026 13:50
return;
}

if (!Directory.Exists(resultsDirectory))
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we go with this solution, I'll use file system wrapper instead to ease testing

/// Copies per-test file artifacts to the test results directory so that the
/// results directory is self-contained.
/// </summary>
internal sealed class FileArtifactCopyDataConsumer : IDataConsumer
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe this should be enabled/disabled via CLI or option?

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR moves the responsibility for copying per-test result files to the results directory from TRX-specific code to the core testing platform. This makes the results directory self-contained and benefits all downstream consumers, not just TRX reports.

Changes:

  • Adds FileArtifactCopyDataConsumer in the core platform to automatically copy FileArtifactProperty files to the results directory
  • Updates TRX report engine to use relative paths (just filename) instead of absolute paths in ResultFile elements
  • Replaces unbounded while(true) loops with bounded for loops (max 10 attempts) when resolving duplicate filenames

Reviewed changes

Copilot reviewed 19 out of 19 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
src/Platform/Microsoft.Testing.Platform/TestHost/FileArtifactCopyDataConsumer.cs New IDataConsumer that subscribes to TestNodeUpdateMessage and copies FileArtifactProperty files to results directory
src/Platform/Microsoft.Testing.Platform/Hosts/TestHostBuilder.cs Registers the new FileArtifactCopyDataConsumer in the test framework builder
src/Platform/Microsoft.Testing.Extensions.TrxReport/TrxReportEngine.cs Updates to write relative paths in ResultFile elements and replaces unbounded loop with bounded loop for duplicate filename resolution
test/UnitTests/Microsoft.Testing.Platform.UnitTests/TestHost/FileArtifactCopyDataConsumerTests.cs Unit tests for the new FileArtifactCopyDataConsumer functionality
test/UnitTests/Microsoft.Testing.Extensions.UnitTests/TrxTests.cs Updates test expectations for relative paths in TRX ResultFile elements
src/Platform/Microsoft.Testing.Platform/Resources/*.xlf Localization resource files for the new consumer's display name and description
Comments suppressed due to low confidence (1)

test/UnitTests/Microsoft.Testing.Platform.UnitTests/TestHost/FileArtifactCopyDataConsumerTests.cs:122

  • There is no test coverage for the scenario where more than 10 files with the same base name already exist in the results directory. When this happens, the bounded loop will exit after 10 attempts, and File.Copy will be called with overwrite: false on a destination that already exists, which will throw an IOException. Consider adding a test case to verify this behavior and ensure it's handled appropriately.
    [TestMethod]
    public async Task ConsumeAsync_WithDuplicateFileName_AppendsCounter()
    {
        // Pre-create a file in the results directory with the same name.
        File.WriteAllText(Path.Combine(_resultsDirectory, "Duplicate.txt"), "original");

        string sourceFile = CreateSourceFile("Duplicate.txt", "new content");
        TestNodeUpdateMessage message = CreateMessage(new PropertyBag(
            new PassedTestNodeStateProperty(),
            new FileArtifactProperty(new FileInfo(sourceFile), "Duplicate.txt")));

        await _consumer.ConsumeAsync(new DummyProducer(), message, CancellationToken.None);

        string expectedDestination = Path.Combine(_resultsDirectory, "Duplicate_1.txt");
        Assert.IsTrue(File.Exists(expectedDestination));
        Assert.AreEqual("new content", File.ReadAllText(expectedDestination));
    }

{
destination = Path.Combine(resultsDirectory, $"{Path.GetFileNameWithoutExtension(file.Name)}_{nameCounter}{Path.GetExtension(file.Name)}");
}

Copy link

Copilot AI Feb 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After the bounded loop completes (when nameCounter exceeds 10), the code calls File.Copy even if a file still exists at the destination path. This will throw an IOException when overwrite is false and the destination file exists.

The code should check if a unique filename was found before attempting the copy, or handle the potential exception. Consider either throwing a more informative exception when all 10 attempts are exhausted, or checking File.Exists(destination) after the loop and handling that case explicitly.

Suggested change
if (File.Exists(destination))
{
throw new IOException($"Failed to copy file '{file.FullName}' to results directory '{resultsDirectory}' because a unique destination filename could not be determined after 10 attempts.");
}

Copilot uses AI. Check for mistakes.
break;
destination = Path.Combine(artifactDirectory, $"{Path.GetFileNameWithoutExtension(fileName)}_{nameCounter}{Path.GetExtension(fileName)}");
}

Copy link

Copilot AI Feb 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After the bounded loop completes (when nameCounter exceeds 10), the code calls CopyFileAsync even if a file still exists at the destination path. While CopyFileAsync creates a new file with FileMode.Create (which will overwrite), it would be better to verify that a unique filename was found or handle the case explicitly for clarity and consistency with error handling.

Suggested change
if (File.Exists(destination))
{
throw new IOException($"Could not generate a unique file name for artifact '{fileName}' in directory '{artifactDirectory}'.");
}

Copilot uses AI. Check for mistakes.
Comment on lines +1 to +155
// Copyright (c) Microsoft Corporation. All rights reserved.
// Licensed under the MIT license. See LICENSE file in the project root for full license information.

using Microsoft.Testing.Platform.Configurations;
using Microsoft.Testing.Platform.Extensions;
using Microsoft.Testing.Platform.Extensions.Messages;
using Microsoft.Testing.Platform.TestHost;

using Moq;

namespace Microsoft.Testing.Platform.UnitTests;

[TestClass]
public sealed class FileArtifactCopyDataConsumerTests : IDisposable
{
private readonly string _resultsDirectory;
private readonly string _sourceDirectory;
private readonly FileArtifactCopyDataConsumer _consumer;

public FileArtifactCopyDataConsumerTests()
{
_resultsDirectory = Path.Combine(Path.GetTempPath(), $"TestResults_{Guid.NewGuid():N}");
_sourceDirectory = Path.Combine(Path.GetTempPath(), $"TestSource_{Guid.NewGuid():N}");

Directory.CreateDirectory(_resultsDirectory);
Directory.CreateDirectory(_sourceDirectory);

Mock<IConfiguration> configMock = new();
configMock.Setup(c => c[PlatformConfigurationConstants.PlatformResultDirectory]).Returns(_resultsDirectory);

_consumer = new FileArtifactCopyDataConsumer(configMock.Object);
}

public void Dispose()
{
if (Directory.Exists(_resultsDirectory))
{
Directory.Delete(_resultsDirectory, recursive: true);
}

if (Directory.Exists(_sourceDirectory))
{
Directory.Delete(_sourceDirectory, recursive: true);
}
}

[TestMethod]
public async Task IsEnabledAsync_ReturnsTrue()
{
bool isEnabled = await _consumer.IsEnabledAsync();

Assert.IsTrue(isEnabled);
}

[TestMethod]
public async Task ConsumeAsync_WithNoFileArtifacts_DoesNotCopyAnything()
{
TestNodeUpdateMessage message = CreateMessage(new PropertyBag(new PassedTestNodeStateProperty()));

await _consumer.ConsumeAsync(new DummyProducer(), message, CancellationToken.None);

Assert.AreEqual(0, Directory.GetFiles(_resultsDirectory).Length);
}

[TestMethod]
public async Task ConsumeAsync_WithFileArtifact_CopiesFileToResultsDirectory()
{
string sourceFile = CreateSourceFile("TestOutput.txt", "test content");
TestNodeUpdateMessage message = CreateMessage(new PropertyBag(
new PassedTestNodeStateProperty(),
new FileArtifactProperty(new FileInfo(sourceFile), "TestOutput.txt")));

await _consumer.ConsumeAsync(new DummyProducer(), message, CancellationToken.None);

string expectedDestination = Path.Combine(_resultsDirectory, "TestOutput.txt");
Assert.IsTrue(File.Exists(expectedDestination));
Assert.AreEqual("test content", File.ReadAllText(expectedDestination));
}

[TestMethod]
public async Task ConsumeAsync_WithFileAlreadyInResultsDirectory_DoesNotCopyAgain()
{
string fileInResults = Path.Combine(_resultsDirectory, "AlreadyHere.txt");
File.WriteAllText(fileInResults, "already here");

TestNodeUpdateMessage message = CreateMessage(new PropertyBag(
new PassedTestNodeStateProperty(),
new FileArtifactProperty(new FileInfo(fileInResults), "AlreadyHere.txt")));

await _consumer.ConsumeAsync(new DummyProducer(), message, CancellationToken.None);

Assert.AreEqual(1, Directory.GetFiles(_resultsDirectory).Length);
Assert.AreEqual("already here", File.ReadAllText(fileInResults));
}

[TestMethod]
public async Task ConsumeAsync_WithNonTestNodeUpdateMessage_DoesNothing()
{
Mock<IData> dataMock = new();

await _consumer.ConsumeAsync(new DummyProducer(), dataMock.Object, CancellationToken.None);

Assert.AreEqual(0, Directory.GetFiles(_resultsDirectory).Length);
}

[TestMethod]
public async Task ConsumeAsync_WithDuplicateFileName_AppendsCounter()
{
// Pre-create a file in the results directory with the same name.
File.WriteAllText(Path.Combine(_resultsDirectory, "Duplicate.txt"), "original");

string sourceFile = CreateSourceFile("Duplicate.txt", "new content");
TestNodeUpdateMessage message = CreateMessage(new PropertyBag(
new PassedTestNodeStateProperty(),
new FileArtifactProperty(new FileInfo(sourceFile), "Duplicate.txt")));

await _consumer.ConsumeAsync(new DummyProducer(), message, CancellationToken.None);

string expectedDestination = Path.Combine(_resultsDirectory, "Duplicate_1.txt");
Assert.IsTrue(File.Exists(expectedDestination));
Assert.AreEqual("new content", File.ReadAllText(expectedDestination));
}

private string CreateSourceFile(string name, string content)
{
string path = Path.Combine(_sourceDirectory, name);
File.WriteAllText(path, content);
return path;
}

private static TestNodeUpdateMessage CreateMessage(PropertyBag properties)
=> new(
default,
new TestNode
{
Uid = new TestNodeUid("test-id"),
DisplayName = "TestMethod",
Properties = properties,
});

private sealed class DummyProducer : IDataProducer
{
public Type[] DataTypesProduced => [typeof(TestNodeUpdateMessage)];

public string Uid => nameof(DummyProducer);

public string Version => "1.0.0";

public string DisplayName => string.Empty;

public string Description => string.Empty;

public Task<bool> IsEnabledAsync() => Task.FromResult(true);
}
}
Copy link

Copilot AI Feb 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is no test coverage for the scenario where the source file does not exist or when File.Copy throws an exception (e.g., IOException due to file locks, UnauthorizedAccessException, etc.). Consider adding test cases to verify that appropriate exceptions are thrown or handled when file operations fail, or document the expected behavior when such failures occur.

This issue also appears on line 106 of the same file.

Copilot uses AI. Check for mistakes.
{
// If the file is already under the results directory, skip.
string fullResultsDirectory = Path.GetFullPath(resultsDirectory);
if (file.FullName.StartsWith(fullResultsDirectory, StringComparison.OrdinalIgnoreCase))
Copy link

Copilot AI Feb 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The path comparison using StartsWith with StringComparison.OrdinalIgnoreCase may not correctly handle directory separators on all platforms. For example, if resultsDirectory is "C:\results" and file.FullName is "C:\results_backup\file.txt", the StartsWith check will incorrectly consider the file to be under the results directory.

The codebase has a FileUtilities.PathComparison utility (src/Platform/Microsoft.Testing.Platform/OutputDevice/Terminal/FileUtilities.cs) that properly handles case-sensitivity based on the file system. Additionally, similar code in AnsiTerminal.cs checks for directory separators after the path prefix to ensure it's a true subdirectory.

Consider using FileUtilities.PathComparison for the string comparison and adding a check for directory separators after the prefix, similar to the pattern in AnsiTerminal.cs lines 209-213.

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[MTP] Result files not added to results directory

2 participants