Skip to content

Conversation

@Roshan-R
Copy link

Update the stream extraction logic to support the com.coreos.stream label in the OCI image configuration metadata.

Previously, the stream was determined using the fedora-coreos.stream annotation. While that annotation is still supported as a fallback, the preferred method is now to use the new com.coreos.stream label. This change aligns with the new Konflux-built images, which include only this label and no longer provide the older annotation.

See: coreos/fedora-coreos-config#3867

Copy link

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the stream extraction logic to support the com.coreos.stream OCI label, with a fallback to the older fedora-coreos.stream annotation. The changes look good and correctly implement the desired logic. I've suggested a small refactoring to the fedora_coreos_stream_from_deployment function to improve its clarity and error reporting by removing a redundant check and making the final error message more comprehensive.

Comment on lines 152 to 199
if deploy.base_commit_meta.stream.is_none()
&& deploy.base_commit_meta.oci_image_configuration.is_none()
&& deploy.base_commit_meta.oci_manifest.is_none()
{
bail!("Cannot deserialize ostree base image manifest");
}

if let Some(stream) = deploy.base_commit_meta.stream.as_ref() {
if !stream.is_empty() {
return Ok(stream.clone());
}
}

// Check for `com.coreos.stream` label in OCI ImageConfiguration
if let Some(oci_image_configuration) = deploy.base_commit_meta.oci_image_configuration.as_ref()
{
let image_configuration: ostree_ext::oci_spec::image::ImageConfiguration =
serde_json::from_str(oci_image_configuration.as_str())?;
if let Some(stream) = image_configuration.config().as_ref().and_then(|cfg| {
cfg.labels()
.as_ref()
.and_then(|labels| labels.get("com.coreos.stream"))
}) {
if !stream.is_empty() {
return Ok(stream.clone());
}
}
}

// Fallback to `fedora-coreos.stream` label in OCI ImageManifest
if let Some(oci_manifest) = deploy.base_commit_meta.oci_manifest.as_ref() {
let manifest: ostree_ext::oci_spec::image::ImageManifest =
serde_json::from_str(oci_manifest.as_str())?;
if let Some(stream) = manifest
.annotations()
.as_ref()
.and_then(|a| a.get("fedora-coreos.stream"))
{
if !stream.is_empty() {
return Ok(stream.clone());
}
}
(None, None) => bail!("Cannot deserialize ostree base image manifest"),
};
ensure!(!stream.is_empty(), "empty stream value");
Ok(stream)
}

return Err(anyhow!(
"Missing `com.coreos.stream` label in the base image configuration,
or `fedora-coreos.stream` annotation in the base image manifest"
));

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

This function can be simplified for better readability and more helpful error reporting.

  1. The initial check for all sources being None is redundant. If all optional fields are None, the function will correctly fall through to the final error. Removing this check simplifies the code.
  2. The final error message is incomplete. It should mention all three possible sources of the stream information (base_commit_meta.stream, oci_image_configuration, and oci_manifest) to be more helpful for debugging.

Here is a suggested refactoring that addresses these points:

    if let Some(stream) = deploy.base_commit_meta.stream.as_ref() {
        if !stream.is_empty() {
            return Ok(stream.clone());
        }
    }

    // Check for `com.coreos.stream` label in OCI ImageConfiguration
    if let Some(oci_image_configuration) = deploy.base_commit_meta.oci_image_configuration.as_ref()
    {
        let image_configuration: ostree_ext::oci_spec::image::ImageConfiguration =
            serde_json::from_str(oci_image_configuration.as_str())?;
        if let Some(stream) = image_configuration.config().as_ref().and_then(|cfg| {
            cfg.labels()
                .as_ref()
                .and_then(|labels| labels.get("com.coreos.stream"))
        }) {
            if !stream.is_empty() {
                return Ok(stream.clone());
            }
        }
    }

    // Fallback to `fedora-coreos.stream` label in OCI ImageManifest
    if let Some(oci_manifest) = deploy.base_commit_meta.oci_manifest.as_ref() {
        let manifest: ostree_ext::oci_spec::image::ImageManifest =
            serde_json::from_str(oci_manifest.as_str())?;
        if let Some(stream) = manifest
            .annotations()
            .as_ref()
            .and_then(|a| a.get("fedora-coreos.stream"))
        {
            if !stream.is_empty() {
                return Ok(stream.clone());
            }
        }
    }

    Err(anyhow!(
        "Missing or empty stream from all sources: `fedora-coreos.stream` in commit meta, `com.coreos.stream` in OCI config, or `fedora-coreos.stream` in OCI manifest"
    ))

#[derive(Clone, Debug, Deserialize)]
struct BaseCommitMeta {
#[serde(rename = "fedora-coreos.stream")]
stream: Option<String>,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was the old old fallback. We should probably drop it now and stop considering it in the code below.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed the old fallback code but had to copy the ostree.container.image-config from rpm-ostree-oci-status.json fixture to the rpm-ostree-status.json fixture for tests to pass correctly

Copy link
Member

@jbtrystram jbtrystram Nov 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As I wrote in my review below this is effectively breaking support for non OCI deployements I think.
I am OK to do this but this should be done in the other relevant places too, and probably in a separate release

Comment on lines 159 to 163
if let Some(stream) = deploy.base_commit_meta.stream.as_ref() {
if !stream.is_empty() {
return Ok(stream.clone());
}
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

delete - see comment above

@Roshan-R Roshan-R force-pushed the fix/replace-stream-annotation-with-label branch 4 times, most recently from cfd03d6 to 9744cd5 Compare November 13, 2025 08:05
@dustymabe
Copy link
Member

Thanks - the code is looking good. I would like to test it myself if I can get a chance.

Let's also delete tests/fixtures/rpm-ostree-oci-status.json and just update tests/fixtures/rpm-ostree-status.json with the latest rpm-ostree status --json from latest stable F43 FCOS.

The reason we had both is we were trying to test both cases (from the old fallback).

Or actually I guess we should pivot and still have two:

  • rpm-ostree-status.json should include the com.coreos.stream label
  • rpm-ostree-status-annotation.json should test the fallback annotation case (i.e. you can just use rpm-ostree status --json from latest stable F43 FCOS for this)

@Roshan-R Roshan-R force-pushed the fix/replace-stream-annotation-with-label branch from 9744cd5 to bc47465 Compare November 19, 2025 09:32
@Roshan-R
Copy link
Author

The tests have been updated

Copy link
Member

@jbtrystram jbtrystram left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks generally sane to me, i still want to test it manually.
One thing this does is breaking support for encapsulated ostree deployments built with COSA.
This is probably OK since we won't ship newer Zincati in old-style images but that's something we want to think about, and if we drop support for those we should do it accross the whole codebase I think.

@Roshan-R
Copy link
Author

Roshan-R commented Dec 2, 2025

steps to test the PR. I have uploaded the modified status.json with the fedora-coreos.stream annotation removed and the com.coreos.stream label added

# buildfetch fcos that can be updated using zincati
cosa buildfetch --build=42.20251012.3.0 --artifact qemu

# cosa run with zincati auto updates disabled
cosa run --qemu-image builds/42.20251012.3.0/x86_64/fedora-coreos-42.20251012.3.0-qemu.x86_64.qcow2 --add-ignition noautoupdate

# Become root user
[core@cosa-devsh ~]$ sudo su

# Stop the zincati service
[root@cosa-devsh core]# systemctl stop zincati

# Create a usroverlay to change the zincati binary
[root@cosa-devsh core]# rpm-ostree usroverlay 
Development mode enabled.  A writable overlayfs is now mounted on /usr.
All changes there will be discarded on reboot.

# Replace the zincati binary with the one generated from the PR 
[root@cosa-devsh core]# cp /mnt/workdir-tmp/zincati /usr/libexec/zincati 

# Add Override to systemd zincati service file to change the log level
[root@cosa-devsh core]# mkdir /etc/systemd/system/zincati.service.d
[root@cosa-devsh core]# echo "[Service]
ExecStart= 
ExecStart=/usr/libexec/zincati agent -vvvv" > /etc/systemd/system/zincati.service.d/override.conf

# Dump rpm-ostree status to workdir-tmp/status.json
[root@cosa-devsh core]# rpm-ostree status --json > /mnt/workdir-tmp/ostree-status.json

# Prepare a Modified Booted-Status Override
    1. Remove all fedora-coreos.stream annotations.
    2. Add the com.coreos.stream label
    3. Keep only the "deployments" array (this overrides the “booted” view seen by Zincati). 
    see https://github.com/coreos/zincati/blob/a8392314943fbe80b7dcb15d2180d17dca1095f6/src/rpm_ostree/cli_status.rs#L295

# Add the modified status.json file to /run/zincati/booted-status-override.json inside the VM 
[root@cosa-devsh core]# cp /mnt/workdir-tmp/status.json /run/zincati/booted-status-override.json

# Reload the systemd daemon
[root@cosa-devsh core]# systemctl daemon-reload

# Remove the zincati disable auto updates override
[root@cosa-devsh core]# rm /etc/zincati/config.d/90-disable-auto-updates.toml 

# Start the zincati service and profit!
[root@cosa-devsh core]# sudo systemctl start zincati
[root@cosa-devsh core]# sudo systemctl status zincati
● zincati.service - Zincati Update Agent
     Loaded: loaded (/usr/lib/systemd/system/zincati.service; enabled; preset: enabled)
    Drop-In: /usr/lib/systemd/system/zincati.service.d
             └─010-container-signing-migration.conf
             /usr/lib/systemd/system/service.d
             └─10-timeout-abort.conf
             /etc/systemd/system/zincati.service.d
             └─override.conf
     Active: active (running) since Tue 2025-12-02 12:41:15 UTC; 2s ago
 Invocation: be24085b3f91441d85864b0d1ca5fc7a
       Docs: https://github.com/coreos/zincati
    Process: 2674 ExecStartPre=/usr/libexec/coreos-container-signing-migration (code=exited, status=0/SUCCESS)
   Main PID: 2687 (zincati)
     Status: "found update on remote: 43.20251110.3.1"
      Tasks: 25 (limit: 985)
     Memory: 23.3M (peak: 23.8M)
        CPU: 449ms
     CGroup: /system.slice/zincati.service
             ├─2687 /usr/libexec/zincati agent -vvvv
             └─2773 rpm-ostree deploy --lock-finalization sha256:ab87a46a63c05aaeeceaa1fcc7e464c983d0797e7ed150cd5a179d435ad6c816 --disallow-downgrade

Dec 02 12:41:15 cosa-devsh systemd[1]: Started zincati.service - Zincati Update Agent.
Dec 02 12:41:15 cosa-devsh zincati[2687]: [TRACE zincati::cincinnati] got an update graph with 28 nodes and 29 edges
Dec 02 12:41:15 cosa-devsh zincati[2687]: [INFO  zincati::cincinnati] current release detected as not a dead-end
Dec 02 12:41:16 cosa-devsh zincati[2687]: [DEBUG zincati::cincinnati] MOTD updated with no dead-end state
Dec 02 12:41:16 cosa-devsh zincati[2687]: [TRACE zincati::update_agent::actor] update agent tick, current state: UpdateAvailable((Release { version: "43.20251110.3.1", payload: Pullspec(Reference { registry: "qu>
Dec 02 12:41:16 cosa-devsh zincati[2687]: [TRACE zincati::update_agent::actor] trying to stage an update
Dec 02 12:41:16 cosa-devsh zincati[2687]: [INFO  zincati::update_agent::actor] target release '43.20251110.3.1' selected, proceeding to stage it
Dec 02 12:41:16 cosa-devsh zincati[2687]: [DEBUG zincati::rpm_ostree::cli_status] Detected stream 'stable' from com.coreos.stream label
Dec 02 12:41:16 cosa-devsh zincati[2687]: [TRACE zincati::rpm_ostree::actor] request to stage release: Release { version: "43.20251110.3.1", payload: Pullspec(Reference { registry: "quay.io", mirror_registry: No>
Dec 02 12:41:16 cosa-devsh zincati[2687]: [TRACE zincati::rpm_ostree::cli_deploy] Requesting rpm ostree deploy with arguments: CommandArgs { inner: ["deploy", "--lock-finalization", "sha256:ab87a46a63c05aaeeceaa>

@jbtrystram
Copy link
Member

Nice, thanks Roshan !

@Roshan-R Roshan-R force-pushed the fix/replace-stream-annotation-with-label branch from bc47465 to 8476170 Compare December 3, 2025 08:58
}

// Fallback to `fedora-coreos.stream` label in OCI ImageManifest
if let Some(oci_manifest) = deploy.base_commit_meta.oci_manifest.as_ref() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's weird that we have to get the labels through ImageConfig and the annotation through ImageManifest

Looking deeper I find that annotations and labels are basically the same thing (except annotations can be added to multiples files in an image), annotation being the OCI spec way to do it and labels being from Docker.
oh well, now i wonder if we should have done an annotation :)

Interesting read : https://adrianmouat.com/posts/annotations-and-labels-in-container-images/

Copy link
Member

@jbtrystram jbtrystram left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dustymabe
Copy link
Member

@jbtrystram
One thing this does is breaking support for encapsulated ostree deployments built with COSA.
This is probably OK since we won't ship newer Zincati in old-style images but that's something we want to think about,

Exactly. We'll never ship a new release of zincati with the old style (non container based) deployments. The only reason to keep it I can think of is if someone other than us were using Zincati and I don't know of anyone doing that.

and if we drop support for those we should do it accross the whole codebase I think.

SGTM. I think now would be the best time to do this. Do you?

…or stream extraction

Update the stream extraction logic to support the `com.coreos.stream` label
in the OCI image configuration metadata.

Previously, the stream was determined using the `fedora-coreos.stream`
annotation. While that annotation is still supported as a fallback, the
preferred method is now to use the new `com.coreos.stream` label. This change
aligns with the new Konflux-built images, which include only this label and
no longer provide the older annotation.

See: coreos/fedora-coreos-config#3867
@Roshan-R Roshan-R force-pushed the fix/replace-stream-annotation-with-label branch from 8476170 to a142c98 Compare December 5, 2025 06:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants