[Rust] Generate setters for optional primitive fields

[leonardoarcari]

Problem

The Rust codegen has no actionable way to set optional scalar primitive fields to their null-value, which is only made available as text in the generated docstring. Moreover, there's no uniform way to null-initialize all optional primitive scalar fields at the Encoder level.

In practice, callers have to remember and invoke each optional field setter individually with its null representation, which is error-prone, harder to audit, and easy to miss as schemas evolve.

Design Proposal

The generated encoders now expose a dedicated nullification path for optional primitive scalar fields:

• A generated method that clears all optional primitive scalar fields in one place (inspired by sbepp approach).
• Per-field optional-friendly setters for optional primitive scalar fields, so generated nullification logic can remain schema-driven and centralized.
• For message/composite/group scopes with no optional primitive scalar fields, the generated nullification method is still present as an intentional no-op with explicit documentation in the body (principle of least-surprise)

Generated API Examples

// Existing required-value setter remains unchanged.
pub fn cup_holder_count(&mut self, value: u8) { ... }

// New additive optional-friendly setter.
pub fn cup_holder_count_opt(&mut self, value: Option<u8>) {
    match value {
        Some(value) => self.cup_holder_count(value),
        None => self.cup_holder_count(0xff_u8), // schema null value
    }
}

// Generated when optional primitive scalar fields exist.
pub fn nullify_optional_fields(&mut self) {
    self.cup_holder_count_opt(None);
    self.some_other_optional_field_opt(None);
}

// Generated when no optional primitive scalar fields exist in that scope.
pub fn nullify_optional_fields(&mut self) {
    // No optional primitive scalar fields exist; nothing to do.
}

This design improves ergonomics and correctness for callers that need deterministic “absent” encoding semantics, while keeping generated API behavior explicit across all encoder scopes.

Compatibility Decision

I considered changing the existing required-value setter signatures to accept Option<T> directly.
I did not take that path because it would be a breaking API change for existing generated-code consumers and would force updates across downstream codebases.

Instead, I introduced additive methods so existing call sites continue to compile unchanged, and clients can adopt optional-aware behavior incrementally.

Test Coverage Summary

The test suite covers:

• Optional field setters encoding both present and absent values correctly.
• End-to-end nullification on message encoders with optional primitive fields.
• End-to-end nullification on composite encoders with optional primitive fields.
• End-to-end nullification on group encoders with optional primitive fields.
• No-op behavior on message/composite/group encoders that have no optional primitive fields.
• Regression checks that decoding after nullification returns None where expected.

[vyazelenko]

@mward Can you please review this PR? Thanks.

[mward]

I'll take a look

[leonardoarcari]

I'll take a look

Appreciate it, @mward!

[mward]

looks good, thanks!

[leonardoarcari]

looks good, thanks!

Thanks for the quick turnaround @mward 💪

[mward]

no problem

[ZachBray]

IIUC, the SBE v1 specification permits optional non-scalar fields. For instance, there is an example where an enum is optional:

Source: SBE v1 Field Encoding

As a user, I might expect nullify_optional_fields to "nullify" these fields too. What do you (collectively) think?

This design improves ergonomics and correctness for callers that need deterministic “absent” encoding semantics

Is the aim that a user can, without "zeroing out" the underlying buffer, only set all the required fields and have a guarantee that any "lingering data" from a previous encode is overwritten? In other implementations, e.g., Java/C++, I've seen missing calls to reset() trip people up, in a similar way.

[leonardoarcari]

IIUC, the SBE v1 specification permits optional non-scalar fields. For instance, there is an example where an enum is optional:

Source: SBE v1 Field Encoding

As a user, I might expect nullify_optional_fields to "nullify" these fields too. What do you (collectively) think?

@ZachBray Good catch, you are right! Pushed a change in 50fab5b to generate _opt setters for optional enum fields too and nullify them in nullify_optional_fields.

To clarify, I only nullify enum fields that are marked as optional at the field level, not at the enum definition level. One could argue otherwise, but this seems more in-line with how C++ and Java generators deal with enums with an optional presence.

@mward Can you take a look at these changes too? Thanks 🙏

[leonardoarcari]

This design improves ergonomics and correctness for callers that need deterministic “absent” encoding semantics

Is the aim that a user can, without "zeroing out" the underlying buffer, only set all the required fields and have a guarantee that any "lingering data" from a previous encode is overwritten? In other implementations, e.g., Java/C++, I've seen missing calls to reset() trip people up, in a similar way.

@ZachBray Correct and imo it's even worse than that. Optional enum and fields can specify their own encoding for NullValue, so starting from a zero'd out buffer won't even do the trick. Code encoding SBE values would need to explicitly go through each optional field and set it to their NullValue not to cause havoc in Decoders. I know this is intentional, but in practice SBE users often get their types generated from schemas updated by many other people and it's easy to miss a new optional field added to the generated types.

By null-initializing all the optional fields in nullify_optional_fields(), Encoders using it are guaranteed not to miss optional fields initialization anymore across schema evolutions.

[ZachBray]

To clarify, I only nullify enum fields that are marked as optional at the field level, not at the enum definition level. One could argue otherwise, but this seems more in-line with how C++ and Java generators deal with enums with an optional presence.

My understanding is that the XML parser assigns fields presence values based on their types when left unspecified. According to the spec, if presence is specified at both levels, the values should align. Therefore, I think trusting the field level is fine in messages.

When enums are inside composites, there are no surrounding field tokens carrying the presence information. Your code looks correct there: it pulls the presence value from the enum token.

The change LGTM.

Another question: in your implementation, does nullify_optional_fields (at the message, group, or composite level) delegate to composites within the associated "block"? Or is the intention that the developer adds another call when they add a new composite? Clearly, one has to add another (repeated) call for any new group structures (evolution rules permitting).