feat(server): implement log rotation based on size and retention #2452

Svecco · 2025-12-07T06:09:36Z

Redirected from issue #46 , the progress:

Implemented _install_log_rotation_handler function, which was previously marked as todo;
Implemented cleanup_log_files to delete logs based on the configured time threshold & size;
Merged multiple directory traversals into a single traversal;
Reduced multiple metadata() calls by collecting data in single traversal;
Split log cleanup logic into smaller, more manageable functions;
Improved error handling with detailed error messages and proper error propagation;
Fixed an issue where the directory did not exist when loading logs;
Added disk space checking before initializing file logging;

Before merge:

Verified to comply with Apache specifications;
Add a relevant unit test for logging;
Confirm that the code has been optimized well?
Squash commits to a proper number after polishing;
Solve conflicts and confirm it is ready for being merged;

Plan:
Around the next several weekends.

hubcio · 2025-12-07T09:53:08Z

I think changes in core/message_bus are not related to log retention, looks like merge problem.

…retention - Configurable maximum log size and retention period from server configuration - Optimized single directory traversal for collecting file information - Eliminated redundant metadata() calls for improved performance - Enhanced error handling with detailed logging for all failure cases - Merged directory traversals into a single operation to improve performance Fixes apache#46, detail changes can be found at apache#2452.

Svecco · 2025-12-13T03:09:20Z

I think changes in core/message_bus are not related to log retention, looks like merge problem.

Yes, that was the case. I have just rebased it to the correct state.

core/server/src/log/logger.rs

…retention - Configurable maximum log size and retention period from server configuration - Optimized single directory traversal for collecting file information - Eliminated redundant metadata() calls for improved performance - Enhanced error handling with detailed logging for all failure cases - Merged directory traversals into a single operation to improve performance Fixes apache#46, detail changes can be found at apache#2452.

Svecco · 2025-12-27T12:31:00Z

@hubcio @spetz I've just squashed commits into one.

core/server/src/configs/defaults.rs

core/server/src/configs/system.rs

core/configs/server.toml

core/server/src/configs/system.rs

core/server/src/log/logger.rs

core/server/Cargo.toml

core/server/src/log/logger.rs

…retention - Configurable maximum log size and retention period from server configuration - Optimized single directory traversal for collecting file information - Eliminated redundant metadata() calls for improved performance - Enhanced error handling with detailed logging for all failure cases - Merged directory traversals into a single operation to improve performance Fixes apache#46, detail changes can be found at apache#2452.

- Log directory creation functionality - Disk space checking feature - Log file count calculation - Log cleanup functions - Logging struct initialization - Fixed some problems

Svecco · 2026-01-01T10:06:16Z

I deliberately set some values to conduct a simple test of the current log rotation functionality. Generally speaking, it should now be operating normally.

Console

2026-01-01T09:03:36.553786Z  INFO shard-12 server::websocket::websocket_listener: WebSocket config: max_message_size: None, max_frame_size: None, accept_unmasked_frames: false
2026-01-01T09:03:37.523106Z DEBUG log-rotation server::log::logger: Starting log cleanup for directory: "local_data/logs", retention: 3600s, max_total_size: 4000000000 bytes, max_single_file_size: 512000000 bytes
2026-01-01T09:03:37.523237Z DEBUG log-rotation server::log::logger: Processed 4 log files from directory: "local_data/logs"
2026-01-01T09:03:37.523250Z DEBUG log-rotation server::log::logger: Mark old log file for remove: "local_data/logs/iggy-server.log.4"
2026-01-01T09:03:37.523260Z DEBUG log-rotation server::log::logger: Mark old log file for remove: "local_data/logs/iggy-server.log.1"
2026-01-01T09:03:37.523265Z DEBUG log-rotation server::log::logger: Mark old log file for remove: "local_data/logs/iggy-server.log.3"
2026-01-01T09:03:37.523270Z DEBUG log-rotation server::log::logger: Mark old log file for remove: "local_data/logs/iggy-server.log.2"
2026-01-01T09:03:37.530455Z DEBUG log-rotation server::log::logger: Removed log file: "local_data/logs/iggy-server.log.2"
2026-01-01T09:03:37.562069Z DEBUG log-rotation server::log::logger: Removed log file: "local_data/logs/iggy-server.log.3"
2026-01-01T09:03:37.569756Z DEBUG log-rotation server::log::logger: Removed log file: "local_data/logs/iggy-server.log.1"
2026-01-01T09:03:37.657651Z DEBUG log-rotation server::log::logger: Removed log file: "local_data/logs/iggy-server.log.4"
2026-01-01T09:03:37.657759Z  INFO log-rotation server::log::logger: Completed log cleanup for directory: "local_data/logs". Removed 4 files.
2026-01-01T09:03:38.657860Z DEBUG log-rotation server::log::logger: Starting log cleanup for directory: "local_data/logs", retention: 3600s, max_total_size: 4000000000 bytes, max_single_file_size: 512000000 bytes
2026-01-01T09:03:38.657958Z DEBUG log-rotation server::log::logger: Processed 0 log files from directory: "local_data/logs"
2026-01-01T09:03:38.657972Z  INFO log-rotation server::log::logger: Completed log cleanup for directory: "local_data/logs". Removed 0 files.
2026-01-01T09:03:39.658081Z DEBUG log-rotation server::log::logger: Starting log cleanup for directory: "local_data/logs", retention: 3600s, max_total_size: 4000000000 bytes, max_single_file_size: 512000000 bytes
2026-01-01T09:03:39.658171Z DEBUG log-rotation server::log::logger: Processed 0 log files from directory: "local_data/logs"
2026-01-01T09:03:39.658184Z  INFO log-rotation server::log::logger: Completed log cleanup for directory: "local_data/logs". Removed 0 files.
2026-01-01T09:03:40.658282Z DEBUG log-rotation server::log::logger: Starting log cleanup for directory: "local_data/logs", retention: 3600s, max_total_size: 4000000000 bytes, max_single_file_size: 512000000 bytes
2026-01-01T09:03:40.658376Z DEBUG log-rotation server::log::logger: Processed 0 log files from directory: "local_data/logs"
2026-01-01T09:03:40.658389Z  INFO log-rotation server::log::logger: Completed log cleanup for directory: "local_data/logs". Removed 0 files.

…ssues - Rename variables (max_size => max_file_size, internal attributions) - Configure customize rotation check option and redirect default unwrap values - Fix prevention of active log file deletion - Optimize thread handling (graceful shutdown, thread communication) - Validate invalid configuration parameters - Sync e2e/server.toml configuration - Use full semantic versioning for dependencies

hubcio

I tested your code manually. In server.toml i changed settings to:

max_file_size = "1 MB"
max_total_size = "10 MB"
rotation_check_interval = "1 s"

then I rebulilt server and started it via:
RUST_LOG=trace target/debug/iggy-server --with-default-root-credentials --fresh

and in other shell i run benchmark to generate logs:
target/debug/iggy-bench -T 1000MB pp -p 1 tcp

result:
I can never see more than 2 files, it looks like retention is ignored and 10 MB of max total size is not reached.

can you write integration test that would test for this scenario? see how TestServer is used, in integration module (e.g. core/integration/tests/server/specific.rs)

hubcio · 2026-01-02T08:51:47Z

core/server/src/log/logger.rs

+            }
+        }
+
+        info!(


dont print anything if 0 files were removed

Svecco force-pushed the 46-size-logs branch from d51fcec to 6b8546c Compare December 7, 2025 06:18

Svecco force-pushed the 46-size-logs branch from 6b8546c to 8390a77 Compare December 13, 2025 03:00

Svecco marked this pull request as ready for review December 13, 2025 03:04