Contribution Guidelines

[jperkin]

This is for now a brief FAQ for how to submit changes and what the guidelines are for getting changes integrated into dreckly.

It is expected that this document will change over time as we improve our workflows and find more efficient ways to test and integrate changes.

Submitting Changes

From Outside Contributors

Pull requests from outside the organisation are accepted, but there are limitations on the testing that can be performed on them. GitHub Actions limits access to variables, so:

• CI CAN run on GitHub-hosted runners, after a maintainer has approved the workflow run.
• Jenkins builds CANNOT run as the trigger requires access to secrets.

This limits builds to macOS, Linux, and Windows, but is still enough to get started, and a maintainer can always pull the change in manually to get wider testing.

From Maintainers

Users who have access to the drecklypkg organisation are recommended to use the automated ci/* branch matching rather than pull requests to test changes, and are asked to start a manual update-binpkg workflow for the packages they intend to modify first, so that binary packages for dependencies can be cached, as well as providing a baseline set of CI results to compare against.

As an example, let's say I want to test an update to postfix:

• First go to update-binpkg, click "Run workflow", and input mail/postfix in the list of packages to build.
• Create a ci/postfix-x.y.z branch, push changes there, check for results.
• Once everything looks good, squash the changes to trunk and push.

Platform Support / CI Results

The biggest difference in dreckly compared to pkgsrc is that we care about all of the platforms that we claim to support, not just NetBSD. Our motto is "No Regressions". Having all changes automatically tested prior to commit is a demonstration of our commitment to this motto.

Many CI runs will show failures across different platforms. To be clear, that is ok! We do not, and never will, claim that all packages will build on all platforms. What is not acceptable is causing a regression. It is unfair, rude, and ultimately just bad software engineering to make a change that benefits you at the cost of someone else.

• If a package previously built on a platform but your proposed change makes it fail, it's not acceptable.
• If a package did not build and the fix is easy, it is highly recommended to spend some time fixing it.
• If a package did not build and is a lot of work to fix, it's fine to leave it broken, but maybe see if someone else might be interested in fixing it.

Larger Changes

Sometimes a change is bigger than CI can test, for example updating boost or making a change to the infrastructure. In these cases we need to be much more careful to avoid regressions that CI does not (currently) show.

• Any larger changes must be reviewed by at least one other maintainer.
• Before-and-after bulk builds must be performed to ensure no regressions across the tree (@jperkin can run these).

If the change is on the larger side but doesn't affect too many other packages, you can e.g. make dummy changes to those packages so that CI is triggered for them, and then make sure those changes are not in the final commit.

Update Cadence

Every package update has a cost. Not only in CI time, but for every user and bulk build that now needs to rebuild the package. Of course for many updates this is fine, we obviously want improvements, new features, security fixes, etc. However, the cost needs to be carefully considered. Even on the fastest modern hardware a full bulk build will take around a day from scratch. Older platforms may be counting in weeks or even months.

In dreckly the general rule is package update frequency should be inversely proportional to its reverse dependency score.

Let's consider this by showing an example of how pkgsrc does this badly. devel/cmake is a core package, and affects around 10,000 dependencies. Every time it is touched, every bulk build, whether running on fast x86_64 or NetBSD/vax now needs to rebuild those 10,000 packages, many of which are huge (llvm, rust, nodejs, etc). Depending on the software used to update their packages, users may also have to rebuild all of their installed packages that use cmake.

How many times was it updated in the past year?

672ffe3d4785 2025-01-29 cmake cmake-gui: updated to 3.31.5
f15e74642cb1 2025-01-14 cmake cmake-gui: updated to 3.31.4
b28a2391a4f9 2024-12-24 cmake cmake-gui: updated to 3.31.3
33dc9f967cbc 2024-12-11 cmake: downgraded to 3.31.1 for the sake of 1 December
eb661d9f41eb 2024-12-11 cmake cmake-gui: updated to 3.31.2
84d219fa7e79 2024-11-27 cmake cmake-gui: updated to 3.31.1
ad236b6e534f 2024-11-10 cmake cmake-gui: updated to 3.31.0
85c22e62c146 2024-10-10 cmake: updated to 3.30.5
6563c45a8e37 2024-09-30 cmake: updated to 3.30.4
254db3685012 2024-08-30 cmake cmake-gui: updated to 3.30.3
5898a8ebbdd0 2024-08-03 cmake: updated to 3.30.2
3580b2e8552d 2024-07-19 cmake cmake-gui: updated to 3.30.1
e6a278b386c6 2024-07-04 cmake cmake-gui: updated to 3.30.0
1fb497c82095 2024-06-19 cmake cmake-gui: updated to 3.29.6
e0066b5a6427 2024-06-10 cmake cmake-gui: updated to 3.29.5
6ef35771b7b9 2024-06-04 cmake cmake-gui: updated to 3.29.4
1764818b5c70 2024-05-10 cmake: updated to 3.29.3
3b329370faea 2024-04-13 cmake cmake-gui: updated to 3.29.2
611ade583aca 2024-04-05 cmake: updated to 3.29.1
d35104ec1e11 2024-04-02 cmake: updated to 3.29.0
e37cb563f68f 2024-03-22 cmake: updated to 3.28.4
338a0556d015 2024-02-11 cmake: updated to 3.28.3
7d7ecefed18e 2024-02-01 cmake cmake-gui: updated to 3.28.2

This doesn't even include the many changes that were made to e.g. devel/cmake/build.mk that would also cause rebuilds but are necessary for platform support and other fixes.

This is quite clearly ridiculous. It should be updated once, maybe twice per year if the changes are urgent, to the latest stable major once it has had time to receive any necessary fixes.

As another example, databases/sqlite3 received 9 version updates last year, even though it has a reverse dependency count of 17,000.

If you are unsure of the reverse dependency count it can be easily generated using the pbuild file from a completed bulk build, e.g.:

$ sort -t'|' -k4rn pbuild | awk -F'|' '{print $4 "\t" $1}' | head
28330   cwrappers-20220403
28312   checkperms-1.12
25191   pkg_install-info-4.5nb3
24907   m4-1.4.19nb1
24826   libtool-base-2.4.7nb1
24599   gmake-4.4.1
23683   pkgconf-2.3.0
23401   pax-20210219
23212   perl-5.38.2
21435   ncurses-6.5

and a full, updated list can be provided by @jperkin on request.

[outpaddling]

How about updates to mk files? I would propose the following as a fix for the sra-tools failure on FreeBSD:

--- a/mk/platform/FreeBSD.mk
+++ b/mk/platform/FreeBSD.mk
@@ -70,6 +70,21 @@ SERIAL_DEVICES?=     /dev/cuaa0
 OPSYS_HAS_KQUEUE=      # defined
 .endif
 
+# check for eventfd(2) support
+.if exists(${_CROSS_DESTDIR:U}/usr/include/sys/eventfd.h)
+OPSYS_HAS_EVENTFD=     # defined
+.endif
+
+# check for timerfd(2) support
+.if exists(${_CROSS_DESTDIR:U}/usr/include/sys/timerfd.h)
+OPSYS_HAS_TIMERFD=     # defined
+.endif
+
+# check for epoll(2) support
+.if exists(${_CROSS_DESTDIR:U}/usr/include/sys/epoll.h)
+OPSYS_HAS_EPOLL=       # defined
+.endif
+
 # check for posix_spawn(3) support, added in FreeBSD-8.0
 .if exists(/usr/include/spawn.h)
 OPSYS_HAS_POSIX_SPAWN= # defined

[jperkin]

Yep, absolutely fine as it can only affect the platform in question. In terms of the actual change, remove the _CROSS_DESTDIR stuff as we're not planning to support cross-compilation (#28).

[outpaddling]

Thanks, any preferred procedure for committing mk changes?

[jperkin]

I think it makes sense for those to be pull requests for now so that they can be approved by another maintainer.

[outpaddling]

By "squash the changes to trunk and push", I'm guessing you mean:

1. git clone https://<my-user-name>@github.com/drecklypkg/dreckly
2. cd dreckly
3. git merge --squash <my-test-branch>
4. git add <new-files>
5. git commit -a
6. git push

I'm not seeing a way to merge branches via the web interface.