Improper key existence checks in hashindex code

[rolandrc]

In 1.4-maint, on line 240 (in NSIndex), it checks if marker is None. Instead, I think that's meant to check the key it received the line before.

borg/src/borg/hashindex.pyx

Lines 233 to 243 in b1aa1a4

	def iteritems(self, marker=None):
	cdef const unsigned char *key
	iter = NSKeyIterator(self.key_size)
	iter.idx = self
	iter.index = self.index
	if marker:
	key = hashindex_get(self.index, <unsigned char *>marker)
	if marker is None:
	raise IndexError
	iter.key = key - self.key_size
	return iter

The same goes for ChunkIndex:

borg/src/borg/hashindex.pyx

Line 357 in b1aa1a4

if marker is None:

As far as I can tell, Borg never passes in a marker that does not exist, so this isn't a big deal. It can be broken manually though. On a system with Borg 1.2.8 (which has the same code):

Python 3.12.3 (main, Jan 22 2026, 20:57:42) [GCC 13.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import borg.hashindex
>>> nsindex = borg.hashindex.NSIndex()
>>> 
>>> # The hash table stores them in this order.
>>> nsindex[b'\xbb'*32] = (123, 456)
>>> nsindex[b'\xaa'*32] = (234, 567)
>>> 
>>> # Show all items:
>>> list(nsindex.iteritems())
[(b'\xbb\xbb\xbb\xbb\xbb\xbb\xbb\xbb\xbb\xbb\xbb\xbb\xbb\xbb\xbb\xbb\xbb\xbb\xbb\xbb\xbb\xbb\xbb\xbb\xbb\xbb\xbb\xbb\xbb\xbb\xbb\xbb', (123, 456)), (b'\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa', (234, 567))]
>>> 
>>> # Show items after the `bb` item:
>>> list(nsindex.iteritems(marker=b'\xbb'*32))
[(b'\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa', (234, 567))]
>>> 
>>> # Show items after a `cc` item, which does not exist:
>>> list(nsindex.iteritems(marker=b'\xcc'*32))
Segmentation fault (core dumped)

Copy-paste code

import borg.hashindex
nsindex = borg.hashindex.NSIndex()

# The hash table stores them in this order.
nsindex[b'\xbb'*32] = (123, 456)
nsindex[b'\xaa'*32] = (234, 567)

# Show all items:
list(nsindex.iteritems())

# Show items after the `bb` item:
list(nsindex.iteritems(marker=b'\xbb'*32))

# Show items after a `cc` item, which does not exist:
list(nsindex.iteritems(marker=b'\xcc'*32))

[ThomasWaldmann]

Yeah, that looks like a bug, thanks for reporting this!

[ThomasWaldmann]

Bug in borg 1.x confirmed.

• borg < 1.4 won't get fixed, 1.2.9 was last release, bug is harmless (borg always gives existing marker).
• borg 1.4-maint: fix in Fix hashindex .iteritems (1.4-maint) #9370
• master branch: does not contain this code. hashtable code was moved to borghash project and reimplemented in Cython, does not have this bug anymore. https://github.com/borgbackup/borghash/blob/master/src/borghash/HashTable.pyx