Lucene: Status=Open

53.9 minutes ago 12 comments 0 votes 0 watches github-actions[bot], github-advanced-security[bot], jimczi, michaeljmarshall, mikemccand, msokolov, neoremind, rmuir

Adds JMH benchmarks to compare read I/O strategies in memory constrained scenario, related to 16044 ... I/O strategies tested: - mmap no madvise - mmap + MADV_NORMAL + MADV_WILLNEED - mmap + ... How to run ` dd if=/dev/urandom of=/path/to/pread-bench-16G.dat bs=1M count=16384 java -jar ...

jimczi 54.6 minutes ago:	LGTM, thanks @neoremind I'll merge once green
neoremind 21.4 hours ago:	Hi @jimczi, I'd like to revive this PR. Would you mind taking another look? As time goes, I find this benchmark useful for multiple other PRs, and it can also be seen as a ...

#16883 PR: Open files twice for sequential reads only when it helps

1.1 hours ago 35 comments 0 votes 0 watches ChrisHegarty, github-actions[bot], jimczi, mikemccand, thecoop, uschindler

Readers open a file a second time for sequential reads, such as merges, only when the input they ... . - Add IndexInputisEfficientFor(IOContext), a hint for whether callers reading with a context ...

jimczi 15.3 hours ago:	A mapping serves every inner file opened on it, so it can't be keyed by file. I inlined mappingFor into one loop in openInput: reuse the first mapping where the file is not open ...
jimczi 15.3 hours ago:	The extra mapping is shared by all inner files, so it's opened without advice, and each inner file ... With context, the whole compound file would get the advice, including inner files that are later ...

#16723 PR: Support SIMD Faiss library variants with runtime fallback (#15287)

1.3 hours ago 46 comments 0 votes 0 watches CH-Abhinav, github-actions[bot], kaivalnp, mikemccand

Description Addresses enhancement proposal 1 from 15287: allow Faiss codec to use SIMD ... Previously, FaissLibrary.java and FaissNativeWrapper.java hardcoded System.loadLibrary("faiss_c"). In Conda/PyTorch, libfaiss_c.so dynamically links against the generic, non-vectorized libfaiss.so ...

CH-Abhinav 1.3 hours ago:	This warning is intentional. It is just saying your CPU is avx512 but SIMD faiss is not available so we will see if you have ...
kaivalnp 9.5 hours ago:	Thanks @CH-Abhinav, left some small comments!

#16145 PR: Delay and cap the prefetch backoff in MemorySegmentIndexInput

3.6 hours ago 51 comments 0 votes 0 watches github-actions[bot], goankur, gsmiller, iprithv, jimczi, michaeljmarshall, mikemccand, navneet1v, neoremind, uschindler

Description MemorySegmentIndexInput.prefetch() currently uses a power-of-two back off to ... This implementation works well for files that are fully loaded into the page cache, especially ... This scheme also works okay for files that have very low cache hit rates because they rarely escape ...

michaeljmarshall 3.6 hours ago:	Thanks @gsmiller. I decided to double check on my benchmarks to confirm the performance of this solution. The tl;dr is that I think this is a good change, but there may be some room for debating the ...
gsmiller 1.5 days ago:	This looks like a good change to me, so I'll contribute my 👍 . Since there's been a lengthy conversation history and multiple people with opinions, I think it's ... But thank you for all the hard work an iterations on this!

#16894 PR: ci: add a label-triggered luceneutil benchmark workflow for pull requests

9.3 hours ago 0 comments 0 votes 0 watches owaiskazi19

Description - Part of 16398: committers can benchmark a PR by adding a label. - luceneutil compares the PR's head commit against its merge base on a GitHub runner and posts ...

#16893 PR: Skip index-sort range shortcuts on segments with blocks and multi-valued fields

9.5 hours ago 0 comments 0 votes 0 watches HoustonPutman, github-actions[bot]

Fixes https://github.com/apache/lucene/issues/16884, https://github.com/apache/lucene/issues/16885, ... Description This fixes a few range query bugs regarding multi-valued fields and segments with ...