Lucene: Status=Open

1.4 hours ago 32 comments 0 votes 0 watches github-actions[bot], goankur, iprithv, jimczi, michaeljmarshall, mikemccand, navneet1v, neoremind, uschindler

Description The power-of-two back off in MemorySegmentIndexInput.prefetch() assumes consecutive ... This assumption works well for files that are already warm, but is too aggressive when files are ... The proposed solution makes two primary changes: 1.

michaeljmarshall 1.4 hours ago:	My initial goal with the bit mixing and counting local clone calls to prefetch was to spread out ... Then, @goankur suggested that wasn't ideal in the first case: > 1. shouldProbe(prefetchCount++) ... SegmentTermsEnum clones termsIn for each lookup, so every term lookup in every segment probes.
uschindler 14.4 hours ago:	Why do we need a random for the probing? I would never ever use ThreadLocalRandoms at such places in code. IMHO, real randomness is not needed here.

#12892: Remove all deprecated IndexSearcher#search(Query, Collector) usage / methods in the next major ...

1.5 hours ago 6 comments 0 votes 0 watches gaobinlong, github-project-automation[bot], gsmiller, javanna, msfroh, msokolov, romseygeek, sgup432, vijaykriishna, zacharymorn

Description As a follow-up of https://github.com/apache/lucene/issues/11041, we would like to ... A list of the leftover usages follows:- [x] facet: FacetsCollector (ongoing discussion at #13725, ...

msokolov 1.5 hours ago:	I made a stab at converting the JoinUtils over to CollectionManager, but it is not trivial as I ... I wonder if we could somehow disable the multithreading. Maybe the CollectorManager could signal its need to run on a single thread somehow?
gsmiller 2.1 years ago:	Note that we added a factory method in CollectorManager that might make these migration tasks a ... See CollectorManagerforSequentialExecution (and thanks to @romseygeek for the idea!). EDIT: After more consideration, this is probably too trappy for users and is likely getting removed ...

#16800 PR: Backport "Fix Lucene104ScalarQuantizedVectorsWriter to skip deleted docs when recomputing merge ...

2.3 hours ago 0 comments 0 votes 0 watches github-actions[bot], kaivalnp

DescriptionBackport #16764 to branch_10x

#16794 PR: Speed up concurrent HNSW merges by joining the smaller graphs

2.4 hours ago 3 comments 0 votes 0 watches LantaoJin, Pulkitg64, benwtrent, github-actions[bot], kaivalnp, mayya-sharipova, msokolov

Description ConcurrentHnswMerger never uses the join-set merge from 14331. It initializes the merged graph from the largest input graph and then inserts every other vector ... Only the serial MergingHnswGraphBuilder joins them.

mayya-sharipova 2.4 hours ago:	There is also https://github.com/apache/lucene/pull/16664 that addresses the same problem, but I ... @LantaoJin Would you mind rebasing against main so we can cleanly see your changes against main
Pulkitg64 3.9 hours ago:	Thanks @LantaoJin for making this change. I will try to have a look. JFYI: This PR (#16627) got approved which adds some concurrent addition of nodes during graph ...

#16762: Lucene104ScalarQuantizedVectorsWriter includes deleted docs when recalculating the centroid

2.6 hours ago 1 comments 0 votes 0 watches Pulkitg64, kaivalnp

Description Lucene104ScalarQuantizedVectorsWriter computes a per-field centroid before quantizing ... The centroid is the mean of the full-precision vector values gathered from the segments being ... When any source segment has deletions, the recalculation path does not skip the deleted documents.

kaivalnp 2.6 hours ago:

16764 was merged, but there are some merge conflicts while backporting to branch_10x. Awaiting a backport PR on branch_10x from the author for visibility. I don't want to delay the 10.6 release, so please feel free to skip this change (we'll just need to ...

#16799 PR: Bulk score 4-bit Lucene104 vectors with a single native call

5.1 hours ago 0 comments 0 votes 0 watches github-actions[bot], slow-J

I was experimenting with native bulk scoring and I saw some good results in benchmarks with this ... I am routing the scorer through VectorizationProvider so the native provider can score a batch of ... The C code scores four vectors per load of the query, with NEON dot product instructions on aarch64 ...