Lucene: Status=Open

[Save this search]

Author relation

Updated ago

Comment count

Reaction count

Review Requested

User

Last comment user

Type

Filters: Status (Open), Issue type, Author relation, Created, Updated, Updated ago, Comment count, Reaction count, Review Requested, Mentioned, Reviewed, Commented, User, Last comment user, Draft, Component, Type, Labels, Commits?, Reporter, Assignee

#16771 PR: Page-align vector data for all encodings

1.4 hours ago 3 comments 0 votes 0 watches CH-Abhinav, Pulkitg64, github-actions[bot], goankur

Description Fast-follow to 16705. Rescoring with full-precision vectors reads one whole vector per candidate. A 1024-dim float32 vector is exactly 4096 bytes, but Lucene99FlatVectorsWriter aligns the vector ...

goankur 1.4 hours ago:	@Pulkitg64 Yes, and it matters most for large vectors. Today FLOAT16 and BYTE align to 4 bytes, so at 1024 dims every other FP16 vector (2 KiB) and every ... At 4096 dims an FP16 vector is 8 KiB, exactly two pages.
CH-Abhinav 4.6 hours ago:	Yeah can't we do similar alignment to Bytes and F16's? 1024 for BYTES and 2048 for F16. what do you guys think?

#16729: A briefly saturated intra-merge executor can serialize an entire HNSW graph merge

5.1 hours ago 6 comments 0 votes 0 watches CH-Abhinav, abernardi597

Description Findings by humans, report and reproductions were AI-assisted (Claude Opus 5). Spinoff from this thread on luceneutil. Summary HnswConcurrentMergeBuilder hands out work through one shared batch counter, so any single ...

CH-Abhinav 5.1 hours ago:	any more suggestion you have @abernardi597 ? we can try your idea if you want but i am not sure about it's broader changes in codebase.
CH-Abhinav 2.3 days ago:	Thanks for sharing your ideas @abernardi597! Just to clarify on the approach in this PR: this isn't static work assignment — work remains fully ... Instead of pre-assigning static slices to specific workers, it converts the batches themselves into ...

#16731 PR: Add dedup-aware HNSW format that builds the graph over distinct vectors and expands matches to all ...

8.8 hours ago 3 comments 0 votes 0 watches Pranshu-S, github-actions[bot]

The dedup vector formats in de-duplicate flat vector storage but delegate graph construction to ... When documents share vectors, that redundantly indexes the same point once per document. This PR adds a de-duplication-aware HNSW writer/reader that build and search the graph over ...

Pranshu-S 8.8 hours ago:	Ran some benchmark suits, I was able to find meaningful results only when there close to 70-80% ... Even in those cases, it would make sense to just run exact KNN on the dedup than build a Hnsw Graph ... What might be good next points to explore would be IVF and tiered search similar to Vamana?
Pranshu-S 9.2 hours ago:	Few optimisations that I can think off if I were to pull this to a state it could be used: 1. Decide dynamically when to use this new Writer/Reader 2. Performing post-filter on results on the Hnsw results since if we go with (1) - high change that we ...

#16769 PR: Map vectors again for the merges #16749 missed

14.7 hours ago 1 comments 0 votes 0 watches github-actions[bot], jimczi

16749 made Lucene99FlatVectorsReader.getMergeInstance() map the raw vectors again for a merge, so a ... Two kinds of readers never got there. Readers that wrap a raw reader returned this from getMergeInstance(), so the raw reader's merge ...

jimczi 1.5 days ago:

I've taken Lucene104ScalarQuantizedVectorsReader out of this PR: 16622 already forwards ... This PR keeps the Faiss reader, the de-duplicating formats, and the release bookkeeping in ...

#16627 PR: Parallelize the repair process for disconnected nodes during HNSW merge-re-use

19.2 hours ago 51 comments 0 votes 0 watches Pulkitg64, ethbak, github-actions[bot], msokolov

Overview Currently, when IncrementalHnswGraphMerger reuses an existing graph, InitializedHnswGra ... This logic runs inside ConcurrentHnswMerger.createBuilder. Which is called before HnswConcurrentMergeBuilder even exists, so the copy/repair/rebalance process ...

Pulkitg64 20.1 hours ago:	What do you think about selecting random delete % under DELETE_PCT_THRESHOLD instead of using ...
Pulkitg64 20.2 hours ago:	Thanks!

#16666 PR: Iouring rerank reads for larger-than-RAM KNN search

20.1 hours ago 1 comments 0 votes 0 watches 190, 230, 250, github-actions[bot], goankur

Title: **(Implemented with AI and Human in the loop)** Sandbox IoUringDirectory: batched ... **Nothing outside lucene/sandbox/ is touched.** Description In 16656 a rerank shortlist is ... Each read costs a syscall, a thread hand-off and a blocked worker. io_uring replaces the pattern: ...

github-actions[bot] 20.1 hours ago:

This PR has not had activity in the past 2 weeks, labeling it as stale. If the PR is waiting for review, notify the dev@lucene.apache.org list. Thank you for your contribution!

#16766 PR: Fix outdated SegmentInfos#readCommit javadocs

1.4 days ago 2 comments 0 votes 0 watches github-actions[bot], javanna, link, rahulgoswami

readCommit needs updating after #14607

rahulgoswami 1.4 days ago:	Thanks for catching this @javanna . I missed this java doc edit during 14607 . One edit I might suggest is since minVersion is a segment property, it might be more appropriate to ...
javanna 2.3 days ago:	@rahulgoswami this relates to the changes you made with #14607 . Could you please take a look?

«
1
2
3
4
5
»

2,656 issues

[138.8 msec search, 142.5 msec total]