Lucene: Status=Open

[Save this search]

Author relation

Updated ago

Comment count

Reaction count

Review Requested

User

Last comment user

Type

Filters: Status (Open), Issue type, Author relation, Created, Updated, Updated ago, Comment count, Reaction count, Review Requested, Mentioned, Reviewed, Commented, User, Last comment user, Draft, Component, Type, Labels, Commits?, Reporter, Assignee

#16792: Explore ALP encoding for numeric doc values in sandbox

5.3 minutes ago 1 comments 0 votes 0 watches sgup432

Description As of today, when one index a double field(prices, coordinates etc) as doc values, ... That's because doubles are converted to a long via NumericUtils.doubleToSortableLong before storage ... So the numeric doc values encoders (GCD, FOR, etc.) don't work well on them, the decimal structure ...

sgup432 5.8 minutes ago:

I have been doing some experimentation and built a sandbox prototype (ALPDocValuesFormat, opt-in ... **Storage: real on-disk size, 100k values, bits per value** | Dataset | Default | ALP | Ratio | |: ... The ALP index routes the field to ALPDocValuesFormat through getDocValuesFormatForField.

#16791 PR: Validate dict and block lengths in preset-dict decompressors

36.4 minutes ago 0 comments 0 votes 0 watches github-actions[bot], harshvardhanrp

Description <Write 1 to 3 sentences here in your own words. For example: how you found this and why you think it is worth fixing.> > The preset-dict ... .

#16771 PR: Page-align vector data for all encodings

1.8 hours ago 6 comments 0 votes 0 watches CH-Abhinav, Pulkitg64, github-actions[bot], goankur, kaivalnp

Description Fast-follow to 16705. Rescoring with full-precision vectors reads one whole vector per candidate. A 1024-dim float32 vector is exactly 4096 bytes, but Lucene99FlatVectorsWriter aligns the vector ...

goankur 3.7 hours ago:	Thanks @kaivalnp. Agreed, main does not rescore from bytes today, so for BYTE the padding only helps when HNSW reads ... Byte rescoring is planned.
kaivalnp 1 day ago:	> can't we do similar alignment to Bytes and F16's? I think the alignment is beneficial for something that is repeatedly read from the index on the " ... Vector re-scoring might be a good use-case, because raw vectors can be "cold".

#16778 PR: Fix parallelism cap when current thread is enrolled in concurrent HNSW merge

2 hours ago 0 comments 0 votes 0 watches abernardi597, github-actions[bot]

Closes 16729. Description HnswConcurrentMergeBuilder submitted all of its workers through TaskExecutorinvokeAll ... The intra-merge executor runs a command on the calling thread when it has no spare thread, so when ...

#16729: A briefly saturated intra-merge executor can serialize an entire HNSW graph merge

2 hours ago 9 comments 0 votes 0 watches CH-Abhinav, abernardi597

Description Findings by humans, report and reproductions were AI-assisted (Claude Opus 5). Spinoff from this thread on luceneutil. Summary HnswConcurrentMergeBuilder hands out work through one shared batch counter, so any single ...

abernardi597 2 hours ago:	I pushed a new revision that fixes those oversights and generally improves the quality of the PR. Probably ready for review :)
CH-Abhinav 1.7 days ago:	Draft looks good but CI fails because started is List<Future<Void>> and TaskExecutor.collectResults ... Also once check other files which use TaskExecutor .

#16710 PR: Add scalar quantization support for float16 vectors in the dedup KNN Codec

3 hours ago 47 comments 0 votes 0 watches Pranshu-S, github-actions[bot], kaivalnp

Description Addresses one of improvements tracked in 16595 The sandbox dedup KNN codec stores ... Its scalar-quantized variant (DedupScalarQuantizedVectorsFormat) additionally stores a data-blind ... Previously, only FLOAT32 vectors were quantized — FLOAT16 (and BYTE) were stored raw only.

Pranshu-S 4.2 hours ago:	Looks like all recent PRs are failing due to CodeQL, will merge once the fix for it lands in main ...
Pranshu-S 4.7 hours ago:	Not really, the older implementation was a bit twisted with consumers. This one with the non-casted way of doing things reads well and is easier for any reader to ... What do you think?

#16790 PR: Vectorize 3 loops in OptimizedScalarQuantizer

3.5 hours ago 0 comments 0 votes 0 watches github-actions[bot], slow-J

Starting as a draft, as I want to make sure this approach is valid first. I was looking into HNSW indexing performance (also https://github.com/apache/lucene/pull/16631 ) ... OptimizedScalarQuantizer.scalarQuantize centers the vector, then runs a coordinate descent to pick ...

«
1
2
3
4
5
»

2,659 issues

[21.5 msec search, 22.4 msec total]