Lucene 2.3.1 阅读学习(41)

xiaoxiao2022-06-16  39

当执行Hits htis = search(query);这一行代码的时候,到底中间经过了怎样的过程,最终使得我们获取到了含有检索结果的集合Hits hits呢?

这里,以最简单的检索为例,追踪并理解Lucene(2.2.0版本)获取到检索结果的过程。

1、 IndexSearcher继承自Searcher类的最简单的search方法,如下所示:

public final Hits search(Query query) throws IOException {     return search (query, (Filter)null); }

这里,调用了Searcher类的带有两个参数的search方法,返回Hits,如下所示:

public Hits search(Query query, Filter filter) throws IOException {     return new Hits (this, query, filter); }

这时,是通过根据传递进来的Query、Filter(简单地考虑,Filter=null)和实例化的Searcher来构造一个Hits的,其实,没有那么简单,想也能想得到,一定是在Hits类中,通过Searcher实例进行了具体的检索过程。

Hits类拥有如下成员:

private Weight weight; private Searcher searcher; private Filter filter = null; private Sort sort = null;

private int length;      // 检索到的匹配的Document的个数 private Vector hitDocs = new Vector();   // 最终将检索命中的Document(封装到Hit中,包含了一个Document的丰富信息)放到hitDocs中,取出来呈现给检索的用户

private HitDoc first;         // head of LRU cache private HitDoc last;          // tail of LRU cache private int numDocs = 0;      // number cached private int maxDocs = 200;    // max to cache

2、 在Hits类中,只有构造好了含有检索结果的Hits才是最终的检索目标,如下所示:

Hits(Searcher s, Query q, Filter f) throws IOException {     weight = q.weight(s);     searcher = s;     filter = f;     getMoreDocs (50); // 缓存检索结果50*2=100个 }

因为一个Query实例需要被多次使用,故而使用Weight来记录Query的状态。重点看在Hits类调用getMoreDocs方法,该方法实现如下:

private final void getMoreDocs(int min) throws IOException {     if (hitDocs.size() > min) {    // hitDocs.size() <min=50,说明缓存的结果数量小于预计要加入到缓存中的结果的数量,因此不执行该if分支,无需改变min(缓存容量)的大小       min = hitDocs.size();     }

    int n = min * 2; // 由50变成100,目的是:直接扩充缓存为原来的2倍,缓存双倍的检索结果以备用户直接从缓存中提取检索结果;间接地一次性为HitQueue(优先级队列)分配大小,防止频繁分配增加系统开销     TopDocs topDocs = (sort == null) ? searcher.search (weight, filter, n) : searcher.search(weight, filter, n, sort);     length = topDocs.totalHits;     ScoreDoc[] scoreDocs = topDocs.scoreDocs;

    float scoreNorm = 1.0f;         if (length > 0 && topDocs.getMaxScore() > 1.0f) {       scoreNorm = 1.0f / topDocs.getMaxScore();     }

    int end = scoreDocs.length < length ? scoreDocs.length : length;     for (int i = hitDocs.size(); i < end; i++) {       hitDocs.addElement(new HitDoc(scoreDocs[i].score * scoreNorm,                                     scoreDocs[i].doc));     } }

第一次调用该方法的时候,作为Hits类的一个成员private Vector hitDocs = new Vector();,它是用作缓存检索结果的。

当sort == null)的时候,执行searcher.search(weight, filter, n),这时,又回到了IndexSearcher类中,执行含有三个参数的重载search方法(见下文),返回检索结果存放于TopDocs topDocs中。关于TopDocs有必要熟悉一下:

public class TopDocs implements java.io.Serializable { public int totalHits; public ScoreDoc[] scoreDocs; private float maxScore; public float getMaxScore() {       return maxScore; } public void setMaxScore(float maxScore) {       this.maxScore=maxScore; } TopDocs(int totalHits, ScoreDoc[] scoreDocs, float maxScore) {     this.totalHits = totalHits;     this.scoreDocs = scoreDocs;     this.maxScore = maxScore; } }

totalHits成员是本次查询Query检索到的Hits的总数;maxScore成员是最大分数。

ScoreDoc[]成员就是从所有满足查询Query的得分不为0的Document的数组,根据要求选择前n个Docunent返回,它包含Document的编号和得分这两个成员:

package org.apache.lucene.search; public class ScoreDoc implements java.io.Serializable { public float score; public int doc; public ScoreDoc(int doc, float score) {     this.doc = doc;     this.score = score; } }

接着,根据返回的TopDocs topDocs取得其长度length = topDocs.totalHits;用来计算得分标准化因子,以对检索结果按照得分进行排序,并把相关的检索结果缓存到hitDocs中。

3、 在IndexSearcher类中,根据searcher.search(weight, filter, n),执行public TopDocs search(Weight weight, Filter filter, final int nDocs)方法,如下所示:

public TopDocs search(Weight weight, Filter filter, final int nDocs)        throws IOException {

    if (nDocs <= 0) // nDocs=n=2*50=100       throw new IllegalArgumentException("nDocs must be > 0");

    TopDocCollector collector = new TopDocCollector(nDocs);     search (weight, filter, collector);     return collector.topDocs(); }

上面,构造了一个TopDocCollector实例,使用如下构造方法:

public TopDocCollector(int numHits) {     this(numHits, new HitQueue(numHits)); }

调用重载的构造方法TopDocCollector(int numHits, PriorityQueue hq),初始化一个Hit的优先级队列,大小为100。其实,关于Hit的优先级队列就是这样的:对于ScoreDoc hitA和ScoreDoc hitB,如果hitA.score>=hitB.score,则选择hitA入队列。

根据构造的TopDocCollector collector,再次调用IndexSearcher类的另一个不同的重载的search方法search(weight, filter, collector);,将检索结果存放到了TopDocCollector collector中,最后根据TopDocCollector collector返回TopDocs。

关于search(weight, filter, collector);调用,实现如下所示:

public void search(Weight weight, Filter filter,                      final HitCollector results) throws IOException {     HitCollector collector = results;     if (filter != null) {    // 我们一直设定Filter=null,该分支不执行       final BitSet bits = filter.bits(reader);       collector = new HitCollector() {           public final void collect(int doc, float score) {             if (bits.get(doc)) {                  // skip docs not in bits               results.collect(doc, score);             }           }         };     }

    Scorer scorer = weight.scorer (reader);     if (scorer == null)       return;     scorer.score (collector); }

先看Scorer scorer,通过weight的scorer方法获取到一个Scorer,在内部类org.apache.lucene.search.TermQuery.TermWeight中可以看到TermWeight的score方法如何实现的:

    public Scorer scorer(IndexReader reader) throws IOException {       TermDocs termDocs = reader.termDocs (term);

      if (termDocs == null)         return null;

      return new TermScorer(this, termDocs, similarity,reader.norms(term.field()));     }

通过IndexReader reader获取到一个TermDocs,其中TermDocs是与检索相关的(即查询中使用Term构造了查询)包含Term的所有<document, frequency>对,document为Document的编号,frequency为Term在该Document中出现的频率(次数)。

执行reader.termDocs(term);时,调用了IndexReader类的termDocs方法,如下所示:

public TermDocs termDocs(Term term) throws IOException {     ensureOpen();     TermDocs termDocs = termDocs();     termDocs.seek(term);     return termDocs; }

TermScorer是Scorer的子类,它有如下一些成员:

private Weight weight; private TermDocs termDocs; private byte[] norms; private float weightValue; private int doc;

private final int[] docs = new int[32];         // buffered doc numbers private final int[] freqs = new int[32];        // buffered term freqs private int pointer; private int pointerMax;

private static final int SCORE_CACHE_SIZE = 32; private float[] scoreCache = new float[SCORE_CACHE_SIZE];

通过这些成员,我们能够知道返回一个TermScorer对象都包含哪些内容,从而为使用Scorer scorer = weight.scorer(reader);便于理解。

weight.scorer(reader) 返回的是一个Scorer,它是用来迭代得分与查询匹配的Docuemnt的。

最后,执行scorer.score(collector);,最终的检索结果可以直接从一个HitCollector collector中提取出来,返回最终的检索结果。

Scorer类的score方法如下:

public void score(HitCollector hc) throws IOException {     while (next()) {       hc.collect (doc(), score());     } }

再看TopDocCollector(extends HitCollector)类的collect方法的实现:

public void collect(int doc, float score) {     if (score > 0.0f) {       totalHits++;       if (hq.size() < numHits || score >= minScore) {         hq.insert(new ScoreDoc(doc, score));         minScore = ((ScoreDoc)hq.top()).score; // maintain minScore       }     } }

根据Document的编号和得分,调整Hit的优先级队列,获得满足条件的Docuemt的集合,最终构造出ScoreDoc,实际上是得到了一个ScoreDoc[]数组,从而访问TopDocs的这个ScoreDoc[]成员,就返回了最终检索的结果集合。

转载请注明原文地址: https://www.6miu.com/read-4940944.html

最新回复(0)