[FIXED] Highlighting in Hibernate Search 6 and Elasticsearch backend

Issue

We're in the process of converting our java application from Hibernate Search 5 to 6 with an Elasticsearch backend.

For some good background info, see href="https://stackoverflow.com/questions/50841758/how-to-do-highlighting-within-hibernatesearch-over-elasticsearch">How to do highlighting within HibernateSearch over Elasticsearch for a question we had when upgrading our highlighting code from a Lucene to Elasticsearch backend and how it was resolved.

Hibernate Search 6 seems to support using 2 backends at the same time, Lucene and Elasticsearch, so we'd like to use Elasticsearch for all our queries and Lucene for the highlighting, if that's possible.

Here is basically what we're trying to do:

    public boolean matchPhoneNumbers() {
        String phoneNumber1 = "603-436-1234";
        String phoneNumber2 = "603-436-1234";

        LuceneBackend luceneBackend =
                Search.mapping(entityManager.getEntityManagerFactory())
                        .backend().unwrap(LuceneBackend.class);

        Analyzer analyzer = luceneBackend.analyzer("phoneNumberKeywordAnalyzer").get();

        //... builds a Lucene Query using the analyzer and phoneNumber1 term     
        Query phoneNumberQuery = buildQuery(analyzer, phoneNumber1, ...);

        return isMatch("phoneNumberField", phoneNumber2, phoneNumberQuery, analyzer);
    }

   private boolean isMatch(String field, String target, Query sourceQ, Analyzer analyzer) {
        Highlighter highlighter = new Highlighter(new QueryScorer(sourceQ, field));
        highlighter.setTextFragmenter(new NullFragmenter());

        try {
            String result = highlighter.getBestFragment(analyzer, field, target);
            return StringUtils.hasText(result);
        } catch (IOException e) {
            ...
        }
    }

What I've attempted so far is to configure two separate backends in the configuration properties, per the documentation, like this:

properties.setProperty("hibernate.search.backends.elasticsearch.analysis.configurer", "com.bt.demo.search.AnalysisConfigurer");
properties.setProperty("hibernate.search.backends.lucene.analysis.configurer", "com.bt.demo.search.CustomLuceneAnalysisConfigurer");
properties.setProperty("hibernate.search.backends.elasticsearch.type", "elasticsearch");
properties.setProperty("hibernate.search.backends.lucene.type", "lucene");
properties.setProperty("hibernate.search.backends.elasticsearch.uris", "http://127.0.0.1:9200");

The AnalysisConfigurer class implements ElasticsearchAnalysisConfigurer and CustomLuceneAnalysisConfigurer implements from LuceneAnalysisConfigurer.

Analyzers are defined twice, once in the Elasticsearch configurer and again in the Lucene configurer.

I don't know why both hibernate.search.backends.elasticsearch.type and hibernate.search.backends.lucene.type are necessary but if I don't include the lucene.type, I get Ambiguous backend type: configuration property 'hibernate.search.backends.lucene.type' is not set.

But if I do have both backend properties types set, I get HSEARCH000575: No default backend. Check that at least one entity is configured to target the default backend, when attempting to retrieve the Lucene backend, like:

Search.mapping(entityManager.getEntityManagerFactory())
                        .backend().unwrap(LuceneBackend.class);

And the same error when trying to retrieve the Elasticsearch backend.

I've also added @Indexed(..., backend = "elasticsearch") to my entities since I wish to have them saved into Elasticsearch and don't need them in Lucene. I also tried adding a fake entity with @Indexed(..., backend = "lucene") but it made no difference.

What have I got configured wrong?

Solution

I don't know why both hibernate.search.backends.elasticsearch.type and hibernate.search.backends.lucene.type are necessary but if I don't include the lucene.type, I get Ambiguous backend type: configuration property 'hibernate.search.backends.lucene.type' is not set.

That's because the backend name is just that: a name. Hibernate Search doesn't infer particular information from it, even if you name your backend "lucene" or "elasticsearch". You could have multiple Elasticsearch backends for all it knows :)

But if I do have both backend properties types set, I get HSEARCH000575: No default backend. Check that at least one entity is configured to target the default backend, when attempting to retrieve the Lucene backend, like:

Search.mapping(entityManager.getEntityManagerFactory())
                        .backend().unwrap(LuceneBackend.class);
``

You called .backend(), which retrieves the default backend, i.e. the backend that doesn't have a name and is configured through hibernate.search.backend.* instead of hibernate.search.backends.<somename>.* (see https://docs.jboss.org/hibernate/stable/search/reference/en-US/html_single/#configuration-structure ).

But you are apparently mapping all your entities to a named backends, one named elasticsearch and one named lucene. So the default backend just doesn't exist.

You should call this:

Search.mapping(entityManager.getEntityManagerFactory())
                        .backend("lucene").unwrap(LuceneBackend.class);

I've also added @Indexed(..., backend = "elasticsearch") to my entities since I wish to have them saved into Elasticsearch

Since you obviously only want to use one backend for indexing, I would recommend reverting that change (keeping @Indexed without setting @Indexed.backend) and simply making using the default backend.

In short, remove the @Indexed.backend and replace this:

properties.setProperty("hibernate.search.backends.elasticsearch.analysis.configurer", "com.bt.demo.search.AnalysisConfigurer");
properties.setProperty("hibernate.search.backends.lucene.analysis.configurer", "com.bt.demo.search.CustomLuceneAnalysisConfigurer");
properties.setProperty("hibernate.search.backends.elasticsearch.type", "elasticsearch");
properties.setProperty("hibernate.search.backends.lucene.type", "lucene");
properties.setProperty("hibernate.search.backends.elasticsearch.uris", "http://127.0.0.1:9200");

With this

properties.setProperty("hibernate.search.backend.analysis.configurer", "com.bt.demo.search.AnalysisConfigurer");
properties.setProperty("hibernate.search.backends.lucene.analysis.configurer", "com.bt.demo.search.CustomLuceneAnalysisConfigurer");
properties.setProperty("hibernate.search.backend.type", "elasticsearch");
properties.setProperty("hibernate.search.backends.lucene.type", "lucene");
properties.setProperty("hibernate.search.backend.uris", "http://127.0.0.1:9200");

You don't technically have to do that, but I think it will be simpler in the long term. It keeps the Lucene backend as a separate hack that doesn't affect your whole application.

I also tried adding a fake entity with @Indexed(..., backend = "lucene")

I confirm you will need that fake entity mapped to the "lucene" backend, otherwise Hibernate Search will not create the "lucene" backend.

Answered By - yrodiere
Answer Checked By - Pedro (JavaFixing Volunteer)