What is Hybrid Query?

Simply put, it combines half EE syntax with half ElasticsearchClient syntax, similar to a "hybrid electric vehicle." You're likely to fall in love with this "hybrid" mode as it integrates the advantages of both approaches!

Why Have a Hybrid Query?

Since EE hasn't yet fully covered all ElasticsearchClient features (currently around 90% of APIs and 99.9% of core functionalities after a year of development), there are occasional scenarios where EE can't meet specific needs. In such cases, secondary development of EE or submitting requests to its developers might not be timely enough, especially for urgent product manager demands. Here, hybrid queries offer a solution.

Moreover, API design faces a trade-off between functionality and usability. ElasticsearchClient's complex API, with thousands of parameter combinations, can't be simplified through method overloading without becoming bloated and hard to navigate. We've simplified only the 99.9% of high-frequency core APIs, leaving low-frequency ones to be implemented via hybrid queries.

What are the Advantages of Using Hybrid Queries Over Native Queries?

Like how hybrid cars save fuel, hybrid queries save code and are simpler than native queries. They combine the flexibility of native syntax with EE's low-code benefits. SpringData-Es, the pioneer of hybrid queries, is a good reference, but our approach is even more flexible and efficient. Don't believe it? Keep reading!

How to Use Hybrid Queries?

Before this document, despite providing hybrid query APIs and a brief introduction, many were unaware of this feature or how to use it. So, I'll demonstrate with a specific example for your reference. Don't worry about the length; it's actually very simple.

Background

User "Xiangyang" reported via WeChat that EE doesn't support sorting queries by proximity to a given location. In development, this can apply to ride-hailing scenarios where a passenger requests the nearest driver within 3 km. The passenger, a woman concerned about safety, added requirements like the driver being female, with over 3 years of driving experience, and a business-type vehicle...

Using the ride-hailing scenario, let's see how to query with EE. The query can be divided into two parts:

• Conventional queries supported by EE: within 3 km, driver gender is female, driving experience >= 3 years...
• Unconventional queries not supported by EE: sorting by complex rules (unsupported when writing this document, but now supported; however, this is just for demonstrating hybrid query usage)

For the supported part, we can directly call EE to build a Query:

// Assuming the passenger's location is 31.256224D, 121.462311D
LambdaEsQueryWrapper<Driver> wrapper = new LambdaEsQueryWrapper<>();
wrapper.geoDistance(Driver::getLocation, 3.0, DistanceUnit.KILOMETERS, new GeoPoint(31.256224D, 121.462311D))
       .eq(Driver::getGender,"Female")
       .ge(Driver::getDriverAge,3)
       .eq(Driver::getCarModel,"Business car");
Query originalQuery = documentMapper.getSearchBuilder(wrapper)
        .build()
        .query();

For unsupported statements, we can encapsulate them using ElasticsearchClient syntax and then call EE's native query interface:

        SearchRequest finalRequest = SearchRequest.of(s -> s
            .index("index name")
            .query(originalQuery) // Inherits the Query encapsulated above
            .sort(so -> so
                    .geoDistance(g -> g
                        .field("location")
                        .location(l -> l.latlon(ll -> ll
                        .lat(31.256224)
                        .lon(121.462311)
                    ))
                    .order(SortOrder.Desc)
                    .unit(DistanceUnit.Kilometers)
                    .distanceType(GeoDistanceType.Arc)
                    )
            )
        );
        SearchResponse<Document> response = documentMapper.search(finalRequest, new RestClientOptions(RequestOptions.DEFAULT));
        List<Hit<Document>> hits = response.hits().hits();
        // TODO Other subsequent business processing, omitted

Thus, you can enjoy EE's basic query generation and implement unsupported features with minimal code (still saving大量 amounts of code compared to direct ElasticsearchClient usage), similar to how hybrid cars offer a compromise while pure electric vehicles aren't fully mature.

Of course, if you're not comfortable with this mode, you can still use native queries directly. Feel free to use EE worry-free, as we've prepared various fallback options and exit strategies for you, with无忧 after-sales service! If you认可 this approach, consider giving us a star. The author, albeit an old man, has truly tried his best to make EE users happy!

提示

Didn't understand the example above? No problem. The real game-changer was introduced in version 2.0.0-beta2 with a brand-new hybrid query solution, even simpler to use. You can continue reading below; Case "0" is exactly what you're longing for!

Several Correct Ways to Use Hybrid Queries

Although there are many examples below, they are very simple to use. Don't be intimidated by the detailed tutorials.

    /**
     * Correct Usage 0 (Most practical, simplest, and recommended): Use EE-supported syntax directly. For unsupported cases, construct a native QueryBuilder and pass it via wrapper.mix.
     * @since 2.0.0-beta2 Officially introduced in 2.0.0-beta2, this is the optimal hybrid query solution. As QueryBuilder covers all ES queries, this approach can theoretically handle any complex query and seamlessly integrate with EE's four nested types, simplifying queries and boosting productivity!
     */
    @Test
    public void testMix0(){
        // Query data with title "Old Man," content matching "Tui*", and a minimum match score of 80%
        // Our out-of-the-box match doesn't support setting the minimum match score, so we construct a matchQueryBuilder
        LambdaEsQueryWrapper<Document> wrapper = new LambdaEsQueryWrapper<>();
        QueryBuilder queryBuilder = QueryBuilders.matchQuery("content", "Tui*").minimumShouldMatch("80%");
        wrapper.eq(Document::getTitle,"Old Man").mix(queryBuilder);
        List<Document> documents = documentMapper.selectList(wrapper);
        System.out.println(documents);
    }

    /**
     * Correct Hybrid Query Usage 1: When EE lacks support for fine-grained features, construct all query conditions with native syntax and only utilize EE's data parsing functionality.
     */
    @Test
    public void testMix1() {
        // ElasticsearchClient native syntax
        SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
        searchSourceBuilder.query(QueryBuilders.matchQuery("content", "Tui*").minimumShouldMatch("80%"));

        // Only leverage EE's query and data parsing capabilities
        LambdaEsQueryWrapper<Document> wrapper = new LambdaEsQueryWrapper<>();
        wrapper.setSearchSourceBuilder(searchSourceBuilder);
        List<Document> documents = documentMapper.selectList(wrapper);
        System.out.println(documents);
    }


    /**
     * Correct Hybrid Query Usage 2: When all features except the sorter are supported, construct the required SortBuilder using ES native syntax and complete the rest with EE.
     */
    @Test
    public void testMix2() {
        // EE-supported syntax
        LambdaEsQueryWrapper<Document> wrapper = new LambdaEsQueryWrapper<>();
        wrapper.eq(Document::getTitle, "Old Man")
                .match(Document::getContent, "Tui*");

        // ElasticsearchClient native syntax
        Script script = new Script("doc['star_num'].value");
        ScriptSortBuilder scriptSortBuilder = SortBuilders.scriptSort(script, ScriptSortBuilder.ScriptSortType.NUMBER).order(SortOrder.DESC);

        // Use EE for querying and data parsing
        wrapper.sort(scriptSortBuilder);
        List<Document> documents = documentMapper.selectList(wrapper);
        System.out.println(documents);
    }

    /**
     * Correct Hybrid Query Usage 3: When all features are supported except for appending non-query parameters to SearchSourceBuilder.
     */
    @Test
    public void testMix3() {
        // EE-supported syntax
        LambdaEsQueryWrapper<Document> wrapper = new LambdaEsQueryWrapper<>();
        wrapper.eq(Document::getTitle, "Old Man")
                .match(Document::getContent, "Tui*");
        SearchSourceBuilder searchSourceBuilder = documentMapper.getSearchSourceBuilder(wrapper);

        // Append or set parameters supported by SearchSourceBuilder but not yet by EE. Avoid appending query parameters, as they will overwrite the query EE has generated, with the last set query taking precedence.
        searchSourceBuilder.timeout(TimeValue.timeValueSeconds(3L));
        wrapper.setSearchSourceBuilder(searchSourceBuilder);
        List<Document> documents = documentMapper.selectList(wrapper);
        System.out.println(documents);
    }


    /**
     * When most basic query conditions can be met but EE's aggregation features fall short, custom aggregations are required.
     */
    @Test
    public void textMix4() {
        // EE-supported syntax
        LambdaEsQueryWrapper<Document> wrapper = new LambdaEsQueryWrapper<>();
        wrapper.eq(Document::getTitle, "Old Man")
                .match(Document::getContent, "Tui*");
        SearchSourceBuilder searchSourceBuilder = documentMapper.getSearchSourceBuilder(wrapper);

        // ElasticsearchClient native syntax
        AggregationBuilder aggregation = AggregationBuilders.terms("titleAgg")
                .field("title");
        searchSourceBuilder.aggregation(aggregation);
        wrapper.setSearchSourceBuilder(searchSourceBuilder);
        SearchResponse searchResponse = documentMapper.search(wrapper);
        // TODO Aggregated information is dynamic and cannot be parsed by the framework. Users need to parse it from the bucket based on the aggregation type, referring to the official ElasticsearchClient Aggregation parsing documentation.
    }

Incorrect Ways to Use Hybrid Queries

Let me tell a story. Once upon a time, there was a lazy man who didn't like to clean his feet. One day, a diligent old man, seeing him homeless, offered to build him a luxury house for comfort. The old man even showed him the door, but the lazy man ignored it, insisting on climbing in through the skylight, thinking it was cool. Unfortunately, he fell and broke his leg. He then sued the old man for faulty house construction and cursed him on the streets as a rotten old man...

I have listed all supported hybrid query usage scenarios in the correct approaches above, which cover any usage scenario. Please don't fabricate or imagine your own ways without consulting the documentation or source code. If you end up asking why it's not supported or claiming there are bugs in the code, it will tarnish Comrade Ma Baoguo's reputation...

Below are two typical unsupported scenarios:

    /**
     * Unsupported Hybrid Query 1: Overwrite Issues
     */
    @Test
    public void textNotSupportMix() {
        // EE-supported syntax
        LambdaEsQueryWrapper<Document> wrapper = new LambdaEsQueryWrapper<>();
        wrapper.eq(Document::getTitle, "Old Man")
                .match(Document::getContent, "Tui*");
        SearchSourceBuilder searchSourceBuilder = documentMapper.getSearchSourceBuilder(wrapper);

        // The user wants to append personalized query parameters. However, during execution, the query condition will only be the last-set title=Neighbor Old Wang, overwriting the previous Old Man Tui*.
        searchSourceBuilder.query(QueryBuilders.matchQuery("title", "Neighbor Old Wang"));
        wrapper.setSearchSourceBuilder(searchSourceBuilder);
        List<Document> documents = documentMapper.selectList(wrapper);
        System.out.println(documents);
        // Question: Why is it overwritten? Because technically, it's unachievable. The query tree is already built, and ES doesn't provide an API to append query conditions to specific tree levels.
    }

    /**
     * Unsupported Hybrid Query 2: Self-deception Series
     */
    @Test
    public void testNotSupportMix2() {
        // EE-supported syntax
        LambdaEsQueryWrapper<Document> wrapper = new LambdaEsQueryWrapper<>();
        wrapper.eq(Document::getTitle, "Old Man")
                .match(Document::getContent, "Tui*");

        // The SearchSourceBuilder is self-constructed, not via mapper.getSearchSourceBuilder(wrapper). This is like pulling down your pants to fart; the above query conditions (Old Man Tui*) won't take effect.
        SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
        searchSourceBuilder.minScore(10.5f);
        wrapper.setSearchSourceBuilder(searchSourceBuilder);
        List<Document> documents = documentMapper.selectList(wrapper);
        System.out.println(documents);
    }

Conclusion

Given the vast number of features supported by ES's official RestHighLevelClient, I'm still diligently integrating new functionalities, fixing user-reported issues, and optimizing code performance. However, some features may not yet meet your needs. Please understand, as EE is only a year old and can't be perfect. Give us time—we've made significant progress in the past year without any compensation, and these shortcomings will be resolved. Just as new energy vehicles will eventually replace fuel-powered ones, these so-called issues will cease to be problems with time. Hurrah!