Using Haystack with Elasticsearch as Document Database

Developers love Elasticsearch (ES) for its ease of use, scalability, and the speed with which it returns keyword-based search results, even from large datasets. But did you know that you could leverage the latest Transformer models to add semantic search capabilities to your ES index? Read on to learn how Haystack NLP framework brings semantic-based search to your Elasticsearch document database.

What Is Haystack?

Haystack is a Python framework for natural language processing (NLP) that focuses on semantic search and question answering (QA). Haystack enables the users to ask a question in natural language, and receive an informative answer from a large collection of documents — all within seconds.

Haystack is highly composable — it offers versatile NLP building blocks that you easily can combine into question answering and semantic search pipelines. For instance, with just a few lines of code, you can set up ready-made pipelines for use cases like semantic FAQ search or extractive question answering. Alternatively, you can build custom pipelines with nodes that leverage state-of-the-art NLP models for summarization, translation, text generation, and much more. 

Get More Out of Elasticsearch with Haystack

When you use Haystack on top of Elasticsearch, you get the best of both worlds. The engine’s storage and keyword search capacities work well with the latest Transformer-based language models to offer a powerful search experience for the enterprise end-users. At deepset, we took care to make the developer experience enjoyable as well, namely through the following features:

• Advanced NLP techniques like dedicated retriever and reader models for optimal speed and accuracy;
• A modular structure that lets you use only the pieces you need and seamlessly add more components to an NLP pipeline;
• Easy service deployment and integrations through REST API.

Try out Haystack with Elasticsearch

If you’re ready to get started, here is a quick guide to setting up an Elasticsearch data store in Haystack. 

To implement a simple question answering system with Elasticsearch and Haystack, check out our detailed guide on Supercharging Elasticsearch with Haystack.