Milos Rusic

We are thrilled to announce that deepset has been recognized in Sifted's B2B SaaS Rising 100 for 2024. This prestigious list highlights the most promising software-savvy startups in Europe and Israel, showcasing innovation and rapid growth in the B2B SaaS sector.

Being named in the Sifted B2B SaaS Rising 100 is a testament to the hard work and dedication of our entire team on our mission to help all enterprises adopt Generative AI. This recognition fuels our motivation to continue pushing the boundaries of what’s possible in the industry.

Since our founding in 2019, we have focused on providing mission-critical AI solutions for organizations, helping businesses transform their customer experiences and operations through enterprise-grade LLM and NLP technology. Our inclusion in this list is a reflection of our commitment to product excellence, customer success, and continuous innovation.

As we move forward, we remain dedicated to our mission of empowering businesses with top-tier AI solutions. With our leading enterprise platform, deepset Cloud, and our renowned open-source framework Haystack, our customers and community are able to leverage AI to transform their products and productivity workflows to transform customer experiences, streamline internal processes, and accelerate insight. Our leadership in the AI field is demonstrated through numerous successful customer deployments, showcasing our commitment to innovation, reliability, and customer success. We are excited about the future and the opportunities that lie ahead as we continue to innovate and grow.We are excited about the future and the opportunities that lie ahead as we continue to innovate and grow.

Thank you to our customers, partners, and team members for their unwavering support. Together, we are shaping the future of enterprise-grade AI.

The Sifted B2B SaaS Rising 100 is an annual ranking of the most promising European and Israeli B2B SaaS startups. This year’s list includes companies launched in 2017 or later, with valuations between $100-999 million. The ranking is based on data-driven scouting and evaluations by Sifted's editorial and research teams, reflecting trends in AI and data-driven tools in the VC landscape.


For more details, you can read the full report

deepset is featured in Sifted's list of Europe's most promising startups in B2B SaaS.

Celebrating Our Recognition in the 2024 Sifted B2B SaaS Rising 100

The Virtual NAACL 2021 Conference took place last week with more than 2500 attendees and 500 papers about recent advances in computational linguistics and natural language processing. If you’d like to dive deep into the research papers, check out the full proceedings — but you can also just start with a summary of 10 short lessons we learned at the conference!

In his keynote, Hinrich Schütze talked about how humans often learn from task descriptions rather than from a large numbers of training samples. This insight raises the question whether we can integrate task descriptions into the training of NLP models. The idea is to translate task descriptions into something that can be used as a task for masked language model training. For example, the task of anaphora resolution, a subtask of coreference resolution, can be expressed as a task to find the masked word in the following three sentences: “Sally arrived. Nobody saw her. In the previous sentence, the pronoun “her” refers to MASK.”

The pattern that describes the task is “In the previous sentence, the pronoun “p” refers to MASK.” The authors call their approach PET (Pattern Exploiting Training) and show that PET leverages task descriptions for better few-shot learning. Interestingly, the added benefit of task descriptions becomes smaller with an increasing number of training samples. With 1000 samples, there is almost no difference. The slides used in the talk are published here.

2. Relevant, Irrelevant and the Third Class of Document in Ranking

In the tutorial “Pre-trained Transformers for Text Ranking: BERT and Beyond” by Andrew Yates, Rodrigo Nogueira and Jimmy Lin, we learned there can be more types of documents in a ranking problem than relevant and irrelevant ones. Ranking documents about COVID-19 as in the TREC-COVID collection should consider a third type of documents that are misleading because of misinformation. Showing these misleading documents as top ranking results would be even worse than showing irrelevant documents. This insight needs to be taken into account when training and evaluating ranking models.

The tutorial also made us aware of “Approximate nearest neighbor Negative Contrastive Estimation” (ACNE), which is an alternative to Dense Passage Retrieval (DPR). Both approaches use positive samples and hard negative samples to train a retrieval model. DPR chooses hard negatives that are semantically similar to positives, for example, with BM25. In contrast to that, ANCE chooses hard negatives that are specific to the model that is being trained. This selection of hard negatives is updated during the learning process so that the model is shown harder negatives as training progresses. A 155-pages written summary of the tutorial is on arXiv. Great work!

3. Registered Reports for Slow Science in Natural Language Processing

The paper “Preregistering NLP research” by Emiel van Miltenburg et al. won the Best Thematic Paper Award — сongratulations! The paper starts a discussion on preregistration, which means to describe the steps and expectations of a research project before carrying it out. It is rarely seen in NLP so far, but it would be great if our community could change that. A preregistrations answers questions that are good to ask oneself when starting a research project, and we think it can be especially helpful for PhD students who just got started. Example questions are “How will you define outliers in your data and what rules will you apply for excluding observations?” or “How will you measure output quality?” or “What kinds of errors do you expect to find with an error analysis?”. If you are interested in this topic, we recommend watching the video on YouTube. Don’t forget to check out the winners of the other awards at NAACL 2021 here.

4. What if Companies Had Their Own Word Embedding?

Besides the main research track, NAACL also had an industry track. “Query2Prod2Vec: Grounded Word Embeddings for eCommerce” by Federico Bianchi et al. won the best industry paper award in this track. It’s about language grounding, where the meaning of the product names is extracted from eCommerce interactions. Search queries in an online shop and the items a user clicks in the search results help to learn better product embeddings than using only the product name and word2vec. As a result, similar products are closer to each other in the embedding space. Unfortunately, code and data haven’t been published yet.

5. Ask Questions on Tables, Not Just Text

Dense retrieval and question answering not only work on texts but also on tables. “Open Domain Question Answering over Tables via Dense Retrieval” by Jonathan Herzig et al. presents an efficient dense table retriever based on the TAPAS encoder by the same authors and demonstrates that end-to-end question answering of a BERT-based retriever can be improved from from 33.8 to 37.7 exact match. It’s great to see that they published code and models and we are sure that somewhere, someone is already building on their results continuing this research direction.

Splitting up a text document into passages and interpreting them separately sometimes creates the problem where context from one passage is needed to understand another passage. Eunsol Choi et al. address this problem with their paper “Decontextualization: Making Sentences Stand-Alone”. For example, imagine the following paragraph, where the last sentence needs to be decontextualized.

Paragraph: “Croatia national football team have appeared in the FIFA World Cup on five occasions. Their best result thus far was reaching the 2018 final.”

Decontextualized Sentence: “The Croatia national football team’s best result thus far in the FIFA World Cup was reaching the 2018 final.”

The paper has been published earlier in the TACL journal, which gives authors the chance to present their work also at a conference. Thereby, TACL combines the advantages of the reviewing process of a journal with options for revisions besides strict accept/reject world and being part of a conference.

Guanghui Qin and Jason Eisner presented their paper “Learning How to Ask: Querying LMs with Mixtures of Soft Prompts”, which relates to the patterns mentioned in the keynote by Hinrich Schütze. The paper describes how to extract knowledge from language models with the fill-in-the-blank paradigm. They extend the idea of using words in prompts (questions to the language model) to “soft prompts” that do not necessarily consist of real words but “soft words.” These “soft words” are embeddings that do not correspond to an actual word in the vocabulary and the most effective soft prompts can be learned. A picture is worth a thousand words, so have a look at this slide. Last but not least — сongratulations on the Best Short Paper Award!

The survey paper “A Primer in BERTology: What We Know About How BERT Works” by Anna Rogers et al. summarizes еру findings from more than 150 studies on BERT models. For example, “BERT takes subject-predicate agreement into account when performing the cloze task” is a finding that is then put into context with references to related work. What’s also great about this paper is that all key findings are highlighted in bold font, making it easier to read.

9. Explainability and Interpretability of Question Answering Models

There were several birds-of-a-feather sessions and meetups on question answering. A personal highlight was a discussion with Greg Durrett, Behzad Golshan and Yufang Hou about explainability and interpretability of question answering models. Techniques like pruning, perturbation, integrated gradients, and saliency maps have been used to make text classifiers interpretable and we were talking about how to apply these techniques to question answering models. We agreed that they have some obvious limitations but are worth trying out. This is a nice research topic and we are hoping to see more papers on that in the upcoming conferences.

10. Virtual Conferences — Here to Stay?

Most of you probably already had your own experiences with a variety of virtual conferences during the past year. It was great to see how nicely everything worked out at NAACL. With whova, underline, gather.town, and Zoom, the only challenge was about the time zones differences, but also avoiding to get lost in the virtual conference space while searching for the right session. With SIGIR and ACL, there are two more fully virtual conferences coming up in the next few months. With the right tools we are pretty confident these events will be a success. We’re also very much looking forward to attending EMNLP in a “hybrid” format in November.

A summary of the Virtual NAACL 2021 Conference with 2500 attendees and 500 papers about recent advances in computational linguistics and natural language processing.

10 Things We Learned at NAACL’21 | deepset

Search permeates our lives. Maybe we’re trying again to find the apartment keys, or perhaps it’s about searching for an affordable flight on the web — search is and will always be present. Especially in our digital lives, it is difficult to deal with the ever-increasing volumes of information. What we’re looking for is often hidden deep inside all sorts of poorly organized text data. It becomes even more challenging within an enterprise environment where looking for an important piece of information feels more and more like searching for a needle in a haystack.

Search engines work by crawling the web or a local document database. They then output the information deemed most relevant to a given query, based on many proprietary ranking factors. The most basic search method is keyword-based. It returns the search results with the highest lexical overlap with the query. Over the years, popular web search engines like Google and Bing have been pushing for a more semantic search engine that could better understand user intent. Understanding the intent behind a search query goes a long way towards providing more relevant search results.

One way to provide search engines easier access to the contextual meaning of information is by structuring the data in a knowledge graph. Knowledge graphs help search engines to better recognize user intent and context by understanding how the entities in a database are related to one another.

Semantic search solutions, such as knowledge graphs and natural language processing (NLP) techniques, have allowed search engines to handle unstructured textual data and match documents to queries based on semantics rather than lexical overlap. A hybrid search engine can use keyword-based search alongside knowledge graphs and semantic search to get the best of all worlds.

Search tools, often based on simple keywords, are integrated into almost all modern applications. These search tools have saved us much time and effort to find information in our often overwhelming enterprise databases, wikis, websites, or applications. However, in many cases, they still fail. Instead of providing a particular piece of knowledge or pointing in the right direction, these traditional tools fail to meet the searcher's intent.

Lately, modern technologies such as Natural Language Processing (NLP) have enabled a new paradigm of search and offer the power to massively improve our search experience. In short, NLP will improve search accuracy. In particular, NLP allows us to shift, where applicable, from elaborate keyword-based matching towards the semantic understanding of our queries and questions. This way, we do not necessarily have to remember specific keywords or terminology to retrieve the desired information. Semantic search technologies such as Question Answering (QA) enable us to query text data by simply typing our questions into the search bar, then getting full answers rather than just a bunch of links to the documents matching the keywords.

However, when does semantic search enhance our search experience, and when would a keyword-based search be sufficient?

Based on the learnings from our clients and the open source community, we’d argue it all depends on the search intent. Almost every search query is triggered by an intent. We do not search because searching is so much fun. We search because we aim to meet a specific objective. For example, we may need to find a relevant website, explore an interesting topic, come up with an important decision, purchase a much-needed item, or maybe we’re just learning a new language. These intentions make the search either informational or navigational by nature.

Navigational search queries are driven by the need to be directed to a certain point on the Internet or inside the enterprise database. We use navigational search when we have a general idea about what we need to find, but we do not know a direct path and use the “navigation” to suggest one place after another.

This means when conducting a navigational search within enterprise databases our intent is similar to checking “points of interest” in a car navigation system or perhaps entering an incomplete address step-by-step by following suggestions.

In some cases, we also use navigational search to browse for a collection of items. For example, when searching for “project management tools,” we aim to investigate various applications to identify and select the one that fits best. We have an idea about the “destination point,” and we’d like the search engine to suggest various options to pick from.

In many cases, keywords are still sufficient for navigational search.

Informational search is something different. It is usually about the intent of accessing the knowledge directly and right away — it’s not about browsing a collection of potentially relevant results from a keyword-based search. Informational search is also usually stemming from having a certain question in mind.

We often do not know where to look for that particular result. We may be lacking specific terminology to describe the desirable answer, especially when being a newcomer to the knowledge area. Because we don’t know in advance what “keywords” the answer will contain, we can’t quite rely on a keyword-based search in this situation.

Our experience of helping enterprise customers with the above suggests that a semantic search can overcome the limits of a keyword-based approach and drastically improve informational search results.

The creation of and interaction with the knowledge is a major part of an enterprise environment. Many efforts have been made to preserve the knowledge by accumulating reports, executive summaries, meeting protocols, and other unstructured data. But as organizations grow, the amount of such data grows beyond being manageable. Finding a piece of information becomes much less feasible unless we put search terms into context.

Think maybe of someone who wants to learn about the applications for Transformer-based NLP models and what they have been used for within the company. It is unlikely that putting together a search query containing just the keywords “transformer,” “applications,” and “built” will lead to meaningful information — the context has been lost here.

In turn, semantic search allows us to directly leverage that specific question that we wanted to “apply” against the text data, for example, by asking, “What to build with transformers?”

However, knowledge in the enterprise context also goes beyond just quickly solving yet another 80/20 rule and pinpointing those twenty percent of documents containing the most relevant answers. Managing the knowledge within the enterprise is also about finding people — for instance, the experts in a certain field.

The vast amount of documentation about previous projects often speaks of not just what has been done but also about who has done it. With a semantic search engine, it is unnecessary to read the reports one-by-one to identify people with specific expertise. The search will tell us exactly just who’s got the experience we are interested in.

To illustrate this, in the example below, we are looking for someone “Who has experience with Transformers?”

Natural language processing enables search tools with a semantic understanding of text data and the queries. They, therefore, serve as a natural language interface to interact with enterprise knowledge. These semantic search engines will no longer offer just a simple “keyword matcher.”

Instead, they provide an interface to interact with information in all formats bringing us closer to much more generic and flexible virtual assistants. One particular driver for this kind of search will also be the pervasiveness of conversational AI — chatbots and voice assistants.

We believe all of the above will lead further away from using rigid keywords and will push the search systems to interpret the most powerful communication protocol — human language.

Semantic search makes it easier for you to find what you are looking for. Rather than trying to find the keyword combination that will best meet your search intent, semantic search allows you to phrase questions in a natural manner, even when you don’t exactly know what you are after. For example, if you are thinking of an old movie but cannot remember anything about it other than it was based on some board game, you can search Google for “what’s the movie about the board game.” It will return top results for several movies based on games such as Jumanji, Zathura, Ouija, Battleship, Clue, etc. The next top result is a link to a list of movies based on board games. In addition, Google uses semantic search to gauge user intent and provide factual answers directly on the search page, via the Google Knowledge Graph or featured snippets that show film posters. So not only do they link to sites that might have an answer elsewhere, they try to provide an answer directly to you by inferring what you meant. In a keyword-based search, the results would likely just include links about board games or movies. A semantic search, with a Transformer-based model, is more intuitive and conversational between the user and search engine.

On YouTube, searching for “the price of fish,” if that’s all you can remember about some song from the 90’s, returns a documentary of the same name as well as Scooter’s “How Much Is The Fish?” music video. Here, YouTube clearly recognizes the meaning and variations of a word such as “price.” Additionally, YouTube also uses search history and subscriptions, like if you subscribe to channels that feature 90’s music videos, to evaluate the relevance of what results it should provide. New machine learning-driven models, like the Transformer-based methods used by Google and YouTube, create representations of text that are semantically meaningful, giving web or enterprise search engines an understanding of language closer to how people actually use it. Ultimately, this creates a closer connection between you and all of the information at your disposal.

Over the last couple of years, we’ve developed Haystack — our open source NLP framework to build end-to-end question answering systems and semantic search pipelines.

Haystack offers the most practical way of implementing information-intensive search workflows by providing a natural language interface for unstructured data. It’s aimed at developers who just need the job done while bridging the gap between the most recent achievements in NLP and the real-world user requirements.

We want to thank everyone who’s interested in Haystack and has been supporting us through our journey. We’d encourage you to join our friendly community, and please help us spread the word by starring the repository and following us on Twitter.

Stay tuned for more insights around Haystack!

In the near future, will we interact with data using only natural language?

Should we settle for a keyword-based search? Lately, modern NLP technologies such as question answering and neural search have transformed the modern search landscape.

Celebrating Our Recognition in the 2024 Sifted B2B SaaS Rising 100

More to read

Build Powerful AI Applications with Haystack by deepset & NVIDIA NIM

Announcing Our New Funding Round Led by Balderton Capital