Why a vector Database is a must?
A vector search engine is a type of search engine that uses vector representations, also known
as embeddings, to search for relevant information. Unlike traditional keyword-based search engines, vector search engines use mathematical techniques to represent and process the meaning of words, phrases, sentences, or documents in a high-dimensional vector space. The key idea is that semantically similar items have similar vector representations. Vector search engines are particularly useful for information retrieval tasks, such as document retrieval, question-answer, and recommendation systems, where understanding the meaning and context of the query is essential for finding relevant results. Generated prompts can contain data privacy and protection related information
Learnings from the current use cases shows that Generative AI models, such as large-scale language models, are powerful tools for a wide range of applications, but they may require adaptation to perform optimally on specific tasks or domains. To achieve this, we can employ several model adoption techniques, such as:
Prompt Engineering: This technique involves crafting specific tasks or questions in natural language,
which can help guide the foundation model to generate more accurate and relevant responses. By carefully designing prompts, we can effectively instruct the model to focus on the desired aspects of the
task, improving its overall performance.
Embeddings: Incorporating external knowledge through embeddings can significantly improve a
foundation model’s ability to adapt to domain-specific knowledge. Embeddings represent information in
a numerical format that the model can easily process and learn from. By including domain-specific
embeddings (e.g. example for good code, product documentation) or pre-trained embeddings from
various sources, we can enrich the model’s understanding of the domain and provide it with useful
references to generate more accurate and context-aware outputs.
Fine-Tuning: Another way to adapt foundation models is by fine-tuning their parameters on a small set
of labeled data specific to the target task. This process involves updating the model’s weights using
gradient descent and backpropagation, enabling it to learn the nuances of the task and improve its
performance. Fine-tuning can be particularly effective when dealing with few-shot learning scenarios,
where we have limited labeled data available.
By combining these techniques, we can successfully adapt foundation models to a wide range of tasks and domains, unlocking their full potential and enhancing their performance to meet our specific needs. Those model adoption methods are a new aspect in the application development and is therefore reflected accordingly in the solution architecture depicted in figure 1. For the implementation of Generative AI applications, the following realization patterns are so far identified:
Digital Assistant: For Question-Answer use cases the SAP Digital Assistant should be used. Generative AI
capabilities are going to be incorporated into the underlying SAP Digital Assistant technology.
Applications can follow the already established programming model for SAP Digital Assistant and make
use of Generative AI.
Basic Prompting: ABAP-based applications with elementary requirements regarding prompt engineering
should utilize the planned prompt creation capabilities of the ABAP platform. Predefined prompt templates where just parameters are replaced by concrete values are an example for this category. These
basic prompts are executed on Generative AI models which are hosted externally or deployed on SAP
BTP.
Advanced Prompting: ABAP- or BTP-based applications with sophisticated requirements concerning
prompt engineering should facilitate the planned prompt creation capabilities of SAP BTP. Prompts
including embeddings and require therefore vector search engines are an example for this category.
Prompt execution is also here based on Generative AI models which are hosted externally or deployed
on SAP BTP.
Model Retraining: For applications where the pretrained Generative AI models are not sufficient but
fine-tuning is required, the planned deployment and retraining capabilities of SAP BTP for 3
rd party Generative AI models should be used. The idea is to train the highest layers of the 3rd party models with application specific data. The previous three realization patterns can be then applied on these local Generative AI models.
