AI Engineering - RAG
今年过年前，跟朱帅聊天说到RAG。他说RAG没有未来，因为很多大模型都在进行一些垂直领域的优化。 比如前几天在hugging face上看到的基于RAG的微调模 -- RAFT。
我当时是有点怀疑，因为RAG一直以来似乎都还比较流行，在和波士顿咨询的面试里，我们也聊过一些。
首先明确一下RAG的定义：
- Retrieve: 从大量的文本中检索相关的信息
- Answer: 从检索到的信息中回答问题
- Generate: 生成一些文本
和RAFT的区别是，RAFT是一个基于RAG的模型，用于微调。而RAG是一个框架，可以用于很多不同的任务。
我觉得在人工智能工程领域，RAG还是有很多应用的。 我能想到的是，很多公司都有私有的数据，比如金融公司的交易数据，医疗公司的病例数据，这些数据都是非常有价值的，也不适合公开。 RAG可以帮助我们从这些数据中提取出有价值的信息，比如做风险预测，疾病诊断。
基于RAG的微调模型固然也是有他的优势，比如在一些特定的领域，我们用RAFT来微调，可以使大模型拥有某个垂直领域的能力。
这篇blog会是一个长期更新的blog。
我会把最新的一些关于RAG的资料，以及我自己的一些想法，都会在这里更新。
Fine-tune Embedding models for RAG.
Reference:
https://huggingface.co/blog/train-sentence-transformers
Why?
Embedding models are crucial for successfull RAG applications, since if you don't retrieve the right context you can't generate the right answer. Customizing embedding models for domain-specific data can improve retrieval performance significantly compared to using general knowledge models. Fine-tuning embedding models has become highly accessible, and using synthetic data generated by LLMs, one can easily customize models for specific needs, resulting in substantial improvements.
Matryoshka Embeddings
Matryoshka Representation Learning (MRL) is a technique designed to create embeddings that can be truncated to various dimensions without significant loss of performance. This approach frontloads important information into earlier dimensions of the embedding, allowing for efficient storage and processing while maintaining high accuracy in downstream tasks such as retrieval, classification, and clustering.
For example, a Matryoshka model can preserve ~99.9% of its performance while needing 3x less storage. This is particularly useful for applications where storage and processing resources are limited, such as on-device applications or large-scale retrieval systems.
Main Steps
- Create & Prepare embedding dataset
- Create baseline and evaluate pretrained model
- Define loss function with Matryoshka Representation
- Fine-tune embedding model with SentenceTransformersTrainer
- Evaluate fine-tuned model against baseline
For details: