Merge pull request #285 from Hsy-Intel/main

Fix known issues and refine doc in RAG solution

Author: Hsy-Intel

cczoo/rag/README.md

@@ -61,16 +61,16 @@ The steps to download the required model from the Hugging Face mirror website ar
 ```shell
 cd /home/encrypted_storage
 pip install -U huggingface_hub
-export HF_ENDPOINT=https://hf-mirror.com
-huggingface-cli download --resume-download --local-dir-use-symlinks False meta-llama/Llama-2-7b-chat-hf --local-dir Llama-2-7b-chat-hf
+# If your server is in China, you can set this environment variable: export HF_ENDPOINT=https://hf-mirror.com
+huggingface-cli download --resume-download --local-dir-use-symlinks False meta-llama/Llama-2-7b-chat-hf --local-dir Llama-2-7b-chat-hf --token <your huggingface token>
 huggingface-cli download --resume-download --local-dir-use-symlinks False cross-encoder/ms-marco-MiniLM-L-12-v2 --local-dir ms-marco-MiniLM-L-12-v2
 huggingface-cli download --resume-download --local-dir-use-symlinks False facebook/dpr-ctx_encoder-single-nq-base --local-dir dpr-ctx_encoder-single-nq-base
 huggingface-cli download --resume-download --local-dir-use-symlinks False facebook/dpr-question_encoder-single-nq-base --local-dir dpr-question_encoder-single-nq-base
 ```
 
-### 4. Start the RAG service
+### 4. Run the RAG service
 
-#### start the database service container
+#### run the database service container
 
 If you want to use MySQL as the storage:
 
@@ -86,7 +86,7 @@ cd <workdir>/confidential-computing-zoo/cczoo/rag
 ./run.sh es
 ```
 
-#### start the backend service container
+#### run the backend service container
 
 If you use MySQL as the storage:
 
@@ -119,7 +119,7 @@ python3 generate_db.py
 
 If you use MySQL as the storage, you can edit data in `data/data.txt` directly.
 
-#### Start the frontend service container
+#### Run the frontend service container
 In another new terminal execute the following command:
 
 ```bash
@@ -146,3 +146,52 @@ In your local browser, open the Network URL `http://<host_server>:<host_port>` a
 
 For customized modifications and issues with the RAG framework, please refer to [Haystack](https://github.com/deepset-ai/haystack/tree/main).
 
+#### Run RAG service with RA-TLS
+
+If you want to run the RAG service with remote attestation, you need to modify the following steps:
+
+**Get the verification hash value and configure**
+
+We can get the attestation message by running for both backend and frontend containers:
+
+```shell
+docker exec -it tdx_rag_backend bash -c "cd /usr/bin && ./tdx_report_parser"
+docker exec -it tdx_rag_frontend bash -c "cd /usr/bin && ./tdx_report_parser"
+```
+
+In the `frontend/chatbot-rag/dynamic_config.json` and `backend/pipelines/dynamic_config.json` files:
+- Fill in the "ON" or "OFF" to config verification strategy.
+- Fill in the hash value obtained through the above command into the `dynamic_config.json` file of the corresponding directory. For example, the hash value obtained from the backend should be filled in the frontend configuration file.
+
+**Add RA configuration when running backend service container**
+
+```shell
+./run.sh backend ra <ip addr>
+```
+The "ra" means "remote attestation" and the subsequent IP address is the attestation server address.
+
+Then, enter the following message:
+
+```shell
+Enter database ip addr: <database ip addr>
+
+Enter database username: root
+
+Enter database password: 123456
+```
+
+It will finally print the attestation message. It means the backend server runs successfully.
+
+**Add RA configuration when running frontend service container**
+
+```shell
+./run.sh frontend ra <ip addr>
+```
+
+The "ra" means "remote attestation" and the subsequent IP address is the attestation server address.
+
+**Visit the Web UI and ask questions**
+
+You can click the link generated by the frontend service to access the RAG service.
+
+If everything goes well, you should be able to see the green security connection box at the web page, and the detailed information about remote attestation below.

cczoo/rag/backend/pipelines/server.py

@@ -10,7 +10,7 @@
 from haystack.nodes.prompt import PromptNode
 
 
-query_pipeline = Pipeline.load_from_yaml(Path("rag.yaml"))
+query_pipeline = Pipeline.load_from_yaml(Path("rag_mysql.yaml"))
 
 def _get_grpc_streaming_iterator(pipeline, request=None):
     params = request["params"] or {}
@@ -42,6 +42,9 @@ def Status(self, request, context):
 API_PROTOCOL = os.getenv("API_PROTOCOL", "grpc")
 print("API_PROTOCOL:", API_PROTOCOL, flush=True)
 
+os.unsetenv("http_proxy")
+os.unsetenv("https_proxy")
+
 server = grpc.server(ThreadPoolExecutor(max_workers=8))
 query_pb2_grpc.add_QueryServicer_to_server(QueryServicer(), server)
 

cczoo/rag/frontend/chatbot-rag/build-image.sh

@@ -8,7 +8,7 @@ dockerfile=frontend.dockerfile
 
 DOCKER_BUILDKIT=0 docker build \
     -f ${dockerfile} . \
-    -t intelcczoo/rag-llm:ui \
+    -t intelcczoo/tdx-rag:frontend \
     --network=host \
     --build-arg http_proxy=${http_proxy} \
     --build-arg https_proxy=${https_proxy} \

cczoo/rag/run.sh

@@ -16,14 +16,25 @@
 
 SERVICE_NAME="${1:-}"
 
-[ "$2" == "ra" ] && REMOTE_ATTESTATION="grpc-ratls"
+pccs_addr=127.0.0.1
+
+if [ "$2" == "ra" ]; then
+    REMOTE_ATTESTATION="grpc-ratls"
+    if [ -z "$3" ]; then
+        echo "Error: No PCCS address provided."
+        exit 1
+    else
+        pccs_addr="$3"
+    fi
+fi
 
 function show_help {
     echo "Usage: ./script.sh SERVICE_NAME [ra]"
     echo ""
     echo "Arguments:"
     echo "  SERVICE_NAME  : Name of the service (db: Mysql, es: ElasticSearch, backend, backend_es, frontend)"
     echo "  ra            : Enable remote attestation (optional)"
+    echo "  <ip addr>     : PCCS service address"
     exit 1
 }
 
@@ -64,8 +75,9 @@ elif [ "$SERVICE_NAME" == 'backend' ]; then
         docker rm tdx_rag_backend
     fi
     rm -rf backend/pipelines/faiss-index-so.*
+    rm -rf /home/encrypted_storage/faiss-index-so.*
     echo -e "\nstart backend container..."
-    mv -n data/data.txt /home/encrypted_storage
+    cp data/data.txt /home/encrypted_storage
     docker run -itd --privileged --network host \
        -e http_proxy=${http_proxy} \
        -e https_proxy=${https_proxy} \
@@ -81,6 +93,7 @@ elif [ "$SERVICE_NAME" == 'backend' ]; then
        -v /var/run/aesmd/aesm.socket:/var/run/aesmd/aesm.socket \
        -v /home/encrypted_storage:/home/rag_data/ \
        -v $(pwd)/backend/pipelines:/home/user/workspace \
+       --add-host pccs.service.com:${pccs_addr} \
        --shm-size=64gb --name tdx_rag_backend intelcczoo/tdx-rag:backend /bin/bash
     sleep 5
 
@@ -116,6 +129,7 @@ elif [ "$SERVICE_NAME" == 'backend_es' ]; then
        -v /var/run/aesmd/aesm.socket:/var/run/aesmd/aesm.socket \
        -v /home/encrypted_storage:/home/rag_data/ \
        -v $(pwd)/backend/pipelines:/home/user/workspace \
+       --add-host pccs.service.com:${pccs_addr} \
        --shm-size=64gb --name tdx_rag_backend intelcczoo/tdx-rag:backend /bin/bash
     sleep 5
 
@@ -135,8 +149,6 @@ elif [ "$SERVICE_NAME" == 'frontend' ]; then
     fi
     echo -e "\nstart frontend container..."
     docker run -itd --privileged --network host \
-        -e http_proxy=${http_proxy} \
-        -e https_proxy=${https_proxy} \
         -e no_proxy=${no_proxy} \
         -e API_PROTOCOL=${REMOTE_ATTESTATION} \
         -e STREAMLIT_SERVER_PORT=8502 \
@@ -145,6 +157,7 @@ elif [ "$SERVICE_NAME" == 'frontend' ]; then
         -v /dev:/dev \
         -v /var/run/aesmd/aesm.socket:/var/run/aesmd/aesm.socket \
         -v $(pwd)/frontend/chatbot-rag:/home/user/workspace \
+	--add-host pccs.service.com:${pccs_addr} \
         --shm-size=64gb --name tdx_rag_frontend intelcczoo/tdx-rag:frontend /bin/bash
     sleep 5
     docker exec -i tdx_rag_frontend /bin/bash -c "streamlit run app.py"