solution benefits

build and deploy a fully private ai setup, completely self hosted, no third party apis and compliant with strict legal and data policies. 

technical solution

tech stack llama 3 70b (quantitized and accelerator using llm) hosted privately on coreweave using dual a100 gpus chromadb as the vecor store to handle document embedding and retrieval, llamaindex to power a rag pipeline enabling real time question and anser over their case files.  n8n as the flue to automate everything from doc updaoad to slack email notifications. simple and clean streamlit based web us fro their staff to chat with the model ask questions and get summarizes instantly.  secure setup with jwt auth, ip access controls, full audit logging.

usage