GenAI Developer, DoaZ (Remote)
Apr, 2024 - Present
Developed and deployed a Retrieval-Augmented Generation (RAG) chatbot to provide rapid answers from complex Korean Construction Rules & Regulation documents.
Improved information retrieval accuracy from 60% to 80% by implementing Contextual Chunking for document preprocessing.
Utilized DSPy for advanced prompt optimization, enhancing chatbot response quality and relevance.
Deployed open-source LLMs (Qwen 148 & 32B) on Vastai GPU servers using VLLM for efficient inference.
Built and maintained FastAPI endpoints for seamless interaction between the AI backend and the frontend application.
Deployed LLMs using candle-rust library for faster LLM inference using Rust Programming Language