Generative AI Project Ideas: LLM, RAG and Agent Topics

What makes a generative AI project worth building

Language models produce fluent output even when it is wrong, which is why evaluation is the hard part. A project becomes substantial when you build a test set of questions with known answers, measure the system against it, change one design decision and measure again.

The ideas below are grouped by the kind of engineering involved. Most can be built with either a hosted API or an open-weight model run locally. Models change quickly, so select the current options when you start and record the exact version in your report.

RAG projects

Retrieval-augmented generation finds relevant passages from your own documents and asks the model to answer from them. It suits student work because the corpus is yours and answers can be checked.

1. University regulations assistant with citations — FAISS, Sentence Transformers, FastAPI
   Rules on backlogs and attendance are buried in long circulars. Answer from these documents only, cite the page, and let students click through to verify.
2. Comparison of chunking strategies — LangChain or LlamaIndex, Ragas
   Fixed-size, sentence-based and heading-based chunks retrieve differently. Build a question set for one corpus and measure which strategy finds the right passage.
3. Hybrid search with reranking — BM25, pgvector, cross-encoders
   Embeddings can miss exact terms like section numbers. Combine keyword and vector search, add a cross-encoder reranker, and measure the gain against its added latency.
4. Marathi and Hindi document question answering — Multilingual embeddings, Qdrant
   Test whether users can ask in one language and retrieve from another, and compare multilingual embeddings against translating the query first.
5. RAG over scanned PDFs and tables — Tesseract, Docling, Chroma
   Tables and scans often break when flattened to plain text. Compare layout-aware parsing with naive extraction on questions that require reading table cells.
6. Answer refusal when evidence is missing — Python, Llama, NLI models
   A trustworthy assistant says it does not know. Add unanswerable questions to the test set, tune a retrieval threshold, and report how often it refuses correctly.
7. Knowledge graph RAG versus vector RAG — Neo4j, LangChain
   Questions linking several facts can defeat similarity search. Extract entities into a graph and compare it against plain vector retrieval on multi-hop questions.

LLM engineering projects

1. Fine-tuning versus prompting for support tickets — Hugging Face, PEFT, scikit-learn
   Is fine-tuning worth the effort? Compare few-shot prompting, a fine-tuned small model and a classical classifier on accuracy, latency and effort.
2. Local LLM benchmark on student laptops — Ollama, llama.cpp
   Many colleges cannot send data to external APIs. Measure speed, memory use and answer quality for quantised models on ordinary hardware.
3. Prompt injection detection — Python, classifier guardrails
   Hidden instructions in a document or web page can hijack an assistant. Assemble attack examples, test a defence layer, and report which attacks still pass.
4. Hallucination detection for summaries — NLI models, Streamlit
   Summaries sometimes add facts absent from the source. Break output into claims and check each using entailment, flagging unsupported sentences for the reader.
5. Structured extraction from invoices and forms — Pydantic, vision-capable LLM
   Turn varied layouts into validated JSON, and retry automatically when validation fails. Evaluate field by field against hand-labelled samples.
6. Natural language to SQL with safeguards — MySQL, SQLGlot, LLM API
   Let staff query a college database in plain English. Use a read-only role and query validation, and score the generated SQL by the rows it returns.
7. Semantic cache for LLM applications — Redis, embeddings
   Repeated questions waste calls and time. Serve similar queries from a cache, and find the similarity threshold beyond which wrong answers start to appear.

AI agent projects

1. Lab scheduling agent with human approval — LangGraph, function calling, MySQL
   The agent reads requests, checks availability and proposes bookings, but pauses for approval before writing to the database. Log every tool call for audit.
2. Pull request review assistant — GitHub API, tree-sitter
   Feed a code diff and lint output to a model that comments on likely bugs. Evaluate it on seeded defects and count both catches and noise.
3. Literature survey assistant — Academic search APIs, LlamaIndex
   Find papers on a topic, extract method and dataset into a comparison table, and verify that every cited paper actually exists to catch invented references.

Image, audio and multimodal projects

1. Diffusion-based augmentation for rare classes — Stable Diffusion, PyTorch
   Generate synthetic images of an under-represented class, then test whether they actually improve a classifier on real test images.
2. AI-generated image detection — CNN or ViT, PyTorch
   Train a detector, then test it on a generator it never saw and after compression. Generalisation is where these systems usually struggle.
3. Image descriptions for visually impaired users — Vision LLM, text-to-speech, Flutter
   Describe notices and textbook diagrams aloud in a regional language, working with real users to learn which details matter most to them.

Decisions to make before you start

• Hosted API or local model: an API is easier and often higher quality, while a local model keeps data private and has no usage bill. Check current pricing and free limits before you plan your demo.
• Evaluation set: write 50 to 100 questions with expected answers early, including unanswerable and tricky ones.
• Failure behaviour: decide what users see when retrieval finds nothing or the model times out.
• Demo plan: keep a cached fallback in case the internet or API fails during the viva.
• Disclosure: follow your institution's rules on AI-assisted writing and code, and be able to explain every part of your system.