Prompt engineering and RAG

Programming with LLMs for Data Practitioners

Julia Silge | Posit PBC

Hello!

@juliasilge.com

Day 1: Build on AI

Talking with LLMs via code
Programming with LLMs
Prompt engineering and RAG (this session!)
Beyond prompts

Prompt engineering and hallucinations

Models will confidently make things up

What hallucination looks like
Why a confident answer is not a correct answer

Your turn

Activity

Open 09-plot-interpret and ask the model about a plot:

Generate a plot of mpg vs. weight from mtcars
Ask the model to interpret it
Discuss what it got right and where it leaned on priors

05:00

Your turn

Activity

Open 10-plot-noise:

Replace the plot with random noise but keep the prompt the same
See what the model says
Pair up with a neighbor and try to steer the model into an “appropriate” interpretation of pure noise

10:00

System prompt best practices

Be explicit about role, audience, and format
Show, don’t just tell (“few-shot” examples)
Tell the model how to handle uncertainty
Read more on how when plotting, LLMs see what they expect to see

Your turn

Activity

Open 11-quiz-game-app and write the system prompt for our quiz game show
The app reads your prompt from 11-quiz-game-prompt.md
Edit with your own quiz show design
Run it to play the game!

10:00

Manual RAG

Context in, better answers out!

“Retrieval-Augmented Generation” without any fancy infrastructure
Stuff the relevant docs into the prompt yourself
Most of what people mean when they say “RAG” is this idea plus automation

Your turn

Activity

Open 12-coding-assistant and try a coding assistant on a niche package (weathR for R, NWS for Python):

First, ask with no context and note what goes wrong
Then paste the README into the prompt and compare

10:00

RAG

How RAG actually works

Chunk your documents
Compute embeddings for each chunk
At query time, find the most similar chunks and add them to the prompt
The “retrieval” is just nearest-neighbor search

Demo

A dynamic RAG system over the R for Data Science book: chunk, embed, and let the model retrieve relevant passages on demand.

# demo/03-rag/03-rag.qmd
ragnar_register_tool_retrieve(chat, store, top_k = 10)
live_browser(chat)

Thanks!

@juliasilge.com