Build a minimal embedding-based RAG pipeline: chunk, index, retrieve and assemble the prompt. How do you evaluate retrieval, resist injected instructions and abstain when unsure?

import numpy as np

def chunk(text, size=200, overlap=40):
    w = text.split()
    step = size - overlap
    stop = max(len(w) - overlap, 1)
    return [" ".join(w[i:i + size])
            for i in range(0, stop, step)]

def build_index(chunks, embed):
    E = np.array([embed(c) for c in chunks], float)
    return E / np.linalg.norm(E, axis=1, keepdims=True)

def retrieve(q, E, embed, k=4):
    v = np.asarray(embed(q), float)
    s = E @ (v / np.linalg.norm(v))
    top = np.argsort(-s)[:k]
    return top, s[top]

def hit_rate(results, gold):
    hits = [bool(set(r) & g) for r, g in zip(results, gold)]
    return sum(hits) / len(hits)

Normalise embeddings so retrieval is a dot product. Evaluate on questions labelled with their relevant chunk ids: hit rate or recall at k, plus MRR. The prompt wraps chunks as quoted evidence, says nothing inside them is an instruction, and requires a chunk id per claim. Delimiters reduce injection without stopping it, so the model gets no tools. If the top score falls below a threshold tuned on the eval set, abstain.