LLM pipeline security: protecting prompts, PII, and vectors

LLM pipeline security means safeguarding every artifact—from the user prompt to the stored embeddings—to stop leakage, injection attacks, and data breaches. In the production systems I ship (e.g., my Portfolio AI Assistant built on Next.js, Neon PostgreSQL + pgvector, and Gemini embeddings), a single unguarded step can expose sensitive business logic or personal data. Below I walk through the concrete safeguards I apply every day.

Why LLM pipeline security matters

An LLM pipeline is more than a single API call; it consists of:

1. Prompt ingestion – the raw text that reaches the model.
2. Embedding generation – turning text into high‑dimensional vectors stored in a vector database.
3. Retrieval & re‑ranking – pulling the most relevant chunks from pgvector.
4. Generation – the final answer sent back to the user.

Each stage can be a vector for data leakage:

• Prompt injection can cause the model to reveal internal prompts or API keys.
• PII in embeddings can be reverse‑engineered or violate GDPR/CCPA.
• Unencrypted vector stores expose proprietary knowledge bases to anyone with DB access.

LLM pipeline security: preventing prompt injection and leakage

The first line of defense is sanitizing the incoming prompt before it ever touches the model. I wrap the ingestion endpoint in a tiny validator that strips suspicious patterns and logs the original request for audit.

// src/middleware/validatePrompt.ts
import type { Request, Response, NextFunction } from 'express';
import xss from 'xss';

// Simple regex‑based injection guard
const INJECTION_REGEX = /(?:\b(?:SELECT|DROP|INSERT|DELETE)\b|<script>)/i;

export function validatePrompt(req: Request, res: Response, next: NextFunction) {
  const { prompt } = req.body;
  if (typeof prompt !== 'string' || prompt.length === 0) {
    return res.status(400).json({ error: 'Prompt required' });
  }

  // XSS sanitization + custom injection check
  const cleaned = xss(prompt);
  if (INJECTION_REGEX.test(cleaned)) {
    // Log the attempt, but never forward the raw prompt
    console.warn('Prompt injection attempt', { raw: prompt });
    return res.status(422).json({ error: 'Potential injection detected' });
  }

  // Attach the safe prompt for downstream handlers
  req.body.safePrompt = cleaned;
  next();
}

Why it works: The middleware removes HTML‑based XSS vectors (xss library) and blocks common SQL‑style injections that could be echoed back by a system‑prompt. Because the raw prompt never reaches the LLM, the risk of prompt leakage is eliminated.

Redacting PII in embeddings and storage

Even a clean prompt can contain personal data. Before we call the Gemini embedding endpoint, I run a lightweight PII detector (based on the pii-detector npm package) and replace matches with placeholders. This keeps the vector store GDPR‑compliant.

import { detectPII } from 'pii-detector';
import { GeminiClient } from '@google/generative-ai';

// Replace detected entities with generic tokens
function redactPII(text: string): string {
  const entities = detectPII(text);
  let redacted = text;
  for (const { type, value } of entities) {
    redacted = redacted.replaceAll(value, `[${type}]`);
  }
  return redacted;
}

export async function embedDocument(doc: string) {
  const safeText = redactPII(doc);
  const client = new GeminiClient(process.env.GEMINI_API_KEY);
  const embedding = await client.embeddings.create({
    model: 'gemini-embedding-2',
    input: safeText,
  });
  return embedding;
}

Key point: The redaction happens before the embedding call, so the vector store never holds raw identifiers like phone numbers or email addresses. If a downstream analyst needs the original text, it can be retrieved from an encrypted audit log (see the next section).

LLM pipeline security: encrypting vectors at rest and in transit

Neon PostgreSQL offers TLS‑encrypted connections out of the box, but the vector column itself is still plain data. I add a thin encryption layer using PostgreSQL’s pgcrypto extension so that even a compromised DB user cannot read the vectors.

-- Enable pgcrypto (run once)
CREATE EXTENSION IF NOT EXISTS pgcrypto;

-- Table definition with encrypted vector column
CREATE TABLE documents (
  id UUID PRIMARY KEY,
  content TEXT,
  embedding BYTEA -- encrypted binary
);

-- Insert with encryption (key stored in Vault)
INSERT INTO documents (id, content, embedding)
VALUES (
  gen_random_uuid(),
  $1,
  pgp_sym_encrypt($2::bytea, current_setting('app.encryption_key'))
);

On the Node.js side, the key is fetched from a secret manager (e.g., HashiCorp Vault) at startup and injected into the app.encryption_key setting:

import { Pool } from 'pg';
import { getSecret } from './vault';

async function initDb() {
  const pool = new Pool({ connectionString: process.env.DATABASE_URL });
  const key = await getSecret('vector-encryption-key');
  await pool.query(`SET app.encryption_key = $1`, [key]);
  return pool;
}

Result: Vectors are encrypted at rest and travel over TLS, satisfying both internal security policies and external compliance checks.

Access control for multi‑tenant RAG systems

When you expose a RAG service to multiple customers, each tenant must only see its own chunks. I enforce this with row‑level security (RLS) policies in PostgreSQL, combined with JWT‑based tenant IDs.

-- Enable RLS
ALTER TABLE documents ENABLE ROW LEVEL SECURITY;

-- Policy: a user can only read rows matching their tenant_id
CREATE POLICY tenant_isolation ON documents
  USING (tenant_id = current_setting('app.tenant_id')::uuid);

And in the application bootstrap:

async function setTenantContext(pool: Pool, tenantId: string) {
  await pool.query(`SET app.tenant_id = $1`, [tenantId]);
}

Because the policy lives in the database, any client—whether it’s a Node worker, a Python script, or a direct psql session—must respect the tenant boundary.

Audit logging and data retention policies

Security isn’t just about prevention; it’s also about visibility. I instrument every LLM request with OpenTelemetry, capturing:

• Prompt hash (SHA‑256) – never the raw text.
• Redaction actions taken.
• Embedding request/response metadata.
• User‑tenant ID and request latency.

import { trace, context } from '@opentelemetry/api';
import crypto from 'crypto';

export function logLlmRequest(req: Request, safePrompt: string) {
  const tracer = trace.getTracer('llm-pipeline');
  const span = tracer.startSpan('llm.request', {
    attributes: {
      'llm.prompt_hash': crypto.createHash('sha256').update(safePrompt).digest('hex'),
      'llm.tenant_id': req.user.tenantId,
      'llm.redacted': safePrompt !== req.body.prompt,
    },
  });
  // Ensure the span ends after the response is sent
  req.on('finish', () => span.end());
}

Retention is configured at the observability layer: logs older than 90 days are archived, while raw prompts are never stored beyond the request lifecycle.

Conclusion

Securing an LLM pipeline is a layered effort: validate prompts, redact PII before embedding, encrypt vectors, enforce tenant‑level access, and keep detailed audit trails. Applying these controls turned my Portfolio AI Assistant into a production‑grade service that respects user privacy and survives real‑world attacks.

If you’re looking for more on cost‑aware RAG deployments, check out my article on RAG cost optimization. For a deeper dive into CI/CD patterns for AI features, see Deploying AI features.

Happy building—securely.