Pipeline Tuning Playbook

Goal: Adjust the governance pipeline for a specific client’s needs.
When: Client reports false positives, missed detections, or performance issues.
Audience: Solutions engineer, platform team

---

Pipeline Architecture

Every request flows through three phases, each with categorized steps:

BEFORE                          CALL                      AFTER
├─ access_control               ├─ route_and_execute      ├─ response_safety
│  (rate limit, tool perms,     │  (LLM call, tool call,  │  (output scan, content
│   tool filter, budget, ...)   │   model routing, ...)   │   safety, PII restore, ...)
├─ data_protection              │                         ├─ tracking
│  (PII, secrets, taint, ...)   │                         │  (cost, audit, taint, ...)
├─ threat_detection             │                         │
│  (injection, unicode tricks,  │                         │
│   shell bleed, priv esc, ...) │                         │
├─ classify_and_prepare         │                         │
│  (classification, input       │                         │
│   validation, ...)            │                         │

Preset Step Counts

Preset	Before	Call	After	Total
Starter	7	2	2	11
Standard	23	5	10	38
Regulated	24	8	13	45

All Registered Steps

These are the 40 pipeline steps available in the system:

Access Control: Rate limit, Tool permissions, Tool filter, Tool constraints, User tool scopes, Budget enforcement, Session budget

Data Protection: PII detection, PII tokenization, PII restore, Secret detection, Taint tracking, Business data detection, Infrastructure data detection

Threat Detection: Prompt injection detection, Unicode trick detection, Shell bleed detection, Privilege escalation detection, Code execution detection, Multimodal content detection

Classify & Prepare: Data classification, Input validation, Image inspection, Adaptive thresholds

Route & Execute: LLM call, Tool call, Model routing, Tool call scan, Tool governance verification, Tool poisoning detection, Loop guard, Session loop guard

Response Safety: Output scan, Output constraints, Output deduplication, Content safety, Session escalation, Artifact verification

Tracking: Cost tracking, Debug logging

---

When to Tune

Signal	Action
Too many false positives	Lower sensitivity or add allowlist entries
Missed real violations	Enable additional steps or tighten thresholds
Agent latency too high	Disable non-critical steps or switch to lighter preset
New compliance requirement	Add custom detection patterns
Client terminology triggers PII	Add to allowlist/exclusion list

---

Viewing Current Pipeline

# CLI
tappass gov pipelines          # List all pipelines
tappass gov pipeline default   # View steps in default pipeline
tappass gov step detect_pii    # View a specific step's config

# API
curl -s http://localhost:9620/admin/pipelines \
  -H "Authorization: Bearer $ADMIN_KEY" | python3 -m json.tool

---

Common Tuning Scenarios

1. False Positive on PII Detection

Symptom: Product names like “Anna Router” or “Matt Board” flagged as person names.

Fix: Add to PII allowlist in pipeline config:

overrides:
  categories:
    before:
      data_protection:
        steps:
          detect_pii:
            options:
              allowlist:
                - "Anna Router"
                - "Matt Board"
              allowlist_patterns:
                - "^[A-Z][a-z]+ (Router|Board|Scanner|Module|Server)$"

Note: PII detection uses both regex patterns and NER (spaCy). NER is enabled by default (TAPPASS_NER_ENABLED=1) using the xx_ent_wiki_sm multilingual model. For English-only deployments, set TAPPASS_NER_MODEL=en_core_web_sm.

2. Prompt Injection Not Caught

Symptom: Red team found a bypass using encoded instructions.

Fix: The regulated preset enables all injection layers. If using standard, ensure all detection steps are active:

overrides:
  categories:
    before:
      threat_detection:
        steps:
          detect_prompt_injection:
            enabled: true
            options:
              sensitivity: high   # low, medium, high
          detect_unicode_tricks:
            enabled: true
          detect_shell_bleed:
            enabled: true
          detect_privilege_escalation:
            enabled: true

For maximum protection, enable the LLM judge (TAPPASS_LLM_JUDGE_ENABLED=1), which adds semantic analysis at ~200ms per call.

3. Latency Too High

Symptom: Agent responses take 2+ seconds longer than without TapPass.

Diagnosis:

# Check overall status
tappass status

# Check per-step timing via the audit trail
tappass logs --limit 10 --verbose

Expected latency per preset:

Preset	Avg added latency	P99
Starter (11 steps)	~20ms	~50ms
Standard (38 steps)	~50ms	~150ms
Regulated (45 steps)	~100ms	~300ms
+ LLM judge enabled	+~200ms	+~300ms

Fix options:

1. Switch to a lighter preset
2. Disable specific heavy steps:

   overrides:
     categories:
       before:
         threat_detection:
           steps:
             detect_multimodal_content:
               enabled: false

3. The LLM judge is the heaviest step (external LLM call). Consider using a local model (TAPPASS_LLM_JUDGE_MODEL=ollama/llama3.1) or disabling it for non-critical agents.

4. Custom Classification Patterns

Symptom: Client has internal document codes that should be classified.

overrides:
  categories:
    before:
      classify_and_prepare:
        steps:
          classify_data:
            options:
              custom_patterns:
                - name: internal_project_code
                  pattern: "PROJ-[A-Z]{3}-[0-9]{4}"
                  level: CONFIDENTIAL
                  description: "Internal project identifiers"
                - name: board_minutes
                  pattern: "(?i)board (minutes|resolution|decision)"
                  level: RESTRICTED
                  description: "Board-level governance documents"

Classification levels: PUBLIC, INTERNAL, CONFIDENTIAL, RESTRICTED, TOP_SECRET.

5. Per-Agent Pipeline Overrides

Different agents can use different pipelines:

# Assign a specific pipeline to an agent
tappass gov assign <agent-id> --pipeline regulated

# Create a custom pipeline
tappass gov create-pipeline custom-claims \
  --base regulated \
  --enable tool_poisoning_detection \
  --disable debug_logging

---

Testing Changes

Always test pipeline changes before deploying to production:

# Interactive dry-run (no real API calls)
tappass test

# With a scenario file
tappass test scenarios/hr-agent.yaml

Scenario file format:

calls:
  - provider: gmail
    action: search
    params:
      query: "salary data"
    expect: allow

  - provider: gmail
    action: send
    params:
      to: "external@competitor.com"
      body: "Here are the salary figures: John $150k"
    expect: block
    expect_reason: pii_detected

---

Governance Flags

Quick tuning via SDK flags (no pipeline YAML changes needed):

agent = Agent(url, flags={
    "pii": "mask",           # mask | redact | tokenize | off
    "mode": "observe",       # observe | enforce | off
    "email": "internal:company.com",  # restrict email sending
    "budget": "standard",    # dev | standard | custom:<per_call>:<per_session>
})

Or via environment:

export TAPPASS_FLAGS="mode=observe, pii=mask, email=internal:company.com"

Flags override pipeline defaults. Useful for per-agent tuning without touching the pipeline config.