---

name: qmd-redundancy-check description: Finds and quantifies redundant content in QMD files - repeated equations, duplicate sections, similar paragraphs, and structural redundancy. allowed-tools:

• Read
• Bash
• Write
• Edit
• Grep
• Glob
• AskUserQuestion

---

QMD Redundancy Check

_latex Variables

All {{< var X >}} variables are hyperlinks to their derivations. When removing {{< var X_latex >}}, just delete it - no need to add links.

---

Phase 1: Generate and Read the Report

Copy.venv/Scripts/python.exe scripts/redundancy-check.py <file.qmd> -v -o _analysis/redundancy-report.md

The report contains:

1. Document Outline - Full heading hierarchy with line numbers
2. Summary Section Flags - Headings containing "Summary", "Conclusion", "Key Findings" (check for overlap)
3. Similar Heading Pairs - Headings with >40% word overlap (may cover same content)
4. Duplicate _latex Variables - Sorted by redundancy cost (occurrences × lines)
5. Duplicate Sentences - Exact matches appearing multiple times
6. Similar Paragraphs - Pairs with >50% word overlap
7. Repeated Phrases - 5+ word n-grams appearing 3+ times

Read the report, then read the target QMD file to understand context.

---

Phase 2: Systematic Issue Resolution

Process issues in priority order. For each issue:

Step 1: Investigate

Read 30-50 lines around each occurrence. Understand the context.

Step 2: Analyze

Determine which is PRIMARY (authoritative) vs SECONDARY (recap/reference):

• Primary: First introduction, methodology section, main explanation
• Secondary: Summary, conclusion, "as mentioned earlier", recap sections

Step 3: Decide

Present findings and recommendation to user (if not obvious):

CopyIssue: [Type] - [Description]
Locations: L123 (Section A), L456 (Section B)
Analysis: [Why one is primary, one is secondary]
Recommendation: [Keep X, delete/merge Y]

Step 4: Execute

Make the edit. For deletions, ensure no information is lost.

Step 5: Continue

Move to next issue. Track progress.

---

Issue Type: Duplicate _latex Variables

Why it matters: Each equation expands to 20-50+ lines. 5 occurrences = 80-200 unnecessary lines.

Rule: Keep 1-2 instances per equation, depending on document structure.

Since all {{< var X >}} variables are hyperlinks to their derivations, readers can click any value to see the full equation. However, long technical documents may have intentional duplication for different audiences.

Acceptable duplication (multi-audience design):

• Executive summary callout + formal verification section
• Body explanation + economist verification section
• These serve different reading paths and should be kept

What to REMOVE:

• 3+ occurrences of the same equation (always excessive)
• Equations 6 lines apart in the same section
• Equations in narrative sections where inline value is stated
• Equations immediately after tables showing the same values

What to KEEP:

• At least 1 instance of each equation
• Intentional pairs serving different audiences (e.g., summary + derivation)

---

Issue Type: Similar Headings

From report: Pairs with >40% word overlap flagged.

Investigation:

1. Read full content under both headings
2. Compare: Same points? One subset of other? Different aspects?

Actions:

Improving Heading Names

When sections cover related but distinct topics, rename headings to be more descriptive of their actual content. This prevents false positives in future scans and helps readers navigate.

Heading naming principles:

• Include the section's PURPOSE, not just its topic
• Add context words: "Assumptions", "Uncertainty", "Mechanism", "Comparison", "Derivation"
• Make parent-child relationships clear

Examples of heading improvements:

Anti-patterns to fix:

• Same heading appearing in Methodology AND Limitations sections
• Generic headings like "Results", "Analysis", "Discussion" without context
• Headings that match parent section name (e.g., "Assumptions" under "Key Analytical Assumptions")

---

Issue Type: Summary/Recap Sections

From report: Sections flagged with keywords: "Summary", "Key Findings", "Conclusion", "Results", "Overview"

Investigation: Compare content across all flagged sections. Do they repeat same bullets/points?

Common pattern: Document has "Key Findings" at top, "Summary of Results" in middle, and "Conclusion" at end - all saying the same thing.

Actions:

---

Issue Type: Duplicate Sentences

From report: Exact sentences appearing at multiple line numbers.

Investigation: Read surrounding paragraphs. Which occurrence is the "home" for this sentence?

Actions:

• Delete the repeat, keep in primary context
• If both contexts need it, rephrase one

---

Issue Type: Similar Paragraphs

From report: Paragraph pairs with >50% word overlap.

Investigation: Read both paragraphs in full. Are they:

• Same point, different words?
• Different points with shared vocabulary?
• One elaborating on the other?

Actions:

---

Issue Type: Structural Redundancy (Manual Review)

The script flags candidates, but these require human judgment:

Multiple summary sections

• Check if "Key Findings", "Summary", "Conclusion" repeat same bullets

Collapsible boxes duplicating body

• Is the "Key Metric Derivations" collapse just repeating body content?

Tables duplicating adjacent prose

• Are values in tables also stated in sentences before/after?

Repetition signal phrases (search for these):

• "As mentioned earlier..."
• "Recall that..."
• "As noted in the previous section..."
• "To reiterate..."

These often mean: content can be deleted, or replaced with a link.

---

Phase 3: Verification

Copy# For _latex: count remaining occurrences
grep -o "{{< var [a-z_]*_latex >}}" <file.qmd> | sort | uniq -c | sort -rn

# Re-run redundancy check
.venv/Scripts/python.exe scripts/redundancy-check.py <file.qmd>

# Validate no broken references
.venv/Scripts/python.exe scripts/pre-render-validation.py 2>&1 | grep <filename>

Success criteria:

• Every _latex variable has 1-2 occurrences (no 3+ duplicates)
• Similar heading pairs resolved (merged, renamed, or confirmed intentional)
• Duplicate sentences eliminated
• Total redundancy reduced while preserving multi-audience structure

---

Quick Reference: Common Patterns

Heading fixes (to prevent future false positives):