Templating System

The mrpf_templates crate provides a standalone, configurable template engine used across MRPF for generating permutations of strings from variable data. It powers HTTP request building, DNS domain generation, task definitions, and more.

Core Concepts

A template is a string containing literals and expressions. Expressions are delimited by a configurable prefix/suffix pair — the default is $[/], while the HTTP scanners use ${/}.

Hello ${name}, welcome to ${place}!

Variables can have multiple values. When a template references multiple multi-valued variables, the engine produces the Cartesian product of all combinations.

Architecture

The system has three layers:

1. Engine — configures delimiters, functions, reserved variables, post-processors, and nesting depth
2. Template — a parsed AST from a template string
3. Renderer — a lazy iterator that produces one rendered string per Cartesian combination

Engine::parse("...") → Template
Engine::renderer(template, variables, round_robin) → Renderer (Iterator<Item = String>)

The Engine is wrapped in Arc so multiple Renderer instances can share it across threads. Renderer is Send + 'static, suitable for scanner transmit threads.

Variable Tiers

The template system has three tiers of variables, resolved in this priority order:

1. Regular Variables (Cartesian Product)

Standard variables whose values are expanded as a Cartesian product. If path has 3 values and host has 2 values, the template produces 6 outputs.

Template: GET /${path} HTTP/1.1\r\nHost: ${host}\r\n\r\n

Variables:
  path: [admin, login, api]
  host: [example.com, test.com]

→ 6 rendered strings (3 × 2)

2. Static Variables (Round-Robin)

User-defined uppercase variables ([A-Z0-9_]+) that cycle through their values per Cartesian output without expanding the product count. Stored in the database as StaticVar with values directly in a TEXT[] column.

Template: ${GREETING} ${name}

Variables (cartesian):
  name: [moon, world]

Static variables (round-robin):
  GREETING: [hi, hello]

→ 2 outputs (NOT 4):
  "hi moon"      (GREETING[0])
  "hello world"  (GREETING[1])

Round-robin wraps around when the Cartesian product is larger than the number of static values:

Variables: name: [a, b, c]
Static:    RR: [x, y]

→ "x a", "y b", "x c"  (wraps at index 2)

Multiple round-robin variables cycle independently based on their own value counts.

3. Reserved Variables (Builtin Passthrough)

Uppercase keywords like IPV4, PORT, SNI, HOST, CONTENT_LENGTH, and FUZZ that are not resolved during template rendering. They pass through as literal ${IPV4} in the output and are filled later at transmit-time by the scanner’s replace_host_variables().

Template: ${host}:${IPV4}:${PORT}
Variables: host: [example.com]

→ "example.com:${IPV4}:${PORT}"

If the engine has a reserved set configured, referencing an unknown uppercase variable (not in data, not reserved) produces an UnknownReservedVariable error — catching typos like ${PROT} instead of ${PORT}.

Resolution Priority

When the engine encounters a variable reference:

1. Data lookup — if the variable exists in the provided data map, use its values
2. Reserved passthrough — if the name is in the reserved set, emit it as a literal ${NAME}
3. Unknown uppercase error — if a reserved set is configured and the name is all-uppercase, error
4. Missing variable error — otherwise, error

This means you can override a reserved variable by providing it in the data map (e.g., providing FUZZ values directly).

Expressions

References

Simple variable reference:

${variable_name}

With JSONPath-style path access:

${object.property}
${array[0]}
${data.items[*].name}
${data.items.*.name}

Functions

Built-in functions: upper, lower, trim, join.

$[upper(name)]           → "HELLO", "WORLD"
$[lower(name)]           → "hello", "world"
$[trim(name)]            → strips whitespace
$[join(names, " - ")]    → "a - b - c"

Functions can be nested and take variable references or literal strings as arguments.

Custom functions can be registered on the engine:

#![allow(unused)]
fn main() {
engine.register("repeat", Box::new(|args| { ... }));
}

Literal Strings

Quoted strings inside expressions:

$[join(names, ", ")]
         ^^^^  literal string argument

Engine Configuration

Use the builder for full control:

#![allow(unused)]
fn main() {
let engine = Engine::builder()
    .delimiters("${", "}")          // custom delimiters
    .with_builtins()                // upper, lower, trim, join
    .reserved(["IPV4", "PORT"])     // passthrough variables
    .post_process(|s| s.to_lowercase())  // transform final output
    .max_depth(5)                   // nested resolution passes
    .build();
}

Nested Resolution

When max_depth > 1, the engine re-parses rendered output that still contains the prefix delimiter. This allows variables whose values contain further template expressions:

Variables:
  greeting: ["${name} from ${place}"]
  name: ["Alice"]
  place: ["Wonderland"]

Template: ${greeting}
Pass 1 → "${name} from ${place}"
Pass 2 → "Alice from Wonderland"

Resolution stops when output stabilizes or max_depth passes are reached. Post-processors only apply to the final result.

Post-Processors

Transform every rendered string after all resolution passes:

#![allow(unused)]
fn main() {
.post_process(|s| s.to_lowercase())
.post_process(|s| format!("[{}]", s))
// Applied in order: "HELLO" → "hello" → "[hello]"
}

Rendering Modes

render_to_strings — Full Cartesian Set

Returns a HashSet<String> with all combinations materialized:

#![allow(unused)]
fn main() {
let results: HashSet<String> = engine.render_to_strings(&template, &data)?;
}

render_to_string — Joined Single String

Joins multi-valued expressions with a separator (default ,):

#![allow(unused)]
fn main() {
let result: String = engine.render_to_string(&template, &data, Some(", "))?;
}

renderer — Lazy Iterator

For scanners that need to stream combinations without materializing the full product:

#![allow(unused)]
fn main() {
let engine = Arc::new(engine);
let renderer = engine.renderer(template, &variables, &round_robin);

// renderer.len() returns total combinations (Cartesian only)
for result in renderer {
    let rendered: String = result?;
    // send to network...
}
}

The Renderer uses an odometer-style index to advance through combinations lazily. Round-robin variables are filled per iteration without affecting the combination count.

Usage Across MRPF

Crate Structure

shared/mrpf_templates/src/
├── lib.rs           # Public API re-exports
├── template.rs      # Engine, EngineBuilder, Template, Renderer
├── parser.rs        # Template string → AST parser
├── ast.rs           # TemplatePart, Expr types
├── data.rs          # TemplateData trait
├── functions.rs     # FunctionRegistry, built-in functions
├── from_template.rs # FromTemplate trait for type conversions
├── error.rs         # Error types
└── transform.rs     # transform() function (feature-gated)