Templating

Our Task Manager has a built-in templating engine that allows you to provide static data, references to data stored in the task collection database table and the use of functions for basic data transformations.

The rendered result of a template is a DataKind, which will always be a collection of values of the same type. This is even the case if there's just a single value, it will be wrapped in a set.

Literals

The most basic type of expression is a literal. This allows you to provide a static value directly to a task argument. Some examples are:

• String literal: some_literal_string
• Integer literal: 42

References

References allow to retrieve data from various places in the system or task definition. The basic syntax for a reference is as follows:

$[<reference_type>:<key>]

Where <reference_type> indicates the type of the reference and <key> is the identifier of the specific data you want to reference. At the moment we support Data references and Task Parameter References. Since data references are the most commonly used, the reference type can be omitted and it will default to a data reference.

Data References

Data references allow you to reference data stored in the task collection's data storage. During runtime of a task, these references will be resolved to their actual values from the database.

The basic syntax for a data reference is as follows:

$[some_key]

Where some_key is the key of the data within a task collection you want to reference. When providing an expression to a task argument, the task manager will evaluate the expression at runtime and replace it with the corresponding value from the task collection data storage (retrieved from the PostgreSQL database).

A lightweight version of JSONPath syntax is supported for accessing nested data structures. Here are some examples:

• Retrieve a nested key $[another_key.hello]

• Access an array element by index $[another_key.hello[0]]

TODO: THis example doesn't match my another_key data example, fix it

• Return an array of values from an array of objects $[another_key.how.are.[*].you] or $[another_key.how.*.you]

Our database supports a variety of data types, including set of strings, JSON objects and predefined common models like IPv4 Ranges and Domains. When referencing data, the task manager will automatically handle type conversions as needed to ensure that the data is in the correct format for the task argument.

Here is an example of some data that might be stored in a task collection. Specific object properties can be extracted through the JSONPath syntax mentioned above.

Task Parameter References

In certain cases you want to be able to reference to parameters defined within the task itself. This can be especially handy if you want to re-use standard values for different arguments.

One example of this is when performing HTTP requests. Often a simple GET request will have the same content. It might look something like this:

GET / HTTP/1.1
Host: example.com

Lets say we want to fuzz across different SNI values. We could construct a standard body and use self referencing data variables to insert the SNI value in both the TLS SNI field and the Host header.

- kind: http1
  ip: 10.0.0.1
  port: 443
  tls: true
  sni: $[host_fuzzing]
  content: |
    GET / HTTP/1.1
    Host: $[task:sni]

Combining this with our data references, we could create standard content payloads and re-use them across multiple tasks.

- insert_data:
    key: http_get_content
    kind: string
    value: |
      GET / HTTP/1.1
      Host: $[task:sni]
      
- kind: http1
  ip: 10.0.0.1
  port: 443
  tls: true
  sni: $[host_fuzzing]
  content: $[http_get_content]
...

Combined literals and references

You can combine literals and data references within a single expression. When doing so, the task manager will evaluate the entire expression and produce a set of values based on all possible combinations of the literals and referenced data.

Lets take the folllowing example:

• Combining a literal with a data reference prefix_$[some_key]_suffix

If some_key contains the values ["some", "string"], the resulting set of values would be:

• prefix_some_suffix
• prefix_string_suffix

Functions

In addition to simple data references, the expression syntax supports a variety of built-in functions that can be used to manipulate and transform data. Here are some examples:

• capitalize($[some_data_key]): Converts a string to uppercase.
• split($[some_data_key], "."): Splits a string into an array based on the specified delimiter (in this case, a period).

Examples

Here are some more complex examples that combine literals, data references and functions:

Retrieving and transforming data

This expression first splits the string retrieved from some_data_key at each period, takes the first element of the resulting array, and then capitalizes it.

capitalize(split($[some_data_key], ".")[0])

Accessing nested data

This expression retrieves an array of values from the property field of each object in the array_key array within the some_data_key data structure.

$[some_data_key.array_key[*].property]

We also support the alternative wildcard JSONPath syntax:

$[some_data_key.array_key.*.property]

Applying templates to task definitions

Here's an example of how these expressions might be used in the context of a TCP SYN task generator:

- kind: tcp_syn_generator
  ipv4_address_ranges: "$[target_ip_ranges]"
  ports:
        - 80
        - "$[https_ports]"

The result of rendering this template would be a set of TCP SYN tasks, each with a specific IP address from the target_ip_ranges data reference and a destination port that is either 80 or one of the ports specified in the https_ports data reference.

Note that templates are always converted to a set of values, even if the result is a single value. This ensures consistency in how task arguments are handled.

In the example above, if https_ports contains the values [443, 8443], the resulting task ports variable would contain a single set of integers {80, 443, 8443}. ipv4_address_ranges would contain a set of all individual IP addresses derived from the provided ranges.

For ease of use, parameters for task templates can be either a single expression or an array of expressions. If an array is provided, the results of each expression will be combined into a single set.

For example, the following two configurations are equivalent:

ipv4_address_ranges: "$[target_ip_ranges]"

ipv4_address_ranges: 
 - "$[target_ip_ranges]"

HTTP fuzzing examples

When performing HTTP fuzzing, we'd often want to iterate over several different data variables (e.g. wordlists) and generate all possible combinations. Lets see how this can be achieved with our templating engine.

Lets say we have the following data stored in our task collection:

We can then define an HTTP fuzzing task template like this:

- kind: http_fuzz
  method: GET
  host: target.com
  sni: target.com
  body: |
    GET /$[path_traversal]/$[paths] HTTP/1.1
    Host: $[hostnames]

When this template is rendered, the task manager will generate a set of HTTP fuzzing tasks that cover all combinations of the provided path traversal strings, paths and hostnames. This would generate $2 (path_traversal) * 2 (paths) * 2 (hostnames) = 8$ unique body payloads:

GET /../admin HTTP/1.1
Host: localhost

GET /.;/admin HTTP/1.1
Host: localhost

GET /../login HTTP/1.1
Host: localhost

GET /.;/login HTTP/1.1
Host: localhost

GET /../admin HTTP/1.1
Host: 127.0.0.1

GET /.;/admin HTTP/1.1
Host: 127.0.0.1

GET /../login HTTP/1.1
Host: 127.0.0.1

GET /.;/login HTTP/1.1
Host: 127.0.0.1

Note that we could have taken this further by also templating the host and sni fields, but this should give you an idea of how powerful the templating engine can be when combined with data stored in the task collection.