Templating
Jinja text templating syntax and context.
Overview
Templating is the process of creating a reusable format or layout for presenting data in a consistent manner. Templating is currently used in Stroom for creating email templates for Analytic Rule detections and Reports.
Stroom's templating uses a sub-set of the template syntax called jinja and specifically the JinJava library. The templating syntax includes support for variables, filters, condition blocks, loops, etc.
When a template is rendered, Stroom will populate the template context with data that can be used by the template.
Basic templating syntax
Jinja templating is very powerful and has a rich language so this is a quick guide to the very basic features. Stroom uses the JinJava templating engine that is based on Jinja templating. Not all feature
See also:
HubSpot's Hubl language also uses the JinJava engine, its documentation may be helpful.
Another reference for jinja templating is here.
Data types
The following data types are available in the Jinja language.
| Data type | What does it represent | Example values |
|---|---|---|
| None | Represents no or lack of value. | none, None |
| Integer | Whole numbers. | 42, 12, 3134 |
| Boolean | Boolean value, i.e. true/false. | true, True, false, False |
| Float | Real numbers (with decimal separator). | 12.34, 423.52341, 3.1415, 15.0 |
| String | Any string of characters. | "dog", 'Cat' |
| List | List/array of items of any type. Can be modified after being assigned. | [1, 2, 3], ["Apple", "Orange"] |
| Tuple | Like a list but cannot be modified. | (1, 2, 3), ("Apple", "Orange") |
| Dictionary | Object containing key/value pairs, also known as a map. | { "fruit": "Apple", "weight": 320 } |
Accessing collection items
A List/tuple item can be accessed by its index (zero based), e.g. fruits[0] returns Apple.
A value in a dictionary can be accessed using its key, e.g. myDict['fruit'] returns Apple. If the key does not contain special characters (with the exception of _, then you can also use this form myDict.fruit to get the same value.
Conditions
Valid conditions are:
==Compares two objects for equality!=Compares two objects for inequality>true if the left-hand side is greater than the right-hand side>=true if the left-hand side is greater or equal to the right-hand side<true if the left-hand side is lower than the right-hand side<=true if the left-hand side is lower or equal to the right-hand side
Logical operators
The logic operators for use with boolean values are:
andBoolean andorBoolean ornotBoolean negation
Expressions
Syntax: {{ ... }}
Expressions will render the value of a variable that has been defined in the template context or in a previous statement. An example of a simple value expression is
Template
The detection time is {{detectTime}}.
### Rendered
The detection time is 2024-05-03T09:15:13.454Z.
An expression can also contain variables that are passed through one or more Filters.
Statements
Syntax: {% ... %}
Statements are used to create conditional blocks, loops, define variables, etc.
Setting variables
Syntax: {% set <variable name> = <value or expression> %}
Template
{% set colour = '#F7761F' %}
Colour: {{colour}}
{% set fruits = ["Apple", "Orange"] %}
Top Fruit: {{ fruits | first }}
{% set fullName = "% %" | format(firstName, surname) %}
Name: {{fullName}}
### Rendered
Colour: #F7761F Top Fruit: Apple Name: Joe Bloggs
Conditional blocks
Syntax:
{% if <value, variable or expression> <condition> <value, variable or expression> %}
< optional content, expressions or statements>
{% elif <value, variable or expression> <condition> <value, variable or expression>%}
< optional content, expressions or statements>
{% else %}
< optional content, expressions or statements>
{% endif %}
Conditional blocks can be used to optional render content depending on the value of a variable. See conditions for the list of valid conditions.
Template
{% if (values | length) > 0 -%}
This detection has {{ values | length }} values.
{%- else -%}
This detection has no values.
{%- endif -%}
### Rendered
This detection has 10 values.
Loops
Syntax:
{% for <item name> in <variable or expression> %}
<content, expressions or statements to repeat for each item>
{% endif %}
For loops allow you to loop over items in a list/tuple or entries in a dictionary.
Template
{% for key, val in values | dictsort %}
{{ key }}: {{ val }}
{% endfor %}
### Rendered
fruit: Apple weight: 320
Note
Note, the filter dictsort is used here to sort the dictionary by its keys.
Note
Note the use of - to prevent an additional line break appearing in the rendered output, see White Space below.
Filters
Syntax: ... | <filter name>
Filters are essentially functions that take an input and return an output. Some functions have additional parameters to control the behaviour of the function.
Filters can be chained together with the output of one filter becoming the input to the next filter in the chain in the same way that Stroom pipeline elements work.
Some useful filters are:
| Filter | Description | Example |
|---|---|---|
length | The number of items in the list/sequence/tuple/string/dictionary | {{ "hello" | length }} => 5 |
escape | Escapes any HTML special characters | <p>{{ "10 > 3" | escape }}</p> => <p>10 > 3</p> |
default | Return the first argument if the input is undefined or empty | {{ None | default("foo", true) }} => foo |
For a full list of filters see here or here.
Comments
Syntax: {# <comment text> #}
Non-rendered Comments can be added to the template.
Template
{#- Show the time -#}
The detection time is {{detectTime}}.
### Rendered
The detection time is 2024-05-03T09:15:13.454Z.
Note
Note the use of - to prevent an additional line break appearing in the rendered output, see White Space below.
White space
When JinJava renders the template, each expression or statement is evaluated and then removed or replaced by its output, but any white space around them, e.g. line breaks remain. This can result in unwanted line breaks in the output.
To avoid unwanted white space you can add the - character to the opening and/or closing tag to strip leading/trailing whitespace outside the block, e.g.
{{ ... }}=>{{- ... -}}{% ... %}=>{%- ... -%}Template
{{ firstName -}}
{{ surname }}
### Rendered
JoeBloggs
For a detailed guide to how white space works see here.
Template context
The context is a data structure that contains the dynamic content to use when rendering the template. The variables and values in the context are set by Stroom depending on where the template is being used.
Rule detections context
When an email subject/body template is rendered for an Analytic Rule detection, field values from the detection are placed in the context which allows them to be used in the template.
For example {{ detectTime }} will render the date/time the detection happened. The fields available in the context are those taken from the detection:1 XMLSchema with some additional fields.
The template has access to the following fields from the detection:
| Field Name | Type | Description |
|---|---|---|
| detectTime | String | When the detection occurred. |
| detectorName | String | Recommended detector detail - name of detector.<br>This should be unique within the system.<br>Some detectors are very specific and only ever create one kind of detection, and in these cases it is likely that the name of the detector will be very similar to the name of the detection headline that it creates.<br>Example: detectorSourceName=Bad Thing Detector headline=Bad Thing Detected.<br>However, it is possible to create detectors that are sufficiently configurable that they can create many kinds of detection.<br>In these cases, this field should always be assigned to the same literal regardless of the detection/headline.<br>Example: detectorSourceName=Really Configurable Detector headline=Good Thing Detected, and detectorSourceName=Really Configurable Detector headline=Bad Thing Detected.<br>For detectors that run within Stroom pipelines, the name of the XSLT can be specified here. |
| detectorUuid | String | This is the UUID of the Analytic Rule document. |
| detectorVersion | String | Recommended detector detail - version of detector.<br>This is the version of the detector identified in detectorSourceName field. Different versions might produce different detections.<br>For detectors that run within Stroom pipelines, the version of the XSLT can be specified here.<br>Example: v1.0. |
| detectorEnvironment | String | Recommended detector detail - where the detector was deployed.<br>For analytics that run within Stroom itself, the name of the processing pipeline can be used.<br>Note: the XSLT function stroom:pipeline-name() can be used within Stroom XSLT processing to determine pipeline name.Other analytics might run within an external processing framework, such as Apache Spark.<br>Example: DevOps Spark Cluster |
| headline | String | |
| detailedDescription | String | Recommended detection detail.<br>A more detailed description of what was detected than provided by headline.<br>This will normally include some information relating to what triggered the detection, such as a specific device, location, or user.<br>In addition to descriptive text that will be the same for all detections of this kind, there are typically several possible variable dimensions that could be used to populate parts of the string that is assigned to this field.<br> Normally, only one such dimension is selected, based on the likely triage process (the kind of analysis that takes place, and principal area of interest of the analysts).<br>It should be possible to group by this field value to collect all detections that relate to the thing that the analysts are most interested in during triage.<br>Example: Charitable Donation By 'Freya Bloggs' Detected or Charitable Donation To 'Happy Cats Cattery' Detected depending on anticipated area of analyst interest(perhaps philanthropic activities of individuals or financial transactions to organisations, respectively).<br>For some detections, this field will have the same value as that for headline as no further information is available. |
| fullDescription | String | Recommended detection detail.<br>Complete description of what was detected.<br> This will normally include some detail relating to what triggered the detection.<br> All dimensions with ordinal (literal) values that are useful for triage are output.<br>Numeric and other continuous values such as timestamps are not included in this full description, in order that grouping by this single field will work effectively.<br>Example: Charitable Donation By 'Freya Bloggs' to 'Happy Cats Cattery' Detected.<br>For some detections, this field will have the same value as that for detailedDescription as no further information is available. |
| detectionUniqueId | String | This field does not need to be assigned.<br> Any assignment should be to a value that is sufficiently unique to identify a specific detection from a specific detector.<br>Typically, but not necessarily, the value of this field is a UUID.<br>It can be useful to assign this field in order to support analytic development / debugging.<br>It is necessary to assign this field if detectionRevision field is assigned a value. |
| detectionRevision | Integer | Can be used, in conjunction with detectionUniqueId to allow detectors that run continuously, in a streaming fashion to revise their detections in the light of new information.<br>For example, it might be useful to revise the same detection with additional linked events and a new standard deviation.<br>Where more than one detection has the same detectionUniqueId value, then the one with the highest detectionRevision will be the current one and all previous revisions (lower numbers in detectionRevision field) are superseded / ignored. |
| defunct | Boolean | This field allows a detection to declare that all previous revisions (same detectionUniqueId, lower detectionRevision numbers) are now considered invalid.<br>For example, new data might arrive later than expected and invalidate a detection that has already been sent into Stroom.<br>Default value is false. |
| values | Dictionary | This a dictionary with all the field/column names from the Query (with the exception of StreamId and EventId) as keys and their respective cell values as the value. |
| linkedEvents | List of DetectionLinkedEvent | This is a list of the event(s) that are linked to this detection. |
| level | String | The severity of the rule that raised the detection, e.g. High, where the rule declares one. See Rule Settings. |
| status | String | How reliable the rule that raised the detection declares itself to be, e.g. Stable, where the rule declares it. See Rule Settings. |
A field that is not set on the detection is not in the context at all, rather than being empty, and a template that references a missing field will fail to render. Guard any field that may not be set, e.g. {% if level is defined %}{{ level }}{% endif %} or {{ level | default('') }}.
DetectionLinkedEvent fields:
| Field Name | Type | Description |
|---|---|---|
| stroom | String | The Stroom instance within which this event exists, assumed to be this instance of Stroom if not supplied. |
| streamId | Long | The ID of the Stream that contains the associated event. |
| eventId | Long | The ID of the Event that is associated with this detection. |
Warning
When choosing the names of the columns in your rule it may be beneficial to use snake_case or UpperCamelCase to make it easier to reference those columns in the detection template (see Accessing Collection Items above). E.g. myDict.some_key vs myDict['some key'].
Report context
When an email subject/body template is rendered for a Report , the context describes the report that was produced rather than a detection.
A Report delivers its data as an attached file, so the context carries facts about that file and about the run that produced it, not the data itself.
| Field Name | Type | Description |
|---|---|---|
| reportName | String | The name of the Report document. |
| description | String | The content of the report's Documentation tab. |
| executionTime | String | This is the actual wall clock time that the report ran. |
| effectiveExecutionTime | String | This is the time used in any relative date expressions in the report's query or time range, e.g. day() - 1w. |
| rowCount | Integer | The number of rows in the report. |
| fileType | String | The format of the attached file, e.g. EXCEL. |
| fileName | String | The name of the attached file. |
| aiSummary | String | The AI summary of the report's data, or an empty string where there is none. |
aiSummary is always present in the context, so that a template using it renders whether or not a summary was produced.
Note
None of the fields from the Rule Detections Context are available to a Report template, and none of the fields above are available to an Analytic Rule template. A template written for one will not render against the other.