Parameterized Workflows

Custom Workflow Parameters

Platform allows you to use custom parameters for workflows. These parameters can be defined either through the UI or within the YAML.

Declare a Parameter from the UI

Start by selecting the Parameters tab for your workflow.  Then click the “Add Parameter” button. 

The UI will display forms for the parameter name, which is required, and the default value, which is optional.  If a default value is entered, that value will be automatically used during the execution.  If no default is entered, the user will have to manually enter a value each time the workflow is executed.  

Note that all parameter values defined or modified via the UI forms will be interpreted as strings. If you want to supply a boolean, number, or an object as the default value for your parameter, define it via the YAML (see below).

Declare a Parameter from YAML

To declare an input parameter, create a field under the top level key in your workflow YAML definition called input.

Under input declare all the names of the parameters you would like to later use. Default values are optionally specified in the following format PARAMETER_NAME: DEFAULT_VALUE. In the following example, the parameter “computer_type” has a default value, but the parameter “year” does not.

Note: if your workflow contains a script that also references your workflow parameters, those parameters will also need to be declared in your script. The script will need to be published as a template, and then referenced in the workflow YAML. Documentation on script parameterization can be found here.  

workflow:
 input:
   - computer_type: “Windows PC”
   - year

You can also use the YAML to define parameters with default value types not yet supported via the UI. For example, numbers, objects, and booleans:

workflow:
 input:
   - test_template_id: 12345
   - database_and_credential:
     database_id: 567
     credential_id: 891
   - i_should_eat_ice_cream_today: true

Access a Parameter

To access a workflow level parameter in a task, you must use YAML. You should utilize the syntax

<% $.NAME_OF_THE_PARAMETER %>

to place your desired parameter wherever you would like the value of that parameter to be substituted. For example, building off of the definition we began above, if a user provided “2023” for the “year” parameter’s value, when we execute the workflow,

   read_data_from_year:
     action: civis.scripts.sql
     input:
       sql: |
        SELECT price FROM computer_data_<% $.year %> WHERE type = <% $.computer_type %> ;
       remote_host_id: 32
       credential_id: 4200

the “read_data_from_year” task would run to create a SQL script with 

 SELECT price FROM computer_data_2023 WHERE type = Windows PC ; 

as its SQL.

Understand which Parameters are used in a Task

In the UI, you can determine what workflow parameters are referenced by a task by selecting a task in the graph, then navigating to the Parameters tab.

If a task has been selected, the Parameters tab will only show the parameters used for that task. If multiple tasks have been selected, the tab will show all parameters referenced by any of the selected tasks.

In the YAML, you can look for the YAQL variable reference syntax, <% $.NAME_OF_THE_PARAMETER %>.

Note: The Parameters tab in the UI is only aware of YAQL parameter references. We do not support Jinga at this time.

Accessing the Platform Workflow ID and Execution ID

You can access the IDs of the current workflow and execution in Platform as parameters in your workflow, with the following syntax:

<% $.get('__env').workflow_id %>

<% $.get('__env').execution_id %>

Executing the Workflow with Parameters

Once a parameter has been defined and accessed, click Execute. Your workflow will run with the parameters you have specified.

The parameters tab will show all parameters and values that were used for the workflow execution when no task is selected. 

If a task has been selected, the pane will only show the parameters used for that task.

Note: All arguments supplied via the UI parameters tab form will be interpreted as strings. If you want to run your workflow with non-default parameter values that are booleans, numbers, or objects, you will need to use the API. 

Advanced Workflow Parameters

Workflow tasks support the attribute with-items, which accepts a list of values. The workflow executes the task once for each supplied value. Tasks initiated by a with-items list will execute in parallel.

Example

version: '2.0'
workflow:
 input:
   - friends: "Tom,Jane,Harry"

 tasks:
   read_data_from_year:
     action: civis.scripts.python3
     with-items: friend in <% $.friends.split(",") %>
     input:         
      source: |
        print ("<% $.friend %> is my close friend")

Note about inputs

The API understands any JSON-compliant type, while the UI assumes all inputs are strings. Therefore when using the UI, you need to ensure that your input to with-items is a string that can be parsed into a list, as in the example above.

Note! Platform limits the number of action executions that can be kicked off by a task to 1000. If you have a workflow task that attempts to execute more than 1000 actions, the task will fail (and any subsequent tasks set to execute on-success will not be kicked off). A common reason for this to happen is the use of with-items in a parameterized workflow; if your task attempts to loop through more than 1000 items (creating an action execution per item), it will fail.