Script Templates

 

What are script templates?

Script templates enable users to develop flexible and repeatable code assets for their teammates. Users develop code in what's known as a backing script, and publish it for use by others as a script template. Unlike ordinary scripts, script templates have no configuration or code, instead they inherit their behavior from the backing script. Teammates can create and run custom scripts off of the script template without having access to the underlying code in the backing script.

 

 

 

The sections below explain the script template process in greater detail, but at a high level this is how script templates work: 

1. User develops code in a Civis Platform script (SQL, Python, Javascript, R, or Container script) that will serve as the backing script
2. User publishes the script as a script template
3. User shares the script template with relevant users and/or groups
4. End users create custom scripts from the template, either from the template detail page or via the API in order to leverage the backing script code

Where to find script templates

Go to the ‘Code’ tab on the top navigation and click ‘More Script Templates’. Search for a template by name or template ID. Clicking on a template name will create a custom script from that template. Alternatively, you can click on the standard action menu (triple dots icon) to navigate to the script template page.

 

The script template page includes information on the parameters in the template as well as dependent custom scripts. The index of custom scripts indicates how many scripts have been created and the users running them -- this information is helpful if changes in the backing script need to be communicated to downstream users. A link to the template page is also accessible from the backing script and all dependent custom scripts.

 

 

How to create a new script template

To create a script template, you must start by creating a backing script which will contain the code you want to share. Be sure to add any parameters your code expects. You can run your backing script as many times as you like to test that it is working as expected.

Next, create a script template by publishing the backing script. You should give the template a descriptive name so users will know what it does. By default, the script template will be shared with users in your primary Civis Platform group. You should share the script template with read permissions with any users who want to use your script. You do not need to share the backing script.

Examples

API Example

Ex: The following request would create a new script template based on an existing script with the ID 44123:

POST /templates/scripts

{

"script_id": 44123,

"name": "My new template"

}

User Interface Example

On an existing script, select "Publish as Template". A link to the template will appear in the backing script's settings. You can also find the template under Code > More Script Templates and searching for the scripts name.

 

 

User Context

When creating a new script template, it is important to determine the appropriate user context for the backing script. This attribute determines whether custom scripts created from the template will run using the custom script's runner credentials, or the backing script's runner credentials. The default and most common user context value is "runner," but in some cases "author" context is useful if the script requires resources you do not wish to share with others. For example:

• Your script may require access to data you don't want to share with your end users (for contractual or other reasons)
• You may want the script to execute on your own compute cluster because it requires more capacity than users have access to
• You may want the script to execute on your own compute cluster to ensure your code and credentials remain on hardware allocated only to your organization

A more detailed look at the differences between "runner" and "author" user context:

• Runner (default): When the backing script's user context is set to "runner", any custom script created from the script template will use the identity and resources available to whoever runs the custom script. Note that the user context value of the custom script will be ignored.
• Author: When the backing script's user context is set to "author", any custom script created from the script template will:
  • Run on the backing script's author's compute cluster
  • Only have access to resources available to the backing script's author
  • Any objects created by the script will be owned by the author (though see below about run outputs)
  • For Container, R, and Python scripts, the script will run with the author's API key (using the author's identity). The user ID of the "runner" is available via the RUNNER_USER_ID environment variable. The API key of the “runner” is available via the RUNNER_CIVIS_API_KEY environment variable.

Note that User Context may only be updated via the Platform API.

Run Outputs

When a script running in author context publishes a run output, Civis will automatically re-assign ownership of the object from the script author to the script runner. This happens immediately as the run output is published. This means that the backing script itself (which is running with the author's API key) will instantly lose access to the object. 

If the custom script has a target project set, then the object will be added to the target project after its ownership is reassigned to the runner. The target project should be updatable by the runner (not the author).

Passing data between user contexts

When designing scripts to run in the author's user context it is advisable to treat the author's context and the runner's context as mutually exclusive. In other words, assume that the runner and author will not have access to each other's objects and cannot share objects directly with each other.

The recommended way to pass data from the runner's context to the author's context is by using script parameters. When data is too large or complex to represent as raw values write the data to a Civis File and use a file-type parameter to your script. Your code will be given a URL from which the file contents can be directly downloaded.

The recommended way to return data from the author's context to the runner's context is to use run outputs. As mentioned above, ownership of objects published as run outputs will automatically be reassigned to the script runner.

 

Sharing Script Templates

By default, your primary group will have edit access on any script templates you create.  If you wish to share a template with users outside of your primary group, you can do this through the Platform UI or the Civis API. Note that only view access is required to create custom scripts from a template, edit access enables users to update the template name, and manage access allows users to share the template. 

Sharing the Template Using the UI

On the script template page, you can access the share pane from the standard action menu. From the share pane you can share the script template with any user or group.

Sharing the Template Using the API

In the API documentation, you'll use the Templates Scripts endpoints, depending on whether you want to share it with a user or group.  In this document we'll be interacting with the API through the web browser. Read about using the API with your browser here before moving forward.

To complete the API call, you'll need the following pieces of information:

• Script Template ID
• This can be obtained from https://api.civisanalytics.com/templates/scripts.
• User or Group ID
• This can be obtained from https://api.civisanalytics.com/users or https://api.civisanalytics.com/groups, depending on who you're sharing with.

Remember you can use JavaScript Scripts to interact with the API.  If I wanted to share a template ID of 1 with a group ID of 2, I would execute the following JavaScript:

civis.api.call("put","/templates/scripts/1/shares/groups",{"groupIds":[2],"permissionLevel":"read"})

 

Updating Script Templates

After a script template is created, only the name and sharing permissions are editable. It is not possible to change which backing script the template is backed by. You can, however, freely update the backing script. Custom scripts based on the template will automatically inherit all changes from the backing script.

You can update the name and permissions on a script template either through the UI or the API. You can read more about updating script template permissions here. 

If you are using a container script as the basis for your script template, updates to your GitHub repository will also be picked up by future runs of custom scripts based on your template, unless you have specified a particular GitHub commit to use in your container script. Similarly, if the backing script no longer has access to your Github repository, all custom scripts based on that template will fail.

Creating Custom Scripts from a Template

When a user wants to use your template, they will create a custom script by selecting your template from the script templates pane,  selecting “New Custom Script” on the script template detail page, or making an API call that references your script template by ID. This new custom script behaves like any other script - it can be run, it has logs, it can be scheduled, shared, added to projects, etc. However, unlike a normal script, this custom script has *no* code or configuration. When the custom script is run, the Civis Platform will use the code and configuration from the script backing script template.

API Example: Create a custom script using a template.

POST /scripts/custom

{

  "name": "My new custom script",

  "fromTemplateId": 512,

  "arguments": {

    "state": "IL"

  }

}

 

Parameterized Script Templates

If you add parameters to the script backing your template, then custom scripts that reference the template will accept those same parameters. Default values set by the backing script will be used if the users do not supply values in their own custom scripts. Each custom script will have its own distinct parameter values.

 

Producing Script Templates & Parameterizing Scripts Training Video

Watch our short video below for how to produce script template and how to parameterize scripts.