The Generic Bulk Export Civis Data to NGPVAN script template allows you to export multiple rows of data to NGPVAN using only one NGPVAN API request, which should be faster than the regular Export for many rows of data. It supports any arbitrary mapping type that NGPVAN allows to their bulkImportJobs API endpoint.

You can create your script using this link, or in Platform you can navigate to Code → Scripts → More Script Templates and search for “85207”.  If you are not shared on this template, please contact support.

Parameters

Cluster (required)

Name of the cluster where data to be exported is stored.

Database Credential (required)

A valid database credential for the cluster

Actions JSON (required)

This is used directly for the "actions" property to the bulkImportJobs endpoint to the NGPVAN API. Must be a JSON array.  See the "Action JSON Examples" section below for more details.

Queue View (optional)

The full table name (including schema) of a Civis table to export. Each row will be uploaded as a unique record in NGP or VAN. Accepted fields vary depending on bulk import mapping type used in the Actions JSON. Note NGPVAN currently supports a maximum of 50 columns from the input table/query.

One of Queue View or SQL Query are required.

SQL Query (optional)

A complete SQL query defining the data to export to NGP or VAN. Each row of the resulting query will be uploaded as a unique record in NGP or VAN. Accepted fields vary depending on bulk import mapping type used in the Actions JSON. Note NGPVAN currently supports a maximum of 50 columns from the input table/query.

One of Queue View or SQL Query are required.

Response Table (optional)

Table to output results from the bulk import job on NGPVAN.

Response Table Setting (optional)

Drop or append API records to the response table. Note that "append" will only work if you explicitly set "resultFileColumnName" in the Actions JSON to make sure the result column names are consistent between runs.

Database Mode (required)

The NGPVAN database to upload data - One of MyVoterfile or MyCampaign. Only MyCampaign is currently supported.

NGPVAN (required)

A custom Civis credential with a username and password for NGPVAN. Note for resources supported in both NGP and VAN (ex: people), the credential determines the destination for the export. Click here for credential creation info.

Script Mode

The generic bulk NGPVAN export uses a paradigm of script “modes”, where different methods are executed by running the script in a different mode.

The modes available are:

• “help” - Get help 
• “test_connection” - Test connectivity to NGPVAN API
• “list_objects” - List mapping types available
• “run” - Run sync (default)
• “view_config” - Output current configuration as a file output

Log Level

“DEBUG” - Log details of all API calls.

“INFO” - Log informational messages about program execution.

“WARNING” - Log potential errors and other warnings.
“ERROR” - Logs only unrecoverable errors in program execution.

Examples

Apply Organizing Turfs

In order to determine the accepted fields for each bulk export mapping type, run the script in the “list objects” script mode. In the logs, you will see the field name accepted for each mapping type. Since organizing turfs are applied via custom fields, these are unique to each account. For example, in Civis’ sandbox, one custom field name is "CF327” with description “Custom Field Civis Administrative Region”, but passing “CF327” through a different VAN committee wouldn’t work. Use these field names to define your table input to “queue view” or defined in the “SQL query” param. The names are case insensitive.

For a resourceType of Contacts, and a mappingType of anything except CreateOrUpdateContact, the first field name of the input must be named "VanId" (case insensitive) and contain the associated VAN id of the contact you're updating.

In the Actions JSON Value parameter, indicate which mapping type should be used. Example JSON below.

[{ 

    "resultFileSizeKbLimit": 500000000,

    "resourceType": "Contacts",

    "actionType": "loadMappedFile",

    "mappingTypes": [

        {

            "name": "OrganizingTurfs",

            "resultFileColumnName": "OrganizingTurfs"

        }

    ]

}]