Civis Files Import

The Import from Civis Files script template (ID 302831) imports from Civis files to a Civis Platform table. The specified Civis files must be CSV-like, either as plain text files or as compressed files. For file compression, only gzip (.gz) and bzip (.bz2) are auto-detected and supported.

This template ID 302831 replaces the older template ID 9002 with the same import functionality. Users of the older template 9002 are encouraged to migrate to the newer template 302831. To facilitate transition, the older template 9002 will remain as-is and won’t be removed.

Parameters

• Civis Files iDs (required): A comma-separated list of Civis File IDs to import data from.
• Database (required): The Civis Platform database to which to write records. Provide all of the parameters “Database”, “Database credential” as well as “Schema and Tablename” if importing data to a Civis database table, or specify none of them when importing data as Civis files instead.
• Database Credential (required): A valid database credential for the Civis Platform database. To use the credential associated with the logged-in user, choose from the dropdown the credential that has this user’s Civis Platform username. Provide all of the parameters “Database”, “Database credential” as well as “Schema and Tablename” if importing data to a Civis database table, or specify none of them when importing data as Civis files instead.
• Schema and Tablename (required): The schema and table name you would like to import the data into, e.g., schema.table. Provide all of the parameters “Database”, “Database credential” as well as “Schema and Tablename” if importing data to a Civis database table, or specify none of them when importing data as Civis files instead.
• Column Data Type Overrides (optional): A mapping from column names to data types to use instead of the data types from the data source. Each line should be in the form of "column_name: DATA_TYPE" (without quotes), e.g., "foo_bar: VARCHAR". Only provide an entry for columns for which you want to override the data type; do not provide entries for columns that should use the default data type from the data source. This parameter is useful for data type issues that would otherwise cause the import to fail, e.g., use "foobar: VARCHAR" if the data source says "foobar" should be a numeric type but it actually contains non-numeric values.
• Existing Table Exists (optional): The action to take if a table with the requested name already exists. Options are:
  • Append: Records will be appended to the existing table
  • Fail: The sync job fails if the table exists
  • Drop: Removes the entire table including column information.
  • Truncate: Records on the table will be wiped out while maintaining the current columns.
• Has Header (optional): If true (the default), the first row in all files is the column header.
• Column Delimiter (optional): Column delimiters, one of comma, tab, or pipe. Default: comma.

• Primary Keys (optional): A comma-separated list of column names as primary keys when the "Existing Table Rows" parameter is set to "upsert".
• Last Modified Keys (optional): A comma-separated list of column names indicating last modified timestamps when the "Existing Table Rows" parameter is set to "upsert".
• Redshift DISTSTYLE (optional): The DISTSTYLE to use when creating a new table in Redshift. This parameter is only applicable when the target database is Redshift and the "Existing Table Rows" parameter is set to "drop", "truncate", or "fail".
• Redshift DISTKEY (optional): The column to use as the DISTKEY when creating a new table in Redshift. This parameter is only applicable when the target database is Redshift, the "Existing Table Rows" parameter is set to "drop", "truncate", or "fail", and the "Redshift DISTSTYLE" parameter is set to "KEY".
• Redshift SORTKEY 1 (optional): The column to use as the first SORTKEY when creating a new table in Redshift. This parameter is only applicable when the target database is Redshift and the "Existing Table Rows" parameter is set to "drop", "truncate", or "fail".
• Redshift SORTKEY 2 (optional): The column to use as the second SORTKEY when creating a new table in Redshift. This parameter is only applicable when the target database is Redshift and the "Existing Table Rows" parameter is set to "drop", "truncate", or "fail".