The purpose of this doc is to describe the differences between Civis’ two AI tool offerings, and provide a few example scenarios in which they might be useful to your organization.

Choosing your AI Tool

There are two Civis AI tools for interacting with generative AI language models:

• Civis AI Core is a general purpose tool that can be used for a variety of text analysis or generation tasks. A few examples are provided below. You can provide instructions and examples for your task, and it can securely process text data which is in Civis Platform.
• Civis AI Taxonomer is a tool built solely for text categorization (i.e., text classification), where the goal is to label a collection of texts (e.g., phrases, sentences, entire documents) according to a set of known categories.

For text categorization tasks, you may want to try both tools. Civis AI Core is more flexible because you can revise its prompt to include detailed instructions or examples, but it requires more setup and uses considerably more credits. For other tasks, such as those involving summarization or text generation, Civis AI Core is the tool to try.

There are two versions of the Civis AI Core script template, one for Anthropic Claude models and one for Meta Llama models. By using a template, you agree to the respective terms of use linked from the template. In general, we recommend trying Anthropic’s Claude model first if the terms of use are acceptable. (Note: Civis AI Taxonomer uses a model from Cohere, and it also has terms of use as discussed in the template.)

Use Case: Summarization

In many organizations, small teams struggle with data analysis due to limited resources and time. For example, an organization might need to analyze a large dataset to identify key trends, high points, and low points to make informed decisions. Manually sifting through this data can be overwhelming and time-consuming.

Civis AI can help users gain insights about large collections of text data by providing detailed summaries. These summaries can aid understanding and help the user generate hypotheses for further analyses. The summarization capability is also flexible. For example, you can ask Civis AI to produce a summary of a specific length, to include examples in the summary, or to suggest potential categories for the data.

Example: Summarizing Free-form Text Survey Responses

Requirements

A table with a column of texts (e.g., documents, emails, survey responses) to be summarized. 

Running Civis AI

1. Navigate to “Civis AI” in Platform’s top navigation menu and open a new Civis AI Core script.
2. Add a System Prompt. In this prompt you should describe how you want the summary to be written.
   
   1. Example: You are a data scientist writing an executive summary for your stakeholders to understand this table’s general content.
3. Add a User Message. This prompt is for you to add the actual requested action. To get the best response from the model, write a clear and detailed description of the data and what the model should do.
   
   • Example: Below is a list of texts that are responses to the question, “What do you think about this policy of mandating that 100% of energy in the US come from renewable sources?” Please provide a summary of the responses.

              {{{ all_texts }}} 

• • Note: if your collection of texts is long, you may want to repeat the specific task at the end (e.g., “Now please provide a summary”).

•  

4. Fill out the appropriate input and output fields for your database.
5. Optionally, set the Sample Size to 0, which tells the model to process all the data. We recommend trying the default value or another nonzero value first so that you can check that the output looks reasonable before processing all of the data.
6. Run the script.
7. Civis AI will create a brand new summary that can be accessed by selecting from the specified output table. The output is just one record with an “output” field containing the summary.

Results

Below is a summary produced for this example. The format of the summary can be adjusted by adding instructions to the system prompt (e.g., “Provide examples along with your summary” or “Make the summary 5-10 sentences long”).

 

 

Token Estimate

The example above used approximately 70 credits using the Civis AI Core (Anthropic Claude) template and the default Anthropic Claude 3.5 Sonnet model to analyze approximately 6,000 brief open-ended survey responses. The credit usage for a job is visible in the logs in the job history.

 

Use Case: Generating Content

Civis AI offers a content generation solution by using samples of the most effective past content—like high-performing emails—to generate new messages. By providing a prompt with successful previous emails, the AI can craft fresh, tailored content that mirrors the effectiveness of earlier communications.

This technology allows teams to produce a larger volume of high-quality, personalized emails efficiently. By automating the content creation process, Civis AI helps teams maintain strong engagement with supporters while freeing up time for other strategic tasks. In the realm of digital communication, Civis AI proves to be an invaluable tool for achieving significant results with less effort.

Example: Drafting Fundraising Emails

Requirements

You will need a database table with the previously created content that you’d like to model your new content on. 

For this example, we use some of Civis’s past marketing emails to write drafts for emails about Civis Platform features. Note that if your table includes HTML, you may use less tokens and get better results by removing the HTML.

Running Civis AI

1. Navigate to “Civis AI” and open a Civis AI Core script.
2. Add a System Prompt. This prompt should lay out the context of how the email should be written, for example, what persona you want the AI to use.
   1. Example: You are a marketing professional who is helping write emails for the Civis Platform software product.
3. Add a User Message. This prompt is for you to add the actual requested action. To get the best response from the model, provide as much detail as possible. 
   1. Example: Here are some past Civis marketing emails as examples of style and structure.

{{{ all_texts }}}

 

Write up a draft email encouraging users to try Civis Platform's notebook functionality. Here are some talking points:

- allows for interactive coding and data exploration

- lets you work in a secure environment

- you can set compute and memory requirements to scale up for larger datasets

- supports notebooks in either Python or R

- integrates with GitHub and DockerHub

4. Fill out the appropriate input and output database fields.
5. Set the Sample Size to 0, which tells the model to use all the available data. 
6. Optionally, specify a higher value than the default 0.0 for the “Temperature” parameter to get more “creative” results. Alternatively, you could modify the prompt to ask the model to output multiple drafts.
7. Run the script
8. Civis AI will create a brand new email based on the style and structure of your examples. You can run it multiple times to produce different draft emails.

Results

Below is an example of the output you will receive after running the script to get an email. You can see a rough output in the run history pane which can be useful for iterating, or access the results more cleanly by querying the output table.

 

 

Token Estimate

The example above used just under 5 credits with the Civis AI Core (Anthropic Claude) template and the default Anthropic Claude 3.5 Sonnet model. The credit usage for a job is visible in the logs in the job history. The input table for this example consisted of only 8 emails, so the credit usage would be somewhat higher with more input data. However, input length has less effect on credit usage than output length.

 

Use Case: Categorization

Many organizations struggle to make sense of large, unstructured datasets. For instance, an organization might have a substantial amount of data, but without proper categorization, the information can be overwhelming and difficult to use.

Civis AI provides a solution by efficiently categorizing unstructured data. The AI analyzes the dataset to identify patterns and classify information into meaningful categories. This allows teams to quickly make sense of large volumes of data, highlighting key insights and trends that would otherwise be obscured. This capability streamlines data analysis, enabling teams to focus on strategic tasks and leverage the data more effectively.

Example: Sentiment Analysis

Requirements

You will need to review your data and create appropriate categories with clear definitions. As an example, one popular way to categorize text data is by sentiment polarity — whether a text expresses a positive, negative, or neutral sentiment. If you are unsure as to what categories might be appropriate, you can ask Civis AI to first suggest potential categories for the data. 

 

Note: Our Civis AI: Taxonomer tool can categorize data at a much lower token count, however using Civis AI: Core may provide you with more flexibility in your outputs.

Running Civis AI

1. Navigate to “Civis AI” and open a new Civis AI Core script.
2. Add a System Prompt. This prompt should give context of the problem, including what the available categories are and some examples of responses for each category.
   1. Example:
      You are tasked with categorizing responses to a survey question asking, “What do you think about this proposed policy of increasing the national minimum wage to 15 dollars an hour?”. The three possible categories relate to the sentiment of the response: “positive”, “negative”, and “neutral”. Respond with just the category, without preamble.
      
      

<example>

H: <text>Will just cause inflation.</text>

A: negative

</example>

 

<example>

H: <text>It’s a great idea that will help workers.</text>

A: positive

</example>

 

<example>

H: <text>not sure</text>

A: other

</example>

3. Add a User Message  This prompt is for you to add the actual requested action. To get the best response from the model, provide as much detail as possible.
   1. Example:

Here is the text to categorize:

{{{ single_text }}}

4. Fill out the appropriate input and output database fields, including “Input Text Column” and “Input ID Column” to specify the columns with the text data and corresponding unique IDs.
5. Optionally, set the Sample Size to 0, which tells the model to process all the data. We recommend trying the default value or another nonzero value first so that you can check that the output looks reasonable before processing all of the data.
6. Run the script
7. In the Query tool, select from the output table that you specified to view the results. 

Results

Listed below are the example results you will see by querying the output table you specified.

Using the ID column that you specified, you can use a SQL join to combine this with your input table to see which output corresponds to which input.

 

You could also, for example, make a SQL Report with a query like the following to visualize the distribution of categories.

 

select
   case when lower(output) in ('positive', 'negative', 'neutral')
       then lower(output)
       else 'other' end
       as category,
   count(*)::float
       / (select count(*) as total_count from output_table)
       as proportion_of_records
from output_table
group by 1

 

Token Estimation

This example job used approximately 0.15 credits per input row with the Civis AI Core (Anthropic Claude) template and the default Claude 3.5 Sonnet model. So, to categorize 1,000 records, it would use 150 credits. Note that there are some cheaper model options available, such as Claude 3 Haiku, which would only use about 12 credits for 1,000 records. As mentioned above, Civis AI Taxonomer could also be used for categorization but is less flexible. In this case, Taxonomer would use <0.5 credits for 1,000 records.