Skip to main content
Code42 Support

Code42 API Script Recipes For Use With Splunk

Applies to:
  • Code42 CrashPlan (previously CrashPlan PROe)

Overview

This article provides examples of scripts that leverage the Code42 API to retrieve useful data. The scripts are particularly useful when integrated with Splunk Enterprise, for use in data analytics, data visualization, and audits.

For basic information on setting up your Code42 environment with Splunk, see Analyzing Data With Splunk And The Code42 API.

Considerations

System requirements

The scripts in this article are written for the Bash shell interpreter. The following software is required:

Linux and Mac OS X provide a Bash shell by default. Windows devices would require extra software, such as Cygwin, to run the scripts.

Splunk Enterprise installation and integration with the Code42 environment

Please see Analyzing Data With Splunk And The Code42 API for a detailed tutorial on installing Splunk Enterprise and integrating Splunk with your Code42 environment.

You may install the scripts (and Splunk Enterprise) on any server that meets the minimum system requirements, and that can connect to the Code42 server via http or https.

This article is applicable to Splunk Enteprise versions 6.0 and later.

Non-Code42 products
​Information about products from other manufacturers is intended as a resource to help you get the most out of Code42 products. However, our Customer Champions cannot provide direct assistance for these products. For assistance with products not developed by Code42, contact the product's manufacturer.

Code42 app for Splunk
Install the Code42 app for Splunk for easier integration of your Code42 environment with Splunk Enterprise. The Code42 app for Splunk provides visibility into your Code42 environment for the purposes of capacity planning, performance monitoring, security monitoring, and user management.

Installation steps

  1. Copy the text of the scripts below (contained in the boxes under the Script heading) into your favorite text editor (vim, nano, Notepad, etc.), then save as a file with the extension "sh".
    • For example, save the User script as the file "user.sh"
    • Use a descriptive filename so that you can easily know which API endpoint the script uses
    • Save the scripts into the Splunk script directory:

      <your_install_dir>/splunk/bin/scripts
      
  2. Set the file system permissions and/or owner to a value that allows the Splunk process to run the script
  3. Add the scripts as a data source in Splunk Enterprise.

Example scripts

User

Suggested script filename: user.sh

Use case

This script retrieves information on all active users. See the Code42 server API doc viewer for list of methods, query parameters, and output values for the User resource of the Code42 API.

Script

/usr/bin/curl -ku 'username:password' "https://<your_proe_server>:4285/console/api/User?srtKey=name&srtDir=asc&incAlertCounts=true&incBackupUsage=true&incRoles=true&incComputerCount=true&targetComputerGuid=rollup&active=true&alerted=false&invited=false&admins=false&export=true"

Computer

Suggested script filename: computer.sh

Use case

This script retrieves information on all active devices. See the Code42 server API doc viewer for list of methods, query parameters, and output values for the Computer resource of the Code42 API.

Script

/usr/bin/curl -ku 'username:password' "https://<your_proe_server>:4285/console/api/Computer?srtKey=name&srtDir=asc&targetComputerGuid=rollup&incBackupUsage=true&incActivity=true&incCounts=true&active=true&alerted=false&export=csv"

Organization

Suggested script filename: organization.sh

Use case

This script retrieve information on your organizations. It exports the information as a CSV file. See the Code42 server API doc viewer for list of methods, query parameters, and output values for the Org resource of the Code42 API.

Script

/usr/bin/curl -ku 'username:password' https://<your_enterprise_server>:4285/api/org?export=1

Destination

Suggested script filename: destination.sh

Use case

This script retrieves information on destinations. It exports the information as a CSV file. See the Code42 server API doc viewer for list of methods, query parameters, and output values for the Destination resource of the Code42 API.

Script

/usr/bin/curl -ku 'username:password' https://<your_enterprise_server>:4285/api/destination?export=1

Archive metadata

Suggested script filename: ArchiveMetadata.sh

Use case

The Archive Metadata script is able to get a list of files, file sizes, and file types in an archive or list of archives. This can be extremely useful in a number of cases, such as choosing file exclusions. See the Code42 server API doc viewer for list of methods, query parameters, and output values for the ArchiveMetadata resource of the Code42 API.

The archivemetadata example
This example is more complex than the others, because :
  • The ArchiveMetadata resource produces json output, which must be converted by the script to CSV format.
  • The script requires you to download and install a file named "json.map", as well as some additional supporting Python modules. This code is needed to do the conversion of json data to CSV format.

The additional steps are documented below.

Additional requirements

The installation of this script and supporting software requires the pip tool, used for installing and managing Python packages.

Install pip on your server using your system's package manager. For example, on Debian-based Linux distributions, enter the following command:
sudo apt-get install python-pip

Script

guid="$1"
/usr/bin/curl -ku 'admin:admin' https://<your_enterprise_server>:4285/api/ArchiveMetadata/$guid?decryptPaths=1 > metadata.json
python json2csv/json2csv.py metadata.json json2csv/json.map -o metadata.temp.csv
cat metadata.temp.csv

Important note: this script requires the command-line argument guid to be passed to the script. For example, the entry that calls the script in Splunk Enterprise might read as follows, where 631663249095393537 is the guid of your Code42 server:
/opt/splunk/bin/scripts/ArchiveMetadata.sh 631663249095393537

Remember to set the permissions on the new script to a value that will allow Splunk to execute the script.

JSON

The ArchiveMetadata API endpoint sends data in the json format, which is not as easily interpreted by Splunk as other formats. Because of this, the script above converts the json data to CSV format. This conversion requires some additional code in order to function.

Install Python conversion script and supporting modules

The Python script json2csv.py does the actual conversion of the json data to CSV format.

You can download and install the script from github, using the following steps:

  1. Go to the Splunk script directory:
    <your_install_dir>/splunk/bin/scripts
    
  2. Enter this command to pull the json2csv.py script from github:
    git clone https://github.com/evidens/json2csv.git

  3. Navigate to the json2csv folder and run this command to install the supporting Python modules:
    pip install -r requirements.txt

    • Example:

root@gamma:/opt/splunk/bin/scripts# cd json2csv/root@gamma:/opt/splunk/bin/scripts/json2csv#
root@gamma:/opt/splunk/bin/scripts/json2csv# pip install -r requirements.txt
Downloading/unpacking unicodecsv==0.9.0 (from -r requirements.txt (line 1))
  Downloading unicodecsv-0.9.0.tar.gz
  Running setup.py egg_info for package unicodecsv

Installing collected packages: unicodecsv
  Running setup.py install for unicodecsv

Successfully installed unicodecsv
Cleaning up...

You must also create a json "outline file", to specify how to convert the data returned by the ArchiveMetadata API endpoint:

  1. Navigate to the directory json2csv, which should now exist in the Splunk Enterprise scripts directory
  2. Use a text editor to create the file json.map
  3. Copy the text below into the file json.map
{  "map": [
   ["path", "path"],
   ["sourceLength", "sourceLength"],
   ["sourceLastModified", "sourceLastModified"]  ],
 "collection": "data" }
  1. Save the file and quit the text editor

Use a loop construct to acquire data on multiple guids

The ArchiveMetadata script is most useful when it is used to acquire data from multiple GUIDs at the same time. In order to do this, the best method is to create an additional script that actually calls the ArchiveMetadata script.
Below is an example Python script that looks up metadata information for four different GUIDs or devices (replace the example GUIDs with GUIDs from endpoint devices in your Code42 environment):

for guid in 631663249095393537 625012394242917251 631663249095393537 631663249095393537
do
    ArchiveMetadata.sh $guid
done

In this case, the additional script that passes a number of GUIDs to the ArchiveMetadata script (ArchiveMetadata.sh) would be the actual script that you would call from Splunk Enterprise. You might name the script above something like ArchiveMetadataMultipleGuids.sh, for example, and then add ArchiveMetadataMultipleGuids.sh as a data source in Splunk Enterprise. Splunk Enterprise would then run the script at intervals configured in your Splunk settings.

Add scripts as a data source in Splunk

In order for Splunk to use your scripts, you must add them as data sources in Splunk. These steps should be repeated for each individual script. This example shows how to add the Archive Metadata script as a data source.

  1. Go to Splunk Home
  2. Click Add Data in the Data panel at the upper right
    API script recipes add data panel
  3. Choose Run and collect the output of a script in the Or Choose a Data Source panelAPI script recipes add script
  4. Enter the path to the script in the Command field in the Source sectionAPI script recipes script path
  5. Set the source type to CSV
    API script recipes source CSV
  6. Select More Settings
  7. Change the Host field value to a value that makes it easier to search for the results of the script separately from other scripts. In this example, since we are invoking the ArchiveMetadata resource to look at filetype data, we set the Host field value to"Filetypes"
    API recipes host change
  8. Leave all other settings at the default values
  9. Click Save
    A success message appears:
    API recipes successful data input addition

That's it! The archive metadata information is now being indexed by Splunk Enterprise.

Visualization of API scripted inputs

The Analyzing Data With Splunk And The Code42 API tutorial explains the process of creating dashboards and panels that can be used to visualize your data, but here is a short explanation of the process. Let's use the data generated by the Archive Metadata script to create two charts:

  • File Types By Count
  • File Types By Storage Amount

File types by count

Step 1: Create the search

  1. Enter the following search into the Splunk Enterprise search field:
host="Filetypes"  | rex field=_raw "(?<type>\.([A-Za-z]{2}\w)(?!\w))"  | eval type=lower(type)| top limit=20 type

Example results:

API script recipes filetypes search results

Step 2: Save the search

  1. Choose Save As > Dashboard Panel from the upper right of the screen:
    API script recipes save search as dashboard
  2. You can save the search to a new or existing dashboard. In this example, we will save the search to a new dashboard named "File Types." Be sure to choose the pie chart as the type of panel content, and give the panel a reasonable name, such as "File Types by Count":API script recipes file types by count save
  3. Click Save
  4. Choose View Dashboard from Your Dashboard Panel Has Been Created dialog box.

Your new dashboard is displayed, including the File Types by Count panel that shows the distribution of different file types in the archives:

API script File Types by Count

File types by storage amount

Step 1: Create the search

  1. Enter the following search into the Splunk Enterprise search field:
host="Filetypes" | rex field=_raw "(?<type>\.([A-Za-z]{2}\w)(?!\w))" | eval type=lower(type) | stats sum(sourceLength) as SumType by type | sort -SumType | head 100

Example results:

API script recipes filetypes by storage amount search

Step 2: Save the search

  1. Choose Save As > Dashboard Panel from the upper right of the screen
    API script recipes save search as dashboard
  2. You can save the search to a new or existing dashboard. In this example, we will save the search to the dashboard that we just created, named "File Types." Be sure to choose the pie chart as the type of panel content, and give the panel a reasonable name, such as "File Types by Storage Amount"
    API script recipes filetypes by storage amout save
  3. Click Save
  4. Choose View Dashboard from Your Dashboard Panel Has Been Created dialog box
    API recipes view dashboard

Your new dashboard is displayed, including the two panels you created
API script recipes Filetypes Dashboard

External resources

  • Was this article helpful?