Skip to main content
Code42 Support

Analyzing Data With Splunk And The Code42 API

Applies to:
  • CrashPlan PROe

Overview

This tutorial explains how to install, configure and test Splunk Enterprise, and then integrate it with the Code42 API. Splunk Enterprise is a solution for data analytics, monitoring, and visualization. The use of Splunk Enterprise provides enterprises with customized solutions to leverage the data available through the Code42 environment. This data is useful for:

  • Resource usage monitoring
  • Security audits
  • Security monitoring
  • Application and user management
  • Compliance
  • Capacity planning
  • Performance and reliability monitoring
  • Leveraging the Code42 API
  • Many other data analytics needs

This tutorial is just the initial introduction to the process of integrating your Code42 environment with Splunk, demonstrating a subset of the functionality available. The possible applications of a Code42 environment integrated with Splunk Enterprise are infinite, limited only by the demands you generate for useful data.

For further examples on using your Code42 environment data with Splunk, see API Script Recipes For Use With Splunk.

Considerations

Code42 API

The Code42 API is a powerful HTTP-based API that you can use to generate reports, acquire information, perform automated actions, and integrate the Code42 environment seamlessly into your overall environment. This tutorial focuses on the integration of the Code42 API with Splunk. Keep in mind, however, the following:

  • Splunk can also search and index enterprise server log files, as well as the system logs of your servers.
  • You can install a data forwarder, if necessary, to send data from log files to your Splunk Enterprise server.
  • The Code42 API as a source of data has advantages over unstructured log files, which is why this tutorial leverages the API. Some of these advantages:
    • more focused data, based on parameters passed to the API
    • less bandwidth utilization
    • faster results

Splunk

Splunk Enterprise is a commercial product that provides tools to store, index, search, monitor, analyze and visualize machine-generated data.

This tutorial uses Splunk version 6.

Licensing

Although it requires a license, you are able to use a 30-day free trial to initially install and configure Splunk Enterprise.

Support

The information that pertains to Splunk Enterprise is provided for your convenience. Our Customer Champions are not able to assist with Splunk-specific steps or issues.

Before you begin

You should have a basic familiarity with the Code42 API. Two good places to start are:

This tutorial assumes that you have administrative access to at least one server running the Code42 environment. We recommend using a test server or virtual machine for initial installation and testing, rather than your production enterprise server.

Additionally, you should choose one of two options before beginning:

  • Install Splunk Enterprise on one of your enterprise servers.
    • No need to build a new server
    • Splunk Enterprise will have access to server logs without needing forwarders installed
  • Install Splunk Enterprise on a dedicated server, either on a VM or dedicated hardware
    • Avoids any impact on performance or configuration of your enterprise server
    • Splunk Enterprise will need additional configuration to connect to your enterprise server

System requirements

Most major, recently-released desktop and server operating systems are supported. Splunk Enterprise can run on a separate server from the Code42 environment. In fact, running Splunk Enterprise on a dedicated server is a recommended best practice, as Splunk's system usage will not affect the performance of you Code42 environment.

For details on supported operating systems, see Splunk Enterprise's:

Install Splunk Enterprise

Before you begin

Review the system requirements for Splunk Enterprise.

Steps

For full installation instructions for all platforms, see Splunk's installation manual.

This tutorial will cover installing Splunk Enterprise on a Linux server using the .deb (Debian package) installation file type (Ubuntu and Debian, for example, use this method).

Step 1: Download installer

Download the installer for your Linux server from Splunk's download site.

Step 2: Run dpkg on the installer file

As root, or using the sudo command, run the following command in a terminal window or shell, while in the directory containing the downloaded Splunk installation file:
dpkg -i splunk_package_name.deb

Splunk will be installed in the directory /opt/splunk by default.

You may want to add the Splunk installation directory to your environment's PATH variable. To make the changes to your PATH variable permanent, add these export commands to your .bashrc file:

# export SPLUNK_HOME=/opt/splunk
# export PATH=$SPLUNK_HOME/bin:$PATH

Step 3: Start Splunk

From the Splunk bin directory, enter the following command:
./splunk start

If you have added the Splunk installation directory to your environmental PATH variable, you can always find the location of the Splunk executable by entering the following command (command shown with results):

root@myServer:~# which splunk
/opt/splunk/bin/splunk

Step 4: Accept license agreement

The first time you run Splunk, you must accept the license agreement. To start Splunk and accept the agreement in one step, you may simply enter the following command to run the application for the first time:
$SPLUNK_HOME/bin/splunk start --accept-license

Step 5: Sign in to Splunk Web

  1. Open a supported web browser
  2. Enter the following URL:
    http://<hostname>:<port>
    • <hostname>: the IP address or hostname of your Splunk Enterprise server.
    • <port>: the port number you chose during installation
  3. Enter the default username admin and the default password changeme the first time you sign in.
    Splunk recommends changing the default password immediately.

Step 6: Learn about basic functionality from Splunk's documentation

Splunk is a feature-rich and customizable application. You may want to read about some of Splunk's features, and how they work, before proceeding with the next steps in this tutorial, to gain a basic understanding of the product.

Create example Api scripts on Splunk server

Before you begin

  • This step requires the use of a text editor such as vim or Notepad that does not add extra characters or text to the files.
  • Requires root access to the server running Splunk Enterprise
  • This part of the tutorial guides you in the creation of four shell scripts that access the Code42 API. You create them on the file system of the Splunk Enterprise server:
    • user.sh
    • computer.sh
    • organization.sh
    • destination.sh
Unsupported scripts
These scripts were written as demonstration scripts, to show you how to integrate your own scripts with Splunk Enterprise. However, they provide real, useful data. They are provided without warranty or support.

Steps

Step 1: Sign in to Splunk server using SSH

  • Using ssh or another secure method, sign in to your Splunk Enterprise server.
  • Switch to root or another user with the necessary privileges
  • Using the "cd" command, change your working directory to the Splunk bin/scripts directory:
joe@splunkServer:~$ su -
Password:
root@splunkServer:~# cd /opt/splunk/bin/scripts/
root@splunkServer:/opt/splunk/bin/scripts#

Step 2: Create the user.sh script

Using your favorite text editor (e.g. vim, Emacs, pico, etc.), create the user.sh script:

  1. Open the text editor
  2. Copy the text below into the editor:
    •  /usr/bin/curl -ku 'username:password' "https://[your_proe_server]:4285/console/api/User?srtKey=name&srtDir=asc&incAlertCounts=true&incBackupUsage=true&incRoles=true&incComputerCount=true&targetComputerGuid=rollup&active=true&alerted=false&invited=false&admins=false&export=true"
      
  3. Replace username:password with your username and password.
  4. Replace [your_proe_server] with your enterprise server's IP address or domain name.
  5. Save the user.sh script then quit the editor.

Step 3: Create the computer.sh script

  1. Open the text editor
  2. Copy the text below into the editor:
    • /usr/bin/curl -ku 'username:password' "https://[your_proe_server]:4285/console/api/Computer?srtKey=name&srtDir=asc&targetComputerGuid=rollup&incBackupUsage=true&incActivity=true&incCounts=true&active=true&alerted=false&export=csv"
      
  3. Replace username:password with your username and password.
  4. Replace [your_proe_server] with your enterprise server's IP address or domain name..
  5. Save the computer.sh script then quit the editor.

Step 4: Create the organization.sh script

  1. Open the text editor
  2. Copy the text below into the editor:
    • /usr/bin/curl -ku 'username:password' https://[your_proe_server]:4285/api/org?export=1
      
  3. Replace username:password with your username and password.
  4. Replace [your_proe_server] with your enterprise server's IP address or domain name..
  5. Save the organization.sh script then quite the editor.

Step 5: Create the destinations.sh script

  1. Open the text editor
  2. Copy the text below into the editor:
    •  /usr/bin/curl -ku 'username:password' https://[your_proe_server]:4285/api/destination?export=1
  3. Replace username:password with your username and password.
  4. Replace [your_proe_server] with your enterprise server's IP address or domain name.
  5. Save the destination.sh script then quite the editor.

Add sample scripts as data sources

Before you begin

It is a good idea to get a basic familiarity with Splunk by viewing some of their introductory tutorials.

This section requires:

  • A working Splunk Enterprise installation
  • A working Code42 environment of at least one master server. It is preferable to use a test platform or server, to avoid affecting the performance of your production environment.
  • Splunk is able to utilize many sources of data, such as log files, CSV files, scripts, API calls, and others. This tutorial focuses on using our API. See Splunk's documentation for the complete list of supported data sources.

Steps

Step 1: Sign in to Splunk Enterprise and navigate to Splunk Home

  1. Navigate to your Splunk Enterprise server's URL in a web browser:Splunk Enterprise sign in screen
  2. Enter your username and password.
  3. You will be taken to Splunk Home. If you ever need to return to Splunk Home, simply click the Splunk icon at the upper left of the Splunk Enterprise console:
    Splunk Enterprise home image

Step 2: Go to add data

In the Data panel at upper right, click Add Data:

Splunk Enterprise Add Data button

Your browser navigates to the Add data window, which lets you add data from various sources and types:Splunk Enterprise Add data window

Step 3: Add the user.sh script as a data source

  1. Click Run and collect the output of a script, located under the heading Or Choose a Data Source
    • Splunk Enterprise Choose data source
  2. Fill in the following fields on the Add new page, then click Save:

Splunk Enterprise add user.sh as data input

Field Value to enter Description
Command /opt/splunk/bin/scripts/user.sh Path to the user.sh script
Interval Leave at default of 60.0 seconds How often script is run and data collected
Source name override Users Changes the name of the data source to an easier to read form
Select source type from list csv Tells Splunk the format of the data

If the user.sh script was successfully added, you will see the following:

Splunk Enterprise add input success

Step 4: Add the rest of the scripts

Click Add more data, then repeat Step 3 above for the remaining scripts. Enter the following values into the Source name override field:

  1. computer.sh
    • Enter Computers in the "Source name override" field
  2. destination.sh
    • Enter Destinations in the "Source name override" field
  3. organization.sh
    • Enter Organizations in the "Source name override" field

Create your first dashboard and visualize devices by architecture

Before you begin

In this section, you will create your first dashboard, and add a panel to the dashboard that visualizes the devices in your Code42 environment by their their operating system and CPU type.

Beyond this example
You can add any number of queries, reports, and visualizations to your Splunk Enterprise server installation.

You can create separate dashboards or panels for separate functions. For example, you could have separate dashboards to provide detailed looks at:

  • Users
  • Security-related data and audits
  • Capacity planning and usage
  • Server update and reliability
  • Logs

Steps

Step 1: Create the search

  1. From Splunk Home, click Search from the Search & Reporting app
    • Splunk Enterprise Search app
  2. Enter the following search term into the Search field (replace YourSplunkServer with the name of your Splunk Enterprise server)
    host=YourSplunkServer | dedup guid | top limit=20 osArch
  3. Press Enter or click the search icon Splunk Enterprise search icon
  4. Splunk Enterprise returns the results:Splunk Enterprise Devices by Arch results

Step 2: Explore the results

You'll notice that the results page containts three tabs:

Splunk Enterprise Server three tabs

  • Events
    • Displays the actual events that were indexed and stored by Splunk Enterprise
  • Statistics
    • Displays the results of any statistical or reporting command in your search
    • Results displayed as a table
  • Visualization
    • Displays statistical results as a chart visualization
    • Different chart types can be selected for visualization

You can click on the tabs to familiarize yourself with the information they display.

Step 3: Save visualization results as a panel in a new dashboard

You will now save the visualization as a pie chart, to a new dashboard. If you have not already created a dashboard, this will be your first dashboard.

  1. Click the Visualization tab
  2. Click Save As at the upper right of the page
    • Splunk Enterprise Save As
  3. Choose Dashboard Panel
  4. From the Save As Dashboard Panel window, choose the following options:
    • Dashboard: New
    • Dashboard Title: My First Dashboard (or another title of your choice)
    • Dashboard ID: default
    • Dashboard Description: Visualization of various Code42 environment API data (or another description of your choice)
    • Dashboard Permissions: Share in App
    • Panel Title: Devices by Architecture
    • Panel Content: Pie
    • Splunk Enterprise Save as Dashboard panel
  5. Click Save
  6. Splunk Enterprise displays Your Dashboard Panel Has Been Created:Splunk Enterprise Dashboard created

Congratulations! You have created your first Splunk Enterprise Dashboard and Panel, and have visualized some real data!

Step 4: View dashboard and explore data

You can now view your first Dashboard, and explore your data.

  1. Click View Dashboard
  2. Mouse over the pie chart in the Devices by Architecture window.
  3. You will notice that placing your mouse pointer over segments of the chart displays data related to that chart segment:
    • Splunk Enterprise Devices by Architecture panel mouseover
  4. Click one of the pie chart's segments. You will see a new search that displays the specific data contained in that pie chart segment:Splunk Enterprise Devices by Architecture click chart
  5. This search itself can be saved as a panel or report, by simply click Save As at the upper right of the search window:Splunk Enterprise search results save as

This shows the power of Splunk and your first query. It allowed you to:

  • visualized the query data
  • created a dashboard and panel to save your search in an easily viewable, graphical form
  • narrow your focus to more specific data with a simple click

Let's create some more queries and add more panels to your first dashboard.

Create new panel to visualize devices by os

This section of the tutorial guides you in creating an additional panel in your Splunk Enterprise dashboard. The panel displays a chart that communicates the distribution of the endpoint devices in your Code42 environment based on operating system.

Before you begin

This step assumes that you have already created your first dashboard, as described above.

Steps

Step 1: Create the search

  1. From Splunk Home, click Search from the Search & Reporting app
    • Splunk Enterprise Search app
  2. Enter the following search term into the Search field (replace YourSplunkServer with the name of your Splunk Enterprise server):
    host=YourSplunkServer | dedup guid | top limit=20 osName
  3. Press Enter or click the search icon Splunk Enterprise green search icon
  4. Splunk Enterprise returns the results:

Splunk Enterprise devices by OS search results

Step 2: Save visualization results as a panel in existing dashboard

You will now save the visualization as a pie chart, to a new panel in your existing dashboard.

  1. Click the Visualization tab, to display the pie chart
    • Splunk Enterprise devices by OS pie chart
  2. Click Save As at the upper right of the page
    • Save Query as Dashboard Panel
  3. Choose Dashboard Panel
  4. From the Save As Dashboard Panel window, choose the following options:
    • Dashboard: Existing
    • Dropdown: My First Dashboard
    • Panel Title: Devices by OS
    • Panel Content: Pie
    • Splunk Enterprise Save as Dashboard panel
  5. Click Save
  6. Splunk Enterprise displays Your Dashboard Panel Has Been Created:Splunk Enterprise your Dashboard has been created

Congratulations! You have added another panel to your dashboard. Click View Dashboard to see your dashboard and two panels:

Splunk Enterprise My First Dashboard with two panels

Visualize backup completion

This section of the tutorial guides you in the creation of another panel in your dashboard. The new panel displays a chart that displays the range of backup completion rates for your endpoint devices.

Before you begin

This section assumes you have already created your first dashboard, as described in Create Your First Dashboard And Visualize Devices By Architecture.

Steps

Step 1: Create the search

  1. From Splunk Home, click Search from the Search & Reporting app
    • Splunk Enterprise Search
  2. Enter the following search term into the Search field (replace YourSplunkServer with the name of your Splunk Enterprise server):
    • host=YourSplunkServer | dedup guid | eval percentComplete = round(percentComplete,0) | top percentComplete
  3. Press Enter or click the search icon Splunk Enterprise green search icon
  4. Splunk Enterprise returns the results

Splunk Enterprise backup completion results

Step 2: Save visualization results as a panel in existing dashboard

You will now save the visualization as a pie chart, to a new panel in your existing dashboard.

  1. Click the Visualization tab, to display the pie chart
    • Splunk Enterprise devices by OS pie chart
  2. Click Save As at the upper right of the page
    • Save Query as Dashboard Panel
  3. Choose Dashboard Panel
  4. From the Save As Dashboard Panel window, choose the following options:
    • Dashboard: Existing
    • Dropdown: My First Dashboard
    • Panel Title: Backups Complete by Percent
    • Panel Content: Pie
    • Splunk Enterprise Save as Dashboard panel
  5. Click Save
  6. Splunk Enterprise displays Your Dashboard Panel Has Been Created:Splunk Enterprise your Dashboard has been created

Congratulations! You have added another panel to your dashboard. Click View Dashboard to see your new panel:

Splunk Enterprise My First Dashboard with two panels

Rearranging panels on a dashboard

You can easily move panels around on your dashboards, to customize the look and feel.

Before you begin

This section of the tutorial assumes that you have already created a dashboard with at least two panels, as described in the sections above.

Steps

Step 1: Edit panels

  1. Click the Edit icon
    • Splunk Enterprise Edit panels
  2. Choose Edit Panels
    • Splunk Enterprise edit panels

Step 2: Move panel

  1. Click and hold the top bar of the panel you want to move, then drag the panel to the new location:
    • Splunk Enterprise move panels
  2. Release mouse pointer while panel is over area you wish to drop the panel into:
    • Splunk Enterprise panel moved

Step 3: Save new arrangement

Click Done at upper right of dashboard

Edit a panel to change the type of chart

You can easily modify the type of chart displayed in one of your panels.

Before you begin

This section of the tutorial assumes that you have created at least one dashboard with at least one panel that displays a chart.

Steps

Step 1: Edit panels

  1. Click the Edit icon
    • Splunk Enterprise Edit panels
  2. Choose Edit Panels
    • Splunk Enterprise edit panels

Step 2: Choose chart type

  1. Click the chart icon on the panel you want to change (the icon reflects the current chart type): Splunk Enterprise chart icon
  2. Choose chart type:
    • Splunk Enterprise choose new chart type 2
  3. The new chart type will be immediately applied:
    • Splunk Enterprise new chart chosen

External resources

This is just the beginning. Explore more features of Splunk that enable you to leverage the data provided by the Code42 environment.

What's next?

Click Next below to learn about using Code42 environment logs as data sources for Splunk.