Analyze data with Splunk and the Code42 API

Code42 API

The Code42 API is a powerful HTTP-based API that you can use to generate reports, acquire information, perform automated actions, and integrate the Code42 environment seamlessly into your overall environment. You should have a basic familiarity with the Code42 API. Two good places to start are:

• Our dedicated Code42 API documentation
• The built-in API Documentation Viewer, available in two ways:
  • Visit the following URL on any Code42 server:
    https://master-server.example.com:4285/apidocviewer
  • Visit our public API Documentation Viewer

This tutorial focuses on the integration of the Code42 API with Splunk and assumes that you have administrative access to at least one server running the Code42 environment. Keep in mind, however, the following:

• Splunk can also search and index Code42 server log files, as well as the system logs of your servers.
• You can install a data forwarder, if necessary, to send data from log files to your Splunk Enterprise server.
• The Code42 API as a source of data has advantages over unstructured log files, which is why this tutorial leverages the API. Some of these advantages:
  • more focused data, based on parameters passed to the API
  • less bandwidth utilization
  • faster results

Splunk

Splunk Enterprise is a commercial product that provides tools to store, index, search, monitor, analyze and visualize machine-generated data. This tutorial uses Splunk version 6. Although Splunk requires a license, you are able to use a 30-day free trial to initially install and configure Splunk Enterprise.

Running Splunk Enterprise on a dedicated server is recommended, as Splunk's system usage will not affect the performance of your Code42 environment. Consider these two options before beginning:

• Install Splunk Enterprise on one of your Code42 servers.
  • No need to build a new server.
  • Splunk Enterprise will have access to server logs without needing forwarders installed.
• Install Splunk Enterprise on a dedicated server, either on a VM or dedicated hardware
  • Avoids any impact on performance or configuration of your Code42 server.
  • Splunk Enterprise will need additional configuration to connect to your Code42 server.

For details on supported operating systems, see Splunk Enterprise's:

• Operating system requirements
• Recommended and minimum hardware requirements

Steps

This tutorial will cover installing Splunk Enterprise on a Linux server using the .deb (Debian package) installation file type (Ubuntu uses this method). For full installation instructions for all platforms, see Splunk's installation manual.

Step 2: Run dpkg on the installer file

As root, or using the sudo command, run the following command in a terminal window or shell, while in the directory containing the downloaded Splunk installation file:
dpkg -i splunk_package_name.deb

Splunk will be installed in the directory /opt/splunk by default.

You may want to add the Splunk installation directory to your environment's PATH variable. To make the changes to your PATH variable permanent, add these export commands to your .bashrc file:

Copy to clipboardCopied!

# export SPLUNK_HOME=/opt/splunk
# export PATH=$SPLUNK_HOME/bin:$PATH

root@myServer:~# which splunk
/opt/splunk/bin/splunk

Before you begin

• This part of the tutorial guides you in the creation of four shell scripts that access the Code42 API. Create them on the file system of the Splunk Enterprise server.
• These steps requires the use of a text editor that does not add extra characters or text to the files such as vim or Notepad.
• Root access to the server running Splunk Enterprise is required.

Step 1: Sign in to Splunk server with SSH

1. Use ssh or another secure method to sign in to your Splunk Enterprise server.
2. Switch to root or another user with the necessary privileges.
3. Use the cd command to change your working directory to the Splunk bin/scripts directory:
   Copy to clipboardCopied!

   root@splunkServer:~# cd /opt/splunk/bin/scripts/
   root@splunkServer:/opt/splunk/bin/scripts#
   

Step 2: Create the user.sh script

Using your favorite text editor (e.g. vim, Emacs, pico, etc.), create the user.sh script:

1. Open the text editor.
2. Copy the text below into the editor:
   Copy to clipboardCopied!

   /usr/bin/curl -ku 'username' "https://<your_proe_server>:4285/console/api/User?srtKey=name&srtDir=asc&incAlertCounts=true&incBackupUsage=true&incRoles=true&incComputerCount=true&targetComputerGuid=rollup&active=true&alerted=false&invited=false&admins=false&export=true"

   Replace username with your username and replace <your_proe_server> with your Code42 server's IP address or domain name. You will enter the password corresponding to the username when prompted after running the request.
3. Save the user.sh script then quit the editor.

Step 3: Create the computer.sh script

1. Open the text editor.
2. Copy the text below into the editor:
   Copy to clipboardCopied!

   /usr/bin/curl -ku 'username' "https://<your_proe_server>:4285/console/api/Computer?srtKey=name&srtDir=asc&targetComputerGuid=rollup&incBackupUsage=true&incActivity=true&incCounts=true&active=true&alerted=false&export=csv"
   

   Replace username with your username and replace <your_proe_server> with your Code42 server's IP address or domain name.
3. Save the computer.sh script then quit the editor.

Step 4: Create the organization.sh script

1. Open the text editor.
2. Copy the text below into the editor:
   Copy to clipboardCopied!

   /usr/bin/curl -ku 'username' https://<your_proe_server>:4285/api/org?export=1
   

   Replace username with your username and replace <your_proe_server> with your Code42 server's IP address or domain name.
3. Save the organization.sh script then quit the editor.

Step 5: Create the destinations.sh script

1. Open the text editor.
2. Copy the text below into the editor:
   Copy to clipboardCopied!

   /usr/bin/curl -ku 'username' https://<your_proe_server>:4285/api/destination?export=1

   Replace username with your username and replace <your_proe_server> with your Code42 server's IP address or domain name.
3. Save the destination.sh script then quit the editor.

Before you begin

It is a good idea to get a basic familiarity with Splunk by viewing some of their introductory tutorials.

This section requires:

• A working Splunk Enterprise installation.
• A working Code42 environment of at least one authority server. It is preferable to use a test platform or server, to avoid affecting the performance of your production environment.

Splunk is able to utilize many sources of data, such as log files, CSV files, scripts, API calls, and others.This tutorial focuses on using our API. See Splunk's documentation for the complete list of supported data sources.

Step 1: Sign in to Splunk Enterprise and navigate to Splunk Home

1. Navigate to your Splunk Enterprise server's URL in a web browser.
2. Enter your username and password.
3. You will be taken to Splunk Home. If you ever need to return to Splunk Home, simply click the Splunk icon at the upper left of the Splunk Enterprise console.
   

Step 2: Go to add data

In the Data panel at upper right, click Add Data.

The Add data window displays, which lets you add data from various sources and types.

Step 3: Add the user.sh script as a data source

1. Click Run and collect the output of a script, located under Or Choose a Data Source.
2. Fill in the following fields then click Save.

   

   Field	Value to enter	Description
   Command	/opt/splunk/bin/scripts/user.sh	Path to the user.sh script.
   Interval	Leave at default of 60.0 seconds	How often script is run and data collected.
   Source name override	Users	Changes the name of the data source to an easier to read form.
   Select source type from list	csv	Tells Splunk the format of the data.

   If the user.sh script was successfully added, you will see the following:

   

Step 4: Add the remaining scripts

Click Add more data, then repeat Step 3 above for the remaining scripts. Enter the following values into the Source name override field:

• For computer.sh enter: Computers
• For destination.sh enter: Destinations
• For organization.sh enter: Organizations

Before you begin

In this section, you will create your first dashboard, and add a panel to the dashboard that visualizes the devices in your Code42 environment by their their operating system and CPU type.

You can create separate dashboards or panels for separate functions. For example, you could have separate dashboards to provide detailed looks at:

• Users
• Security-related data and audits
• Capacity planning and usage
• Server update and reliability
• Logs

Step 1: Create the search

1. From Splunk Home, click Search from Search & Reporting.
   
2. Enter the following search term into the Search field.
   Copy to clipboardCopied!

   host=YourSplunkServer | dedup guid | top limit=20 osArch

   Replace YourSplunkServer with the name of your Splunk Enterprise server.
3. Press Enter or click the search icon:
   Splunk Enterprise returns the results.

Step 2: Explore the results

The results page contains three tabs:

• Events
  • Displays the actual events that were indexed and stored by Splunk Enterprise.
• Statistics
  • Displays the results of any statistical or reporting command in your search.
  • Results displayed as a table.
• Visualization
  • Displays statistical results as a chart visualization.
  • Different chart types can be selected for visualization.

Click on the tabs to familiarize yourself with the information they display.

Step 3: Save visualization results as a panel in a new dashboard

You will now save the visualization as a pie chart, to a new dashboard.

1. Click Visualization.
2. Click Save As at the upper right of the page.
   
3. Choose Dashboard Panel.
4. From the Save As Dashboard Panel, choose the following required options:
   • Dashboard: New
   • Dashboard Permissions: Shared in App
   • Panel Content: Pie
5. Enter text of your choosing for the following options:
   Suggestions for these options are provided below.
   • Dashboard Title: My First Dashboard
   • Dashboard ID: My_first_dashboard
   • Dashboard Description:Visualization of various {{environment}} API data
   • Panel Title: Devices by Architecture
6. Click Save.
   Splunk Enterprise displays Your Dashboard Panel Has Been Created.

Congratulations! You have created your first Splunk Enterprise Dashboard and Panel, and visualized some real data!

Step 4: View dashboard and explore data

1. Click View Dashboard.
2. Hover over the pie chart in the Devices by Architecture window.
   Hovering over segments of the chart displays data related to that chart segment.
   
3. Click one of the pie chart's segments.
   A new search displays specific data contained in that pie chart segment.
4. The new search can be saved as a panel or report, by clicking Save As at the upper right of the search window.
   

Creating your first query shows the power of Splunk. It allowed you to:

• Visualize the query data.
• Create a dashboard and panel to save your search in an easily viewable, graphical form.
• Narrow your focus to more specific data with a simple click.

Let's create some more queries and add more panels to your first dashboard.

Before you begin

This step assumes that you have already created your first dashboard, as described above.

Step 1: Create the search

1. From Splunk Home, click Search from Search & Reporting.
   
2. Enter the following search term into the Search field.
   Copy to clipboardCopied!

   host=YourSplunkServer | dedup guid | top limit=20 osName

   Replace YourSplunkServer with the name of your Splunk Enterprise server.
3. Press Enter or click the search icon:
   Splunk Enterprise returns the results.

Step 2: Save visualization results as a panel in existing dashboard

You will now save the visualization as a pie chart, to a new panel in your existing dashboard.

1. Click Visualization to display the pie chart.
   
2. Click Save As at the upper right of the page.
   
3. Choose Dashboard Panel.
4. From the Save As Dashboard Panel choose the following options and enter a panel title of your choice:
   • Dashboard: Existing
   • Dropdown: My First Dashboard
   • Panel Content: Pie
     
5. Click Save.
   Splunk Enterprise displays Your Dashboard Panel Has Been Created.

Click View Dashboard to see your dashboard and two panels.

Before you begin

This section assumes you have already created your first dashboard, as described in Create Your First Dashboard And Visualize Devices By Architecture.

Step 1: Create the search

1. From Splunk Home, click Search from Search & Reporting.
   
2. Enter the following search term into the Search field.
   Copy to clipboardCopied!

   host=YourSplunkServer | dedup guid | eval percentComplete = round(percentComplete,0) | top percentComplete

   Replace YourSplunkServer with the name of your Splunk Enterprise server.
3. Press Enter or click the search icon:
   Splunk Enterprise returns the results.

Step 2: Save visualization results as a panel in existing dashboard

You will now save the visualization as a pie chart, to a new panel in your existing dashboard.

1. Click Visualization to display the pie chart.
   
2. Click Save As at the upper right of the page.
   
3. Choose Dashboard Panel.
4. From the Save As Dashboard Panel choose the following options:
   • Dashboard: Existing
   • Dropdown: My First Dashboard
   • Panel Title: Backups Complete by Percent
   • Panel Content: Pie
     
5. Click Save.
   Splunk Enterprise displays Your Dashboard Panel Has Been Created.

Click View Dashboard to see your new panel.

Before you begin

This section of the tutorial assumes that you have already created a dashboard with at least two panels, as described in the sections above.

Step 1: Edit panels

1. Click the Edit.
   
2. Choose Edit Panels.
   

Step 2: Move panel

1. Click and hold the top bar of the panel you want to move, then drag the panel to a new location.
   
2. Release mouse pointer while panel is over the area you want the panel.
   
3. Click Done at upper right of dashboard.

Before you begin

This section of the tutorial assumes that you have created a dashboard with at least one panel that displays a chart.

Step 1: Edit panels

1. Click the Edit.
   
2. Choose Edit Panels.