Analyze data with Splunk and the Code42 API
Who is this article for?
Incydr, no.
CrashPlan for Enterprise, yes.
Code42 for Enterprise, yes.
CrashPlan for Small Business, no.
Overview
This tutorial explains how to install, configure and test Splunk Enterprise, and then integrate it with the Code42 API. Splunk Enterprise is a solution for data analytics, monitoring, and visualization. The use of Splunk Enterprise provides enterprises with customized solutions to leverage the data available through the Code42 platform.
For further examples on using your Code42 environment data with Splunk, see API Script Recipes For Use With Splunk.
Considerations
Code42 API
The Code42 API is a powerful HTTP-based API that you can use to generate reports, acquire information, perform automated actions, and integrate the Code42 environment seamlessly into your overall environment. You should have a basic familiarity with the Code42 API. Two good places to start are:
- Our dedicated Code42 API documentation
- The built-in API Documentation Viewer, available in two ways:
- Visit the following URL on any Code42 server:
https://master-server.example.com:4285/apidocviewer - Visit our public API Documentation Viewer
- Visit the following URL on any Code42 server:
This tutorial focuses on the integration of the Code42 API with Splunk and assumes that you have administrative access to at least one server running the Code42 environment. Keep in mind, however, the following:
- Splunk can also search and index Code42 server log files, as well as the system logs of your servers.
- You can install a data forwarder, if necessary, to send data from log files to your Splunk Enterprise server.
- The Code42 API as a source of data has advantages over unstructured log files, which is why this tutorial leverages the API. Some of these advantages:
- more focused data, based on parameters passed to the API
- less bandwidth utilization
- faster results
Splunk
The information that pertains to Splunk Enterprise is provided for your convenience. Our Customer Champions are not able to assist with Splunk-specific steps or issues.
Splunk Enterprise is a commercial product that provides tools to store, index, search, monitor, analyze and visualize machine-generated data. This tutorial uses Splunk version 6. Although Splunk requires a license, you are able to use a 30-day free trial to initially install and configure Splunk Enterprise.
Running Splunk Enterprise on a dedicated server is recommended, as Splunk's system usage will not affect the performance of your Code42 environment. Consider these two options before beginning:
- Install Splunk Enterprise on one of your Code42 servers.
- No need to build a new server.
- Splunk Enterprise will have access to server logs without needing forwarders installed.
- Install Splunk Enterprise on a dedicated server, either on a VM or dedicated hardware
- Avoids any impact on performance or configuration of your Code42 server.
- Splunk Enterprise will need additional configuration to connect to your Code42 server.
For details on supported operating systems, see Splunk Enterprise's:
Install Splunk Enterprise
Steps
This tutorial will cover installing Splunk Enterprise on a Linux server using the .deb (Debian package) installation file type (Ubuntu uses this method). For full installation instructions for all platforms, see Splunk's installation manual.
Step 1: Download installer
Download the installer for your Linux server from Splunk's download site.
Step 2: Run dpkg on the installer file
As root, or using the sudo
command, run the following command in a terminal window or shell, while in the directory containing the downloaded Splunk installation file:
dpkg -i splunk_package_name.deb
Splunk will be installed in the directory /opt/splunk by default.
You may want to add the Splunk installation directory to your environment's PATH variable. To make the changes to your PATH variable permanent, add these export commands to your .bashrc file:
# export SPLUNK_HOME=/opt/splunk # export PATH=$SPLUNK_HOME/bin:$PATH
Step 3: Start Splunk
From the Splunk bin directory, enter the following command:
./splunk start
If you have added the Splunk installation directory to your environmental PATH variable, you can always find the location of the Splunk executable by entering the following command (command shown with results):
root@myServer:~# which splunk /opt/splunk/bin/splunk
Step 4: Accept license agreement
The first time you run Splunk, you must accept the license agreement. To start Splunk and accept the agreement in one step, you may simply enter the following command to run the application for the first time:
$SPLUNK_HOME/bin/splunk start --accept-license
Step 5: Sign in to Splunk web
- Open a supported web browser
- Enter the following URL:
http://<hostname>:<port>
- <hostname>: the IP address or hostname of your Splunk Enterprise server.
- <port>: the port number you chose during installation.
- Enter the default username admin and the default password changeme the first time you sign in.
Splunk recommends changing the default password immediately.
Step 6: Learn about basic functionality from Splunk's documentation
Splunk is a feature-rich and customizable application. You may want to read about some of Splunk's features, and how they work, before proceeding with the next steps in this tutorial, to gain a basic understanding of the product.
Create example API scripts on Splunk server
Before you begin
- This part of the tutorial guides you in the creation of four shell scripts that access the Code42 API. Create them on the file system of the Splunk Enterprise server.
- These steps requires the use of a text editor that does not add extra characters or text to the files such as vim or Notepad.
- Root access to the server running Splunk Enterprise is required.
These scripts were written as demonstration scripts, to show you how to integrate your own scripts with Splunk Enterprise. However, they provide real, useful data. They are provided without warranty or support.
Step 1: Sign in to Splunk server with SSH
- Use ssh or another secure method to sign in to your Splunk Enterprise server.
- Switch to root or another user with the necessary privileges.
- Use the
cd
command to change your working directory to the Splunk bin/scripts directory:Copied!root@splunkServer:~# cd /opt/splunk/bin/scripts/ root@splunkServer:/opt/splunk/bin/scripts#
Step 2: Create the user.sh script
Using your favorite text editor (e.g. vim, Emacs, pico, etc.), create the user.sh script:
- Open the text editor.
- Copy the text below into the editor:
Copied!
/usr/bin/curl -ku 'username' "https://<your_proe_server>:4285/console/api/User?srtKey=name&srtDir=asc&incAlertCounts=true&incBackupUsage=true&incRoles=true&incComputerCount=true&targetComputerGuid=rollup&active=true&alerted=false&invited=false&admins=false&export=true"
- Save the user.sh script then quit the editor.
Step 3: Create the computer.sh script
- Open the text editor.
- Copy the text below into the editor:
Copied!
/usr/bin/curl -ku 'username' "https://<your_proe_server>:4285/console/api/Computer?srtKey=name&srtDir=asc&targetComputerGuid=rollup&incBackupUsage=true&incActivity=true&incCounts=true&active=true&alerted=false&export=csv"
- Save the computer.sh script then quit the editor.
Step 4: Create the organization.sh script
- Open the text editor.
- Copy the text below into the editor:
Copied!
/usr/bin/curl -ku 'username' https://<your_proe_server>:4285/api/org?export=1
- Save the organization.sh script then quit the editor.
Step 5: Create the destinations.sh script
- Open the text editor.
- Copy the text below into the editor:
Copied!
/usr/bin/curl -ku 'username' https://<your_proe_server>:4285/api/destination?export=1
- Save the destination.sh script then quit the editor.
Add sample scripts as data sources
Before you begin
It is a good idea to get a basic familiarity with Splunk by viewing some of their introductory tutorials.
This section requires:
- A working Splunk Enterprise installation.
- A working Code42 environment of at least one authority server. It is preferable to use a test platform or server, to avoid affecting the performance of your production environment.
Splunk is able to utilize many sources of data, such as log files, CSV files, scripts, API calls, and others.This tutorial focuses on using our API. See Splunk's documentation for the complete list of supported data sources.
Step 1: Sign in to Splunk Enterprise and navigate to Splunk Home
- Navigate to your Splunk Enterprise server's URL in a web browser.
- Enter your username and password.
- You will be taken to Splunk Home. If you ever need to return to Splunk Home, simply click the Splunk icon at the upper left of the Splunk Enterprise console.
Step 2: Go to add data
In the Data panel at upper right, click Add Data.
The Add data window displays, which lets you add data from various sources and types.
Step 3: Add the user.sh script as a data source
- Click Run and collect the output of a script, located under Or Choose a Data Source.
- Fill in the following fields then click Save.
Field Value to enter Description Command /opt/splunk/bin/scripts/user.sh Path to the user.sh script. Interval Leave at default of 60.0 seconds How often script is run and data collected. Source name override Users Changes the name of the data source to an easier to read form. Select source type from list csv Tells Splunk the format of the data. If the user.sh script was successfully added, you will see the following:
Step 4: Add the remaining scripts
Click Add more data, then repeat Step 3 above for the remaining scripts. Enter the following values into the Source name override field:
- For computer.sh enter:
Computers
- For destination.sh enter:
Destinations
- For organization.sh enter:
Organizations
Create your first dashboard and visualize devices by architecture
Before you begin
In this section, you will create your first dashboard, and add a panel to the dashboard that visualizes the devices in your Code42 environment by their their operating system and CPU type.
You can add any number of queries, reports, and visualizations to your Splunk Enterprise server installation.
You can create separate dashboards or panels for separate functions. For example, you could have separate dashboards to provide detailed looks at:
- Users
- Security-related data and audits
- Capacity planning and usage
- Server update and reliability
- Logs
Step 1: Create the search
- From Splunk Home, click Search from Search & Reporting.
- Enter the following search term into the Search field.
Copied!
host=YourSplunkServer | dedup guid | top limit=20 osArch
- Press Enter or click the search icon:
Splunk Enterprise returns the results.
Step 2: Explore the results
The results page contains three tabs:
- Events
- Displays the actual events that were indexed and stored by Splunk Enterprise.
- Statistics
- Displays the results of any statistical or reporting command in your search.
- Results displayed as a table.
- Visualization
- Displays statistical results as a chart visualization.
- Different chart types can be selected for visualization.
Click on the tabs to familiarize yourself with the information they display.
Step 3: Save visualization results as a panel in a new dashboard
You will now save the visualization as a pie chart, to a new dashboard.
- Click Visualization.
- Click Save As at the upper right of the page.
- Choose Dashboard Panel.
- From the Save As Dashboard Panel, choose the following required options:
- Dashboard: New
- Dashboard Permissions: Shared in App
- Panel Content: Pie
- Enter text of your choosing for the following options:
Suggestions for these options are provided below.- Dashboard Title:
My First Dashboard
- Dashboard ID:
My_first_dashboard
- Dashboard Description:
Visualization of various {{environment}} API data
- Panel Title:
Devices by Architecture
- Dashboard Title:
- Click Save.
Splunk Enterprise displays Your Dashboard Panel Has Been Created.
Congratulations! You have created your first Splunk Enterprise Dashboard and Panel, and visualized some real data!
Step 4: View dashboard and explore data
- Click View Dashboard.
- Hover over the pie chart in the Devices by Architecture window.
Hovering over segments of the chart displays data related to that chart segment.
- Click one of the pie chart's segments.
A new search displays specific data contained in that pie chart segment. - The new search can be saved as a panel or report, by clicking Save As at the upper right of the search window.
Creating your first query shows the power of Splunk. It allowed you to:
- Visualize the query data.
- Create a dashboard and panel to save your search in an easily viewable, graphical form.
- Narrow your focus to more specific data with a simple click.
Let's create some more queries and add more panels to your first dashboard.
Create new panel to visualize devices by os
This section of the tutorial guides you in creating an additional panel in your Splunk Enterprise dashboard. The panel displays a chart that communicates the distribution of the endpoint devices in your Code42 environment based on operating system.
Before you begin
This step assumes that you have already created your first dashboard, as described above.
Step 1: Create the search
- From Splunk Home, click Search from Search & Reporting.
- Enter the following search term into the Search field.
Copied!
host=YourSplunkServer | dedup guid | top limit=20 osName
- Press Enter or click the search icon:
Splunk Enterprise returns the results.
Step 2: Save visualization results as a panel in existing dashboard
You will now save the visualization as a pie chart, to a new panel in your existing dashboard.
- Click Visualization to display the pie chart.
- Click Save As at the upper right of the page.
- Choose Dashboard Panel.
- From the Save As Dashboard Panel choose the following options and enter a panel title of your choice:
- Dashboard: Existing
- Dropdown: My First Dashboard
- Panel Content: Pie
- Click Save.
Splunk Enterprise displays Your Dashboard Panel Has Been Created.
Click View Dashboard to see your dashboard and two panels.
Visualize backup completion
This section of the tutorial guides you in the creation of another panel in your dashboard. The new panel displays a chart that displays the range of backup completion rates for your endpoint devices.
Before you begin
This section assumes you have already created your first dashboard, as described in Create Your First Dashboard And Visualize Devices By Architecture.
Step 1: Create the search
- From Splunk Home, click Search from Search & Reporting.
- Enter the following search term into the Search field.
Copied!
host=YourSplunkServer | dedup guid | eval percentComplete = round(percentComplete,0) | top percentComplete
- Press Enter or click the search icon:
Splunk Enterprise returns the results.
Step 2: Save visualization results as a panel in existing dashboard
You will now save the visualization as a pie chart, to a new panel in your existing dashboard.
- Click Visualization to display the pie chart.
- Click Save As at the upper right of the page.
- Choose Dashboard Panel.
- From the Save As Dashboard Panel choose the following options:
- Dashboard: Existing
- Dropdown: My First Dashboard
- Panel Title: Backups Complete by Percent
- Panel Content: Pie
- Click Save.
Splunk Enterprise displays Your Dashboard Panel Has Been Created.
Click View Dashboard to see your new panel.
Rearrange panels on a dashboard
You can easily move panels around on your dashboards, to customize the look and feel.
Before you begin
This section of the tutorial assumes that you have already created a dashboard with at least two panels, as described in the sections above.
Step 1: Edit panels
- Click the Edit.
- Choose Edit Panels.
Step 2: Move panel
- Click and hold the top bar of the panel you want to move, then drag the panel to a new location.
- Release mouse pointer while panel is over the area you want the panel.
- Click Done at upper right of dashboard.
Edit a panel to change the type of chart
You can easily modify the type of chart displayed in one of your panels.
Before you begin
This section of the tutorial assumes that you have created a dashboard with at least one panel that displays a chart.
Step 1: Edit panels
- Click the Edit.
- Choose Edit Panels.
Step 2: Choose chart type
- Click the chart icon on the panel you want to change:
The icon reflects the current chart type. - Choose chart type.
The new chart type is immediately applied.
External resources
Explore more features of Splunk that enable you to leverage the data provided by the Code42 environment.
- Splunk Enterprise Installation Manual
- Splunk Enterprise Search Tutorial
- Splunk Enterprise Getting Data In Tutorial
What's next?
Learn about using Code42 environment logs as data sources for Splunk.