Splunk SmartStore with Nutanix Objects

February 23, 2020

Below you’ll find a quick walkthrough on how to configure Splunk SmartStore with Nutanix Objects. Now that Nutanix is Splunk SmartStore certified, I thought it would be a great time to introduce you to the power of running Splunk SmartStore with Nutanix Objects and how easy it is to set up.

Nutanix Objects is an S3-compatible object storage solution that leverages the underlying Nutanix storage fabric which allows it to benefit from features such as encryption, compression, and erasure coding (EC-X).

Objects allows users to store petabytes of unstructured data on the Nutanix platform, with support for features such as WORM (write once, read many) and object versioning that are required for regulatory compliance, and easy integration with 3rd party backup software and S3-compatible applications.

Given Nutanix Objects scalability and attractive pricing, it’s the perfect partner for Splunk deployments.

Assumptions

Create Nutanix Objects AMI User Keys

  1. In Prism Central select ☰ > Services > Objects

    Splunk Objects 1

  2. Click on Access Keys > Add People > Add People not in a directory service.

    Enter in an email address that is unique (it does not need to be able to receive email).

    Splunk Objects 2

  3. Click on Download Keys. Depending on your broser, it will either open a new tab or download a text file.

    Note: It is important you save the Access Key and Secret Access Key as it will only be shown once.

    Splunk Objects 3

    Splunk Objects 4

Create Bucket Using AMI User

Since Object Storage uses API keys to grant access to various buckets, we’ll want to create a bucket using the API key we just created above.

A bucket is a sub-repository within an object store which can have policies applied to it, such as versioning, WORM, etc. By default a newly created bucket is a private resource to the creator. The creator of the bucket by default has read/write permissions, and can grant permissions to other users.

We will use Cyberduck to create and use buckets in the object store using your generated access key.

  1. Launch Cyberduck

  2. Click on Open Connection

    Splunk Objects 5

  3. Select Amazon S3 from the dropdown list

  4. Enter the following fields for the user created earlier, then click Connect.

    • Server - Objects Client IP
    • Port - 443
    • Access Key ID - Generated when User Created
    • Password (Secret Key) - Generated when User Created

    Splunk Objects 6

  5. Click Continue on the The Certificate is not valid dialog box.

    Splunk Objects 7

  6. Right Click and choose New Folder.

    Splunk Objects 8

  7. Enter in a name for your bucket, and click Create:

    Note: Bucket names must be lower case and only contain letters, numbers, periods and hyphens.

    Splunk Objects 9

    If you check in the Objects console, you’ll see that a new bucket has been created.

Configure SmartStore

  1. Gather the required information:

    • MYOBJECTSACCESSKEY: You should have this from the AMI Key Section above
    • MYOBJECTSSECRETKEY: You should have this from the AMI Key Section above
    • OBJECTSCLIENTIP: You can get this from ☰ > Services > Objects

    Splunk Objects 10

  2. SSH into your Splunk Indexer

  3. Edit /opt/splunk/etc/system/local/indexes.conf replacing the ALL CAPS areas with the info gathered above.

     [default]
     remotePath = volume:remote_store/$_index_name
    
     [volume:remote_store]
     storageType = remote
     path = s3://MYAWESOMEBUCKETHERE/
     remote.s3.access_key = MYOBJECTSACCESSKEY
     remote.s3.secret_key = MYOBJECTSSECRETKEY
     remote.s3.endpoint = https://OBJECTSCLIENTIP
     remote.s3.auth_region = us-east-1
    
  4. Restart the Splunk Indexer

     /opt/spluk/bin/splunk restart
    

View Data in Objects

After a little bit of time, you should be able to head over to Objects in PC and see that your bucket is being populated with data. The time it takes to tier from your Hot index to Objects is based on your hot data retention settings for your Indexes.

You can see performance information specific to your bucket in the Objects menu in Prism Central.

Splunk Objects 11