With the creation of the new Tegile IntelliFlash App for Splunk, today we announced a partnership to run Splunk on Tegile.
Through this partnership, Splunk users can now capture insights from machine-generated data by combining Splunk’s platform with Tegile’s unique storage infrastructure that includes multiple grades of storage media in one storage appliance.
Get the Tegile IntelliFlash App for Splunk
Capitalize on Big Data & the Internet of Things (IoT)
IT pros are aware that machine data is one of the fastest growing and most complex areas of Big Data; with the outlook of the Internet of Things (IoT), it’s also one of the most valuable.
It’s one thing to store data created by all of these machines, which Tegile has done well for years. It’s another thing, with Splunk on Tegile, to use the data we store to provide users with actionable events to simplify their storage management with Splunk.
Get the Splunk Reference Architecture from Tegile
Predictive Analytics from the Tegile IntelliFlash App for Splunk
The Tegile IntelliFlash App for Splunk helps our customers monetize their data in new ways. With this app, Tegile allows organizations to store, access and analyze all of their machine data. As storage pros know, real-time analytics have a 24/7 appetite for storage. See below top use cases.
How Splunk on Tegile
Optimizes Storage Management
Top 3 Use Cases
1) Predictive analytics for IT operations
Making sense of syslog or NOC data is especially helpful for CIOs looking to implement predictive analytics for IT operations to help create future actionable events with the data being stored. Essentially this taps into the power of already stored data.
2) Security information and event management
Centralizing and reporting on security-oriented data provides a holistic view of an organization’s IT security. This holistic view can help deter cyber security threats before they ever occur.
3) IoT and real-time business analytics
Monetizing raw data generated by customer machine data helps identify new business models and paths to revenue.
Splunk on Tegile Solution Offering
First off, Splunk is a data processing platform engine used to classify and query machine-generated data (MGD). Splunk places data into a repository that can be queried by administrators and uses a “pooling” architecture to place data in tiered buckets: hot data, warm data, and cold data. Tegile uses a very complementary architecture.
Tegile is Purpose Built for Splunk
<img src=”http://www.tegile.com/wp-content/uploads/2016/09/tegile-array-splunk.png” alt=”tegile-array-splunk” width=”598″ height=”513″ class=”aligncenter size-full wp-image-20738″ />
Tegile automatically places
hot, warm & cold data from Splunk buckets
into the appropriate storage layer on the array
Splunk on Tegile
During hot to warm/cold data bucket movement, Tegile reduces the overall workload for Splunk, array and network. In contrast, a number of different issues can occur with other solutions that don’t have this specific functionality.
The “Alternatives”
Other vendors have offerings for Splunk, but they are not as streamlined as Tegile storage. For some vendors, Splunk hot buckets are segregated into small, fast volumes and warm/cold buckets are segregated into large slow volumes (see diagram at right).
Second, during hot to warm/cold data movement, Splunk must read the entire bucket from the fast volume, then write all that data to the warm/cold bucket. This becomes a big issue for enterprise environments where an admin would have to manage hundreds of volumes separately.
Or, as some storage vendors require silos of storage to achieve functionality similar to that of Tegile, separate storage array appliances are required to provide both performance and capacity while traversing a network, which can induce additional latency during queries.
What’s the downside of storage alternatives for Splunk?
The above alternate solutions have these downsides:
Increased overhead for Splunk
Increased overhead for the arrays themselves
Increased network latency
Increased complexity of deployment and management
The below diagram is another illustration of the increased complexity of storage alternatives for Splunk.
Splunk on alternative storage
Note how data is segregated across multiple arrays,
which increases complexity and latency.
Benefits of “Real-Time Caching” with Tegile
Tegile reduces query times on Splunk by providing flash-optimized storage and metadata. Because Tegile arrays automatically place hot, warm and cold data in the same array, filesystem and volume, the movement of data becomes a simple metadata update within the array.
Benefits of “Real-Time Caching” with Tegile
Flash-optimized separation of metadata and block data
Improved performance with no data segregation into “tiers”
Combines flash and disk in one array
Presents volumes to Splunk indexers in a simple way
Concurrent protocol connectivity
Increased deployment flexibility
Dual-active pooled storage infrastructure
Can flexibly scale performance and capacity – non-disruptively
Inline data reduction technology improves economics and cost savings
Tegile’s patented metadata acceleration process isolates and aggregates metadata on flash, which reduces query time significantly. Also, upon data ingest, Tegile compresses the already compressed Splunk data, which results in improved data reduction ratios and cost savings.
Even though Splunk will compress raw data, Tegile is able to achieve additional data reduction savings (via inline compression and deduplication) on Splunk’s index files. When Splunk compresses raw data and stores it, it also creates time series index files (tsidx) that associate terms in the raw data with the offset (in the raw file) of the occurrence of that term.
Tegile facilitates further reduction of those tsidx files beyond that which Splunk achieves on its own. This is especially true in a indexer cluster, where data can repeat itself. In our own testing, an additional 40% data reduction was achieved.
Existing Splunk Users
If you’re an existing Splunk user, you may wish to take next steps. You might ask yourself these questions:
How do I integrate storage into my Splunk UI for embedded insights?
How do I upgrade my existing slow legacy, disk-based storage for Splunk?
How do I simplify and reduce the cost and risk of my existing Splunk architecture?
If you are ready for next steps, why not download the reference architecture and see how the Splunk on Tegile solution can help.
Existing Tegile Customers
If you’re an existing Tegile customer and you want to use Splunk or use it already, we recommend trying our new Tegile IntelliFlash App for Splunk. As mentioned above, this app provides a single-pane-glass plugin that collects pertinent information from Tegile arrays and displays it directly into the Splunk user interface.
Screenshot of the Tegile IntelliFlash App for Splunk
The type of data retrieved includes array controllers, pools, volumes and mappings to assist administrators in monitoring and management. This data allows more granular management and monitoring capabilities when storing Splunk data on Tegile. The installation is very easy and can be completed in about 5 minutes.
Future Steps
The sky’s the limit for Tegile users with Splunk. Stay tuned to this Tegile blog and chat with us on Twitter, LinkedIn and Facebook, where you’ll hear news about Splunk on Tegile first.