Non-volatile storage system with in-drive data analytics

#A solid state drive (“SSD”) is a non-volatile storage device that stores persistent data in solid state based memory. Due to improvements in cost, power and speed, SSD's have become more popular for storing enterprise data or other large sets of data that are often referred to as “Big Data.”

#Big Data analytics involves running analytical algorithms on large sized unstructured data (on the order of gigabytes, terabytes or higher) to derive information and patterns that help make decisions pertaining to business, security, human behavior, economics, social trends, etc. The data can be super-sized files including text data, video data, voice data, numbers, etc. A typical data analytics program flow involves an analytics application reading a set of files from a hard disk drive (“HDD”) or SSD into a host machine's (e.g., an analytics server) memory, running a data analytics algorithm on the ‘in memory’ data, and utilizing the results for decision making. This approach suffers from the bottleneck of reading huge data sets from storage (e.g., HDD or SSD) into host memory over a storage network (“SAN”). Additionally, the above-described approach for Big Data analytics can be expensive both in terms of host system memory requirements and network usage.

#Like-numbered elements refer to common components in the different figures.

#FIG. 1 is a block diagram of a system having a solid state drive.

#FIG. 2 is a block diagram of a system having using a storage network to access solid state drives.

#In order to reduce the delay in Big Data analytics due to reading the Big Data from the HDD or SDD into the analytics server memory, it is proposed to perform the analytics in-drive, meaning that the analysis of the data is performed inside an SSD. There are several advantages to performing in-drive analytics. First, there will be a reduction in network traffic, which results in faster data analysis and availability of the network for other uses. Second, the analytics server can be a relatively smaller configuration machine with reduced memory size and slower processor, thereby saving cost and power.

#To accomplish such in-drive analytics, it is proposed to dynamically switch portions (e.g., blocks or other units of memory) of the SSD between I/O mode and analytics mode In one embodiment, the switching is performed by the server or storage enclosure to which the SSD is attached and/or enclosed. Portions of the SSD in I/O mode can be used to erase, program and read host data (input/output or I/O operations). Data stored in the portions of the SSD in analytics mode can be analyzed as per any Big Data analytics algorithm. This approach is of great value as it precludes the need for an entire SSD to be made unavailable to a server when ‘in drive’ data analytics is in progress. The benefit is also useful in a virtualized environment when a single physical drive is split between multiple guest operating systems as virtual storage so that locking out all guest operating systems to enable one to perform data analysis is avoided. This proposal allows uninterrupted I/O operations to progress while a subset of SSD blocks will be in the analytics state.

#In some embodiment, to allow for customized data analysis, the system allows for the downloading of analytics logic onto the SSD by a host/analytics server in order to utilize the processing power inside the SSD to perform the analytics on the data stored in the SSD. In this manner, an analytics server may trigger analytics on files containing unstructured data (text, video, audio, images, etc.) resident on the SSD. The analytics logic can be software to program a processor or logic to configure FPGAs.