Not everyone is aware that we use Object Storage technology every day for social networking on sites such as Facebook™, LinkedIn™, Instagram™, Dropbox™ and YouTube™, to name a few. These companies use Object Storage to store users’ content like photos, streaming videos, collaboration files, and much more. Using this technology, they can easily store the content of billions of users. Facebook™, for example, had more than 2.01 billion active users in June 2017.

Until recently, the use cases for Object Storage have been predominantly based around cloud storage applications, but in reality, they are commonly used in markets such as finance, media & entertainment, big data analytics, data science, video surveillance, healthcare and in any context where large amounts of data need to be stored very inexpensively, with the ability to analyze and manage data stored with a level of granularity.

Aside from specific businesses where it is possible to predict storage space occupancy with precision, allowing firms to assign resources precisely, in the enterprise market data is growing exponentially for all organizations, regardless of market or size. Managing, storing and accessing that data is a constant and costly challenge. This is putting increasing pressure on the storage infrastructure which requires constant maintenance, expensive upgrades and continuous investment, as any IT professional knows. Storage costs are rising in particular due to the dramatic growth in unstructured data from multiple sources and formats: commercial, scientific, images, multimedia, podcasts (digital audio files), vodcasts (digital video files), video surveillance, MP3s, Internet of Things (IoT), and all activities that generate huge number of PDF, XLS, PPT, XML, JPEG files and many other types.

Inactive data not only occupies expensive storage space, but is backed up and migrated onto new storage for years, an illogical and crazy state of affairs that goes largely ignored. Rather than address this clear and undisputed problem, companies instead persist in buying traditional RAID and NAS for storing cold data and continuing in this way to sustain high costs and complexity, kicking the problem down the road.

Traditional file systems and storage systems were created in another technology era, when the volume of data to be managed, stored and accessed was in no way comparable to today’s needs. As we know, traditional file systems store data in hierarchical structures consisting of directories and subdirectories with a hierarchy of nested folders, subfolders and files. This creates highly complex tree structures that make it very difficult to manage large sets of files. As the amount of data grows, performance of the file system and storage is dramatically degraded. Standard file systems use only basic metadata to describe the file, such as the filename, date of creation, last modification date and type. This limited amount of metadata does not provide sufficient information to efficiently handle large volumes of unstructured data in the underlying storage infrastructure. Instead, Object Storage can store files known as objects with a rich set of metadata attached, giving users the context and the content. Unlike conventional storage systems, Object Storage is not accessed directly from the operating system but through standard Internet protocols such as HTTP and APIs like S3, REST, OpenStack Swift, SOAP and CDMI and so on.

Objects contain both data (information) and data descriptors called metadata. Objects are addressed using unique ID numbers managed in a flat index, which means they can be found and identified without knowing their exact location in the system. Metadata descriptors are defined by the application when the object is created, and additional information can be attached to a particular object group or individual object at any time. Meanwhile, objects are identified by applications via  metadata, so programs can be designed specifically to manage information using a specific metadata scheme

Object storage solutions provide the ideal way to handle PDF, XLS, PPT, XML, JPEG files, audio, image, video production and post-production files for healthcare, scientific research, big data analytics, IoT, or any sector where information can be very sophisticated and can vary over time. Thanks to rich set of metadata, object storage offers unmatched flexibility compared with traditional storage systems.

It’s also possible define metadata retention and deletion policies for scheduled storage management, and these properties can also be useful for regulatory compliance.

As we know, metadata descriptors are defined by the application: for this reason, object storage is an extremely valuable storage model allowing organizations to enjoy dependable security benefits, manage data growth and exploit a range of new opportunities.

For example, in a healthcare setting, patient information must be archived for years if not decades, but may need to be accessed in a matter of seconds: Thanks to a rich set of metadata, medical reports can be accessed through different keys as shown in the X-ray image example to right.

From a hardware point of view, Object Storage consists of multiple nodes, each of which is a standard commodity X86 server with storage capacity and I/O (input/output) bandwidth. Capacity is increased by adding new nodes which are linked together through a high-speed internal network. The entire cluster of nodes is managed by the administrator as a single storage system. Embedding the resources in each single node means it is possible to grow capacity and performance linearly.

Object Storage data protection offers much higher resilience and capacity efficiency compared to traditional RAID approach. It is certainly true that RAID technology offers ever-higher performance as it has become more and more sophisticated. But at the end of the day, even the most sophisticated RAID architecture has to rely on hard disk performance. RAID technology was born in an era where disks had relatively little capacity, in the order of a few megabytes. Over the years capacity increased significantly to reach the present 12 TB. In the meantime, increase in HDD speed has failed to keep pace with the increase in capacity. As a result, RAID technology is generally quite inadequate for modern enterprise data protection and recovery needs.

Object Storage offers flexible and scalable data storage for most applications and services being developed with modern techniques and tools. The rise of Object Storage is partly due to the simplicity of interacting with this storage technology by software developers.
Because the API consists of standard HTTP requests, libraries were quickly developed for most programming languages. Saving a chunk of data became as easy as an HTTP PUT request to the object store. Retrieving the file and metadata is a normal GET request.

QStar Cloud Gateway solution integrates seamlessly into existing infrastructures without changes for either the applications or users. Migrated or moved data continues to appear locally even when stored in the Cloud. The simple data read/write approach is virtually compatible with all existing applications. QStar Cloud Gateway can archive unlimited amounts of information securely and cost-effectively within any Cloud – private, public or hybrid such as Google, Amazon, Azure, IBM Bluemix and many others. QStar Cloud Gateway is designed to combat the relentless explosion of data, providing transparent and automatic migration of static files from local infrastructure to the Cloud, using attributes such as date created, modified, accessed, file owner, size and extension. Meanwhile, it restores data to its original location in real time, completely transparently to applications, network and users.