Data Pipeline Service — Microservices Tutorial

Mike McWhorter

We recently announced several exciting new features in the ActiveScale™ 5.5 OS, including our Data Pipeline Service. The Data Pipeline Service monitors the ActiveScale object storage system for changes (such as upload, download, copy or deletion) and sends out a notification when any S3 event occurs. This is beneficial for applications that subscribe to and process events – particularly microservices. In this blog I’ll walk you through the feature and show a microservice tutorial in a hybrid cloud model.

Why Microservices

The concept of microservices is that rather than building and maintaining complex applications, an easier, more flexible and robust approach is to break applications into smaller, composable components that can work together. Basically, these are single-function modules that are well-defined and distinct, and together they make up a complex software application.

The trend of microservices has grown more popular in recent years as organizations look to become more agile and lean. Microservices are usually easier to understand, test, and maintain and it opens a broad spectrum of 3rd party tools and services offered through cloud platforms.

Data Pipeline Service — Microservices Tutorial

In the video below I walk you through the new Data Pipeline Service feature and a show a microservice tutorial where files are processed automatically after an event occurs on the ActiveScale system. You’ll see that the event notifications happen on the ActiveScale system on premises, while the process is a public cloud service. This is a great example of a hybrid configuration that allows you to own your data while accessing the full array of public cloud-based services.

If you’re investigating how to best architect your data environment, check out some of my previous blogs on object and cloud storage:

Why Object Storage? A Short, Definitive Explanation

Private Cloud vs. Public Cloud – What Workloads Fit Where

 

The post Data Pipeline Service — Microservices Tutorial appeared first on Western Digital Corporate Blog.

Previous Article
Examples of Unstructured Data ⁠— 4 Enterprise Use Cases
Examples of Unstructured Data ⁠— 4 Enterprise Use Cases

Examples of unstructured data include large media files, massive databases, data lakes, and architectural i...

Next Article
The Key to Your Data’s Golden Copy – Hello ActiveScale™ 5.5
The Key to Your Data’s Golden Copy – Hello ActiveScale™ 5.5

Today, we announced the new ActiveScale™ 5.5 OS. Packed with more capacity, new features, and better perfor...