Operational AI WIKI

All 0-9 A B C D E F G H I J K L M N O P Q R S T U V W X Y Z

Batch processing

Batch processing is a technique for processing high volumes of data in batches before it can be used and then storing it in a more usable format. Batch processes are an efficient way of processing high volumes of data because they are able to run without any user interaction, batches can process simultaneously, and they can be scheduled incrementally.

 

Batch processing can save costs because it reduces the hardware and complexity necessary to achieve real-time processing when dealing with large amounts of data. Batching can also reduce the need for manual data input.

 

Examples of applications where batch processing works well include: payroll processes, end-of-month reconciliation, and periodic billing systems.

Graph that shows the difference between batch processing and stream processing

When deciding between batch processing and stream processing, the optimal method varies depending on needs, the company, and the specific situation. https://www.bmc.com/blogs/what-is-batch-processing-batch-processing-explained/

 

At Molecula

Molecula’s FeatureBase is a feature-oriented database platform for real-time analytics and machine learning applications. FeatureBase eliminates the need to batch data because it simultaneously executes low-latency, high-throughput, and highly concurrent workloads on massive data in real time. Batching data can work for many applications, but the typical Molecula customer needs to immediately return complex queries across both historical and streaming data, which can not be accomplished via batch processing. FeatureBase users are able to eliminate batch processing, simplify their data infrastructure, and significantly reduce data costs.

 

Related Terms

Streaming data

 

Learn More

Wikipedia entry: Batch processing

Talend: Beginner’s Guide to Batch Processing

BMC: Batch Processing: An Introduction

 

Other Occurrences

Related Terms

WIKI Contribution

  • Have a new word to add?
  • Have an updated definition?
Become a contributor