Real time vs Batch Serving

Is Real-Time Processing Worth It For Your Analytical Use Cases?

Towards Data Science

Real-time data pipelines provide a notable advantage over batch processing — data becomes available to consumers faster. In the traditional ETL, you would not be able to analyze events from today…

Batch Processing vs Stream Processing

Level Up Coding

When we encounter systems with large amounts of data, there are 2 main ways we can crunch that data to transform it into something useful for our organisation. One of those approaches is batch…

The magical fusion between batch and streaming insights

Towards Data Science

Business needs data driven insights to reach stakeholders and customers as fast as possible and with clearly defined freshness requirements. The Lambda architecture (Marz, 2015) solves these requireme...

Batch And Streaming Demystified For Unification

Towards Data Science

Understand how batch can be considered a subset of streaming and why data engineering should simplify its usage significantly Continue reading on Towards Data Science

How Modern Inference Servers Supercharge GPU Throughput with Batching

Towards AI

When deploying large language models (LLMs) or embedding models in production, throughput is king. A single request may feel instantaneous, but as traffic scales, the difference between serial executi...

Batch vs. Online Learning: Which Approach Fits Your Machine Learning Needs? (Part 2)

Python in Plain English

This blog post continues the discussion from my previous blog on batch learning, where I explained how models undergo training on large datasets in one go. In this post, we’ll explore online learning ...

Kubernetes Was Never Designed for Batch Jobs

Better Programming

And what it would look like if it had been Photo by Lisa Hanly on Unsplash In this post, we’ll make the case that Kubernetes is philosophically biased towards microservices over batch jobs. This resu...

Four fails and a win at a big data stack for realtime analytics

Towards Data Science

Building a user-friendly app to analyze big data in real time (that is, keeping response times below 60 seconds) is a challenge. In the big data world, you’re either doing batch analytics where…

Introduction to Streaming Frameworks

Towards Data Science

Introduction As data architectures are becoming more and more mature, streaming is no longer considered a luxury but a technology with a wide range of applications across different industries. Because...

Do Real-Time Data Pipelines Even Exist?

Towards Data Science

Sharing a fresh perspective on real-time data pipelines Photo by Djim Loic on Unsplash How often have you heard these terminologies — real-time data pipelines or real-time data processing or real-tim...

Batch Learning vs. Online Learning in Machine Learning

Level Up Coding

Table of Contents 1. Introduction 2. Batch Learning 3. Online Learning 4. Comparison: Batch vs. Online Learning 5. When to Use Batch or Online Learning 6. Interview Questions for Data Scientists and M...

Will 2025 Be the Year Real-Time Analytics Finally Goes Mainstream?

Towards Data Science

Since the Hadoop era and the early days of Big Data, one prediction kept coming back year after year: “Next year will be the year of streaming.” There was a constant expectation that eventually, most ...