86 Open Source Batch Processing Software Projects

Run simple commands without opening AutoCAD.

The Proxima platform.

”Building a concurrent and distributed system for batch processing which is fault tolerant and can scale up or scale out using Akka.NET (based on actor model)”.

Fully automated batch processing of Revit files with your own Python or Dynamo task scripts!

Windows Batch script to automate batch processing using mkvtoolnix.

A reliable queue for batch processing written in Go and backed by Redis.

Automate Excel downloads in seconds. Simply does the tedious, repetitive operations for all rows of excel files step by step and reports after the job is done. It can download files from URL(s) in a column of Excel files. If a new filename is provided at column B it will rename the file before saving. It will even create sub folders if column C is filled with a valid folder name.

Example use of AWS batch

Recursive Neural Networks for PyTorch

DEPRECATED - no longer actively maintained. Automated workflow for harvesting, transforming and indexing of metadata using metha, OpenRefine and Solr. Part of the Hamburg Open Science "Schaufenster" software stack.

Generic file search & replace tool, written in Python 3

Extend python lists operations using .NET's LINQ syntax for clean and fast coding.

Serverless batch computing made simple.

Simple but powerful batch image rescaler.

The grid-control job submission tool

A Free Krita plugin to batch export layers and groups with maximum flexibility. Scale, trim, export multiple copies of your layers...

REST API for Spring Batch using Spring Boot 2.2

Asakusa Framework

Python Data Processing library

A standard implementation conforming to the Global Alliance for Genomics and Health (GA4GH) Workflow Execution Service (WES) API specification and a web application for managing and executing those WES services.

Unofficial alternative launcher for QuantConnect's LEAN allowing for parallel execution and looping/batching with customizable parameters and ranges.

A tiddler toolkit for Tiddlywiki 5

Online batch converter of SVG images to Android vector drawable XML resource files

(Developing)FLink.CSharp - a port of Apache Flink, an open source stream processing framework with powerful stream- and batch-processing capabilities. Learn more about Flink at https://flink.apache.org.

A tiny framework for building batch applications as a collection of tasks in a workflow.

A command-line program for batch processing

Flink & Spark development scaffold, Make stream processing easier!!! The original intention of StreamX is to make the development of Flink easier. StreamX focuses on the management of development phases and tasks. Our ultimate goal is to build a one-stop big data solution integrating stream processing, batch processing, data warehouse and data laker.

Sexy ssh makes batch ssh tasks sexy!

When Apache Pulsar meets Apache Spark

Elastic data processing with Apache Pulsar and Apache Flink

面向单机与分布式 OLTP/OLAP 场景的可暂停的渐进式 SQL 引擎 (只用于研究)

Easily Process a Batch of Cox Models

Sample Spring batch job for processing S3 objects one by one.

This library for Android will take any set of events and batch them up before sending it to the server. It also supports persisting the events on disk so that no event gets lost because of an app crash. Typically used for developing any in-house analytics sdk where you have to make a single api call to push events to the server but you want to optimize the calls so that the api call happens only once per x events, or say once per x minutes. It also supports exponential backoff in case of network failures

Stream processing guidelines and examples using Apache Flink and Apache Spark

Python scripts for the Abaqus FEA Python interpreter that allows for robust, flexible batch processing of output (.odb) files.

Road to Azure Data Engineer Part-I: DP-200 - Implementing an Azure Data Solution

MarkLogic tool for processing and reporting on content, enhanced from the original CoRB

A blazing fast job processing system backed by GenStage & Redis.

Elasticsearch tool for easily collecting and batch inserting Python data and pandas DataFrames

This is simple RPN implementation using tensorflow. It can be used for Faster-RCNN.

Optimal binning: monotonic binning with constraints. Support batch & stream optimal binning. Scorecard modelling and counterfactual explanations.

Road to Azure Data Engineer Part-II: DP-201 - Designing an Azure Data Solution

一个实现自动化导入Excel文件到MySQL数据库的工具

Tensorflow 2 Faster-RCNN implementation from scratch supporting to the batch processing with MobileNetV2 and VGG16 backbones

Go bindings for Imageflow -> https://github.com/imazen/imageflow

智能数据探索服务(Intelligent Data Exploration Service)

Multi-threaded batch processing and background jobs for .NET Core

Sample configuration for using Raspberry Pi's as Python Github Runners

It is an efficient, flexible, and lightweight goroutine pool. It provides an easy way to deal with concurrent tasks with limited resource.

A Mass Image Processing tool for Windows

Simple API, Complex Emails (Jakarta Mail smtp wrapper)

Symfony bundle that offers a batch processing system with Docker containers.

Hands on Batch files with Spring Batch

Video Converter & Transcoder, ffmpeg frontend: 1. set options, drag multiple files, 2. transcode MKV-MP4, 3. full manual edit of ffmpeg command, 4. record network streams, 5. batch list

Batch-processing of Revit models using the Dynamo visual programming environment

Ruby's missing background and batch processing system

BIMP. Batch Image Manipulation Plugin for GIMP.

A microservices-based Streaming and Batch data processing in Cloud Foundry and Kubernetes

Short lived microservices with Spring Batch

Detailed documentation is available here: http://ifiscripts.readthedocs.io/en/latest/index.html

Distributed Stream and Batch Processing

Average and morph faces online http://webmorph.org/

Parallel SSH Tools

The simple, stupid batch framework for Java

Simplify HPC and Batch workloads on Azure

Linux batch scheduler supporting job memory usage limits

*Decompile All the Things* - IDA Batch Decompile plugin and script for Hex-Ray's IDA Pro that adds the ability to batch decompile multiple files and their imports with additional annotations (xref, stack var size) to the pseudocode .c file

A simple utility for executing multiple sequential or multi-threaded applications in a single multi-node batch job

Parallel S3 and local filesystem execution tool.

Programming / Encode RFiD Tags with Batch - Read; Modify; Write Actions

Youtube video and audio downloader without any installation needed

Object Oriented Modular Abstract Calibration Toolbox

Euphoria is an open source Java API for creating unified big-data processing flows. It provides an engine independent programming model which can express both batch and stream transformations.

Batch processing library for Golang.

An applicative monad that batches commands for later more efficient execution

Shell script to run OpenRefine in batch mode (import, transform, export). It orchestrates OpenRefine (server) and a python client that communicates with the OpenRefine API.

Fast asynchronous DNS batch resolver

Batch Guetzli compressions in a manageable fashion

Multi-device OpenCL kernel load balancer and pipeliner API for C#. Uses shared-distributed memory model to keep GPUs updated fast while using same kernel on all devices(for simplicity).

Parallelized and efficient Node.js support for batch-mode child processes

FORMCEPT Whiteboard used by developers for developers

Distributed, fault tolerant batch processing for Natural Language Applications and Search, using remote partitioning

vinyl recommendation engine based on Discogs and engineered data

:truck: Batch requests for Rack apps (works with Rails, Sinatra, etc)