61 Open Source Data Collection Software Projects

ODK Collect is an Android app for filling out forms. It's been used to collect billions of data points in challenging environments around the world. Contribute and make the world a better place! ✨📋✨

Player Analytics plugin for Minecraft Server platforms (Bukkit/Sponge/Nukkit/BungeeCord/Velocity) - View player activity of your server with ease. :calendar:

Python library and web service for Open Source Software Health and Sustainability metrics & data collection.

Social media scraping / data collection tool for the Facebook, Twitter, Reddit, YouTube, Pinterest, and Tumblr APIs

A data collection library for click and exposure event with the UI.

📱🌐📋 wq: a modular framework supporting web / native apps for mobile surveys and geospatial data collection. Powered by Django REST Framework, Redux, React, and Material UI.

IOT DC3 is an open source, distributed Internet of Things (IOT) platform based on Spring Cloud. It is used for rapid development of IOT projects and management of IOT devices. It is a set of solutions for IOT system.

Ground mobile data collection app for Android

☁🌐 wq's db library, extending Django REST framework to support apps for geospatial field data collection, citizen science, and crowdsourcing.

ODK Build is a drag-and-drop form designer for ODK XForms. Thousands of users around the world depend on it for their data collection campaigns. Contribute and make the world a better place! ✨📝✨

💻📱 wq's app library: a JavaScript framework powering offline-first web & native apps for geospatial data collection, mobile surveys, and citizen science. Powered by Redux, React, Material UI and Mapbox GL.

ODK Briefcase is a Java application for fetching and pushing forms and their contents. It helps make billions of data points from ODK portable. Contribute and make the world a better place! ✨💼✨

A data collection and monitoring tool that works anywhere.

This is a data pipeline for Twitter (ETL) using the elastic stack Elasticsearch, Logstash and Kibana (version 6.1)

Open, extensible Python injector/profiler/analyzer

The core library that many of the ODK tools are built around. It's written in Java, implements the ODK XForms spec, and runs on mobile devices and cloud servers. ✨🏗✨

open-source intelligence gathering for SIEMs <3

Automatic Speech Recognition Dataset Generation

这阵子由于项目需要，需要从手机上采集用户的运动轨迹数据，这样的功能大家都见到的很多了，比如咕咚、悦动圈，对跑步运动轨迹数据进行采集，再如，微信运动、钉钉运动，对于每一天你走步进行计数，如果要记录轨迹就离不开的手机定位，如果要记录步数那就离不开陀螺仪（角速度传感器），花了一天多的时间实现了一个定位数据实时采集的功能。

Development of AI audio app to compare the cough of a Coronavirus (COVID-19) infected individual with the cough of an uninfected individual.

Skunkworks-crow is an Android app for sharing filled forms stored in ODK Collect from one device to another

The XForms-derived specification used in the ODK ecosystem. If you are interested in building a tool that is compliant with the forms rendered by ODK tools, this is the place to start. ✨⚒✨

Data Collection System For NLP/Speech Recognition

🌎 Data collection of geographical divisions of Brazil by IBGE (https://servicodados.ibge.gov.br/api/docs)

Simple, RESTful Log Collector

Mobile data collection app using the iOS Runtime SDK.

A python web scraper for public email lists.

Data collection application built using the .NET Runtime SDK.

Akvo Flow app

REST and STREAMING crawlers of Twitter (java)

Python program for automated electron diffraction data collection

💡📋 xlsconv: Command-line tool to convert questionnaires from Open Data Kit's XLSForm format into Django models and HTML (for use with https://wq.io/).

Ingestly Client JavaScript SDK for modern web analytics.

Selenium Image Crawler

pull devices and transform data into events

Sistema para escalonamento e orquestração de execuções, visando a automatização de processos do DadosJusBR

Retrieves the full list of available Google Voice numbers and finds the best ones

Vue.js based frontend for ODK Central

Mobile Android app for data collection in the wild: sensor measurements and self-reported labels describing the user's behavioral context.

API (Python) for Eurostat data collections upload

Web tools to do research with Myo

Open-source Twitter collection and archiving tool for tracking specific topics and collecting bulk data.

Repository to have example code to demonstrate JavaScript Map and Set data structures.

大数据采集,抽取平台

The enterprise-grade behavioral data engine (web, mobile, server-side, webhooks), running cloud-natively on AWS and GCP

Jitsu is an open-source Segment alternative. Fully-scriptable data ingestion engine for modern data teams. Set-up a real-time data pipeline in minutes, not days

:fire: One of the most comprehensive open-source data annotation platform.

An Arduino library providing a minimal interface to log data on flash memory and SD cards with ESP8266 and ESP32

Source code for the paper titled "Speech Denoising without Clean Training Data: a Noise2Noise Approach". Paper accepted at the INTERSPEECH 2021 conference. This paper tackles the problem of the heavy dependence of clean speech data required by deep learning based audio denoising methods by showing that it is possible to train deep speech denoising networks using only noisy speech samples.

Android app for synchronized recording of video and IMU data with advanced camera options, useful for 3D reconstruction, SLAM, AR, video stabilization. Supports remote control over network.

A design environment for creating, customizing, and previewing your forms, data curation, and visualization applications.

Cryptocurrencies price data collection, price tickers, signals notifications, charts and automatic trading.

Goblyn is a Python tool focused to enumeration and capture of website files metadata.

fetches historical OHLC values from most crypto exchanges using ccxt library. saves candles to a sqlite database per symbol.

Coletores de dados sobre remunerações do sistema de justiça brasileiro

This is a QGIS plugin which enables automatic survey form creation from GIS layers and uploading to ODK aggregate server. User can import existing data into GIS layers or enable sync with regular interval.

Ingestly Endpoint for Real-Time Analytics powered by Fastly & Google BigQuery

An infrastructure-agnostic framework for distributed data collection, part of the CACHET Research Platform (CARP).

Turn any live video stream or locally stored video into a dataset of interesting samples for ML training, or any other type of analysis.

Data collection framework for wearable sensors

FACT is a tool to collect, process and visualise forensic data from clusters of machines running in the cloud or on-premise.