107 Open Source Bigquery Software Projects
Free and open source bigquery code projects including engines, APIs, generators, and tools.
Redash 19753 ⭐
Make Your Company Data Driven. Connect to any data source, easily visualize, dashboard and share your data.
Professional Services 1875 ⭐
Common solutions and tools developed by Google Cloud's Professional Services team
Ethereum Etl 1247 ⭐
Python scripts for ETL (extract, transform and load) jobs for Ethereum blocks, transactions, ERC20 / ERC721 tokens, transfers, receipts, logs, contracts, internal transactions. Data is available in Google BigQuery https://goo.gl/oY5BCQ
Dataflowtemplates 745 ⭐
Google-provided Cloud Dataflow template pipelines for solving simple in-Cloud data tasks
Issue Label Bot 303 ⭐
Code For The Issue Label Bot, an App that automatically labels issues using machine learning, available on the GitHub Marketplace. This is also code for the blog article: "How to automate tasks on GitHub with machine learning for fun and profit"
Bigquery Utils 482 ⭐
Useful scripts, udfs, views, and other utilities for migration and data warehouse operations in BigQuery.
NodeJS Bigquery 328 ⭐
Node.js client for Google Cloud BigQuery: A fast, economical and fully-managed enterprise data warehouse for large-scale data analytics.
Almanac.httparchive.org 386 ⭐
HTTP Archive's annual "State of the Web" report made by the web community
Hadoop Connectors 236 ⭐
Libraries and tools for interoperability between Hadoop-related open-source software and Google Cloud Platform.
Mara Example Project 2 162 ⭐
An example mini data warehouse for python project stats, template for new projects
Gpt2 Bert Reddit Bot 170 ⭐
a bot that generates realistic replies using a combination of pretrained GPT-2 and BERT models
Bitcoin Etl 224 ⭐
ETL scripts for Bitcoin, Litecoin, Dash, Zcash, Doge, Bitcoin Cash. Available in Google BigQuery https://goo.gl/oY5BCQ
Gojek Beast 147 ⭐
[Deprecated] Load data from Kafka to any data warehouse. BQ sink is being supported in Firehose now.
Bigquery Schema Generator 142 ⭐
Generates the BigQuery schema from newline-delimited JSON or CSV data records.
Spark Bigquery Connector 170 ⭐
BigQuery data source for Apache Spark: Read data from BigQuery into DataFrames, write DataFrames into BigQuery tables.
Linq To Bigquery 75 ⭐
LINQ to BigQuery is C# LINQ Provider for Google BigQuery. It also enables Desktop GUI Client with LINQPad and plug-in driver.
Ethereum Etl Airflow 145 ⭐
Airflow DAGs for exporting, loading, and parsing the Ethereum blockchain data. How to get any Ethereum smart contract into BigQuery https://towardsdatascience.com/how-to-get-any-ethereum-smart-contract-into-bigquery-in-8-mins-bab5db1fdeee
Circus Train 76 ⭐
Circus Train is a dataset replication tool that copies Hive tables between clusters and clouds.
Samelamin Spark Bigquery 66 ⭐
Google BigQuery support for Spark, Structured Streaming, SQL, and DataFrames with easy Databricks integration.
Bigquery View Analyzer 68 ⭐
A command-line tool for managing permissions and dependencies for BigQuery authorized views
Bigquery To Datastore 53 ⭐
Export a whole BigQuery table to Google Datastore with Apache Beam/Google Dataflow
Gcp Census 41 ⭐
GAE python based app which regularly collects information about GCP resources and stores them in BigQuery
Dlp Dataflow Deidentification 56 ⭐
Multi Cloud Data Tokenization Solution By Using Dataflow and Cloud DLP
Weather Station Gcp Mongoose Os 45 ⭐
A Weather station made with an ESP32, sending data through Google Cloud IoT Core and storing in BigQuery
Github Activity Counter 42 ⭐
Cloud Run service for GitHub event Webhook to monitor repo or org activity in real-time in Stackdriver and analyze activity through ad-hoc SQL queries in BigQuery
Coolretailer 39 ⭐
Microservices with Istio, gRPC, Redis, BigQuery, Spring Boot, Spring Cloud and Stackdriver
Ob_google Bigquery 43 ⭐
This service is meant to simplify running Google Cloud operations, especially BigQuery tasks. This means you do not have to worry about installation, configuration or ongoing maintenance related to an SDK environment. This can be helpful to those who would prefer to not to be responsible for those activities.
Spark On K8s Gcp Examples 33 ⭐
Example Spark applications that run on Kubernetes and access GCP products, e.g., GCS, BigQuery, and Cloud PubSub
Ostelco Core 30 ⭐
Cloud-native Telco BSS hosted in GCP K8s with standalone Diameter to gRPC gateway. Rule Engine using Neo4j graphs. Analytics Events sent to GCP BigData (Dataflow+BigQuery) via PubSub. It's awesome!
Bigquery Sheets Slides 35 ⭐
Code repo for the Google Apps Script BigQuery-Sheets-Slides codelab application
Firestore To Bigquery Export 27 ⭐
NPM package for copying and converting Cloud Firestore data to BigQuery.
Etlflow 36 ⭐
Functional, Composable library in Scala based on ZIO for writing ETL jobs in AWS and GCP
Bigquery Data Lineage 68 ⭐
Reference implementation for real-time Data Lineage tracking for BigQuery using Audit Logs, ZetaSQL and Dataflow.
Prometheus Bigquery Exporter 25 ⭐
An exporter for converting BigQuery results into Prometheus metrics
Hive_compared_bq 24 ⭐
hive_compared_bq compares/validates 2 (SQL like) tables, and graphically shows the rows/columns that are different.
Target And Market 18 ⭐
A data-driven tool to identify the best candidates for a marketing campaign and optimize it.
Datalab Notebooks 15 ⭐
This repository includes end-to-end labs on how to use GCP for applied data science
Nautilus Connectors Kit 29 ⭐
ACK is an E(T)L tool specialized in API data ingestion. It is accessible through a Command-Line Interface. The application allows you to easily extract, stream and load data (with minimum transformations), from the API source to the destination of your choice.
Bigquery Firebase Funnel Builder 15 ⭐
A Python script that builds a funnel for Google BigQuery with Firebase Analytics.
Google_pubsub_bigquery 13 ⭐
Demonstrating the concept of Google PubSub, a messaging queue service in Google, thru streaming fake financial data thru PubSub and querying within BigQuery
Mercari Dataflowtemplates 17 ⭐
Convenient Dataflow pipelines for transforming data between cloud data sources