53 Open Source Information Theory Software Projects
Free and open source information theory code projects including engines, APIs, generators, and tools.
Robust Log Optimal Strategy With Reinforcement Learning55 ⭐
We propose a new Portfolio Management strategy combining Log-Optimal based Strategy and Reinforcement-Learning based Strategy.
An open-source library for Python 3 providing tools for analysis and simulation of analog and digital communication systems.
Interpretable data visualizations for understanding how texts differ at the word level
Code and data for On the Predictability of Infectious Disease Outbreaks by SV Scarpino & G Petri
Information Theory Deep Learning30 ⭐
Resources and Implementations (PyTorch) for Information Theoretical concepts in Deep Learning
Limit Label Memorization34 ⭐
Improving generalization by controlling label-noise information in neural network weights.
Framework for Information Theoretical analysis of Electrophysiological data and Statistics
Code for our paper: "Regularity Normalization: Neuroscience-Inspired Unsupervised Attention across Neural Network Layers".
CorEx or "Correlation Explanation" discovers a hierarchy of informative latent factors. This reference implementation has been superseded by other versions below.
Kxy Python10 ⭐
A Powerful Serverless Analysis Toolkit That Takes Trial And Error Out of Machine Learning Projects
Official Implementation of Learning Diverse and Discriminative Representations via the Principle of Maximal Coding Rate Reduction (2020)
Mdl Complexity13 ⭐
MDL Complexity computations and experiments from the paper "Revisiting complexity and the bias-variance tradeoff".
computes most of information functions (joint entropy, conditional, mutual information, total correlation information distance) and deep information networks
Go Compression Raisin16 ⭐
A simple lightweight set of implementations and bindings for compression algorithms written in Go.
Bestape Alchemy29 ⭐
Generate any a-by-( b + c ) finite rectangle SVG containing potentially Infinitely many a-by-( 2 * b ) finite rectangles animated along a number line of ( ( c - b ) / a )^n scale symmetry.
Tensorflow 2 source code for the PI-SAC agent from "Predictive Information Accelerates Learning in RL" (NeurIPS 2020)
[ICLR 2021] "InfoBERT: Improving Robustness of Language Models from An Information Theoretic Perspective" by Boxin Wang, Shuohang Wang, Yu Cheng, Zhe Gan, Ruoxi Jia, Bo Li, Jingjing Liu
Implementation of the models and datasets used in "An Information-theoretic Approach to Distribution Shifts"
Honest But Curious Nets11 ⭐
Honest-but-Curious Nets: Sensitive Attributes of Private Inputs Can Be Secretly Coded into the Classifiers' Outputs (ACM CCS'21)
The Information Dynamics Toolkit xl (IDTxl) is a comprehensive software package for efficient inference of networks and their node dynamics from multivariate time series data using information theory.
JIDT: Java Information Dynamics Toolkit for studying information-theoretic measures of computation in complex systems
It is my belief that you, the postgraduate students and job-seekers for whom the book is primarily meant will benefit from reading it; however, it is my hope that even the most experienced researchers will find it fascinating as well.
Hierarchical unsupervised and semi-supervised topic models for sparse count data with CorEx
Fast, linear version of CorEx for covariance estimation, dimensionality reduction, and subspace clustering with very under-sampled, high-dimensional data
Math Php2015 ⭐
Powerful modern math library for PHP: Features descriptive statistics and regressions; Continuous and discrete probability distributions; Linear algebra with matrices and vectors, Numerical analysis; special mathematical functions; Algebra
A flexible version of CorEx developed for bio-data challenges that handles missing data, continuous/discrete variables, multi-CPU, overlapping structure, and includes visualizations
Lempel Ziv_complexity23 ⭐
:package: Lempel-Ziv Complexity, fast implementations with :snake: Python (naive, Numba or Cython for speedup), Open-Source (MIT) :+1: →
Probabilistic question-asking system: the program asks, the users answer. The minimal goal of the program is to identify what the user needs (a target), even if the user is not aware of the existence of such a thing/product/service.