Technical Roadmap for 2021
====

Probabilistic Programming Layer
----

Distribution:

- [D1] add support to multivariate continous distributions
- [D1.1] MultivariateNormal
- [D1.2] GumbelSoftmax
- [D1.3] Dirichlet

Expected time: Q3

Modeling Language:

- [M1] Based on the new support of MindSpore, extend the BayesianNet modeling capacity to general
- [M1.1] Implement with dictionaies
- [M1.2] Integrate StochasticTensor with Distribution

Expected time: Q1

- [M2] Hybrid BayesianNet + Bijector modeling language
- [M2.1] Implement a bijector API, and integrate it with the existing bijector in distributions
- [M2.2] Implement affine coupling layer
- [M2.3] Reimplement the existing Glow / VFlow in the test folder with the new API

Depends on: M1
Expected time: Q4

Inference algorithm:

- [I1] Support common black-box algorithms
- [I1.1] Importance sampling
- [I1.2] SGLD
- [I1.3] Reinforce
- [I1.4] HMC

Depends on: M1
Expected time: [I1.1]:Q1, [I1.2]:Q2, [I1.3]:Q3, [I1.4]:Q4

Model and Toolbox Layer
----

- [T1] Reimplement existing model and toolbox with the probabilistic programming layer
- [T1.1] Reimplement VAE
- [T1.2] Reimplement BNN
- [T1.3] Reimplement Uncertainty Estimation
- [T1.4] Reimplement Anormaly Detection

Depends on: M1
Expected time: Q2

- [T2] Implement more SOTA algorithms of toolbox
- [T2.1] BNN with SGLD for uncertainty estimation
  Depends on: M1, I1.2
  Expected time: Q2
- [T2.2] SWAG for uncertainty estimation
  Depends on: D1.1, M1
  Expected time: Q3

(Optional) Test uncertainty estimation on language modeling

Appliation
----

- [A1] Apply our tools to representative applications

Depends on: T2
Expected time: Q4

Miscellanous
----

Keep interacting with MindSpore to improve the coding style.

Expected Goal
----

Q1:

- A general BayesianNet framework, that supports modeling of arbitary Bayesian networks

Q2:

- Support representative inference algorithms in the general format
- Rewrite applications and examples with the new interface

Q3:

- Implement more distributions, inference algorithms
- Implement some SOTA algorithms for uncertainty estimation
- Should have a comparable version of TF ZhuSuan now

Q4:

- Modeling tools for flow bases
- Tentative applications of uncertainty estimation

