Measurement & mixed states for quantum systems.
Notes on measurement for quantum systems.
RNN backprop thru time(BPTT part 2) $$\frac{\delta h_{t}} {\delta h_{t-1}}$$
Notes on the math for RNN back propagation through time(BPTT), part 2. The 1st derivative of \(h_t\) with respect to \(h_{t-1}\).
Given a series: \(X = \{x_1, x_2...x_n\}\)
Given a set of functions that takes in \(X\):
\[Y = F(X)\]We have a vector of functions:
\[Y = \begin{pmatrix} f_1(X) \\ f_2(X) \\ . \\ . \\ . \\ f_n(X) \\ \end{pmatrix}\]A Jacobian is a matrix of 1st derivatives of the functions:
\[\begin{pmatrix} \frac{\delta y_1} {\delta x_1} & \frac{\delta y_1} {\delta x_2} & ... & \frac{\delta y_1} {\delta x_n} \\ . & . & ... & .\\ . & . & ... & .\\ . & . & ... & .\\ \frac{\delta y_n} {\delta x_1} & \frac{\delta y_n} {\delta x_2} & ... & \frac{\delta y_n} {\delta x_n} \end{pmatrix}\]For the calculations below, we are only interested in the diagonal derivatives which are terms with the same subscript in the nominator & denominator (i.e: derivatives with respect to the same time step).
\[\frac{\delta h_t} {\delta E_t} = diag(f'(E_t))\]Deriving \(\frac{\delta h_t} {\delta h_{t-1}}\):
\[Vx_t + Wh_{t-1} + b_{h} = E_t\] \[h_{t} = f_{h} (Vx_t + Wh_{t-1} + b_{h}) = f_{h}(E_t)\] \[\frac{\delta h_t} {\delta E_t} = diag(f'(E_t))\] \[\frac{\delta E_t} {\delta h_{t-1}} = W\] \[\frac{\delta h_t} {\delta h_{t-1}} = \frac{\delta h_t} {\delta E_t} \frac{\delta E_t} {\delta h_{t-1}} = diag(f'(E_t)) W\]2020
Quantum States as a generalization of classical probabilities.
Notes on quantum states as a generalization of classical probabilities.
Finding the ray_results folder in colab
The location of ray_results folder in colab when using RLlib &/or tune.
PBT for MARL
My attempt to implement a water down version of PBT (Population based training) for MARL (Multi-agent reinforcement learning).
RLlib trainer common config
Ray (0.8.2) RLlib trainer common config from:
Output dimension from convolution layer
How to calculate dimension of output from a convolution layer?
Changing G drive directory in Colab
Changing Google drive directory in Colab.
Linear regression (Bayesian)
Notes on the probability for linear regression (Bayesian)
RNN backprop thru time(BPTT part 2) \(\frac{\delta h_{t}} {\delta h_{t-1}}\)
Notes on the math for RNN back propagation through time(BPTT), part 2. The 1st derivative of \(h_t\) with respect to \(h_{t-1}\).
RNN backprop thru time(BPTT)
Notes on the math for RNN back propagation through time(BPTT).
2019
Filter rows with same column values in Pandas dataframe
Filter rows with same column values in a Pandas dataframe.
Custom Sagemaker reinforcement learning container
Building & testing custom Sagemaker RL container.
Reinforcement learning custom environment in Sagemaker with Ray (RLlib)
Demo setup for simple (reinforcement learning) custom environment in Sagemaker. This example uses Proximal Policy Optimization with Ray (RLlib).
Django + Postgres + Travis CI + Heroku CD
Basic workflow of testing a Django & Postgres web app with Travis (continuous integration) & deployment to Heroku (continuous deployment).
Django + Postgres + Docker + Travis CI + Heroku CD
Basic workflow of testing a dockerized Django & Postgres web app with Travis (continuous integration) & deployment to Heroku (continuous deployment).
Dockerized Postgres connection with Django web app in Travis CI
Introducing a delay to allow proper connection between dockerized Postgres & Django web app in Travis CI.
Random policy in RLlib
Creating & seeding a random policy class in RLlib.
Custom MARL (multi-agent reinforcement learning) CDA (continuous double auction) environment
A custom MARL (multi-agent reinforcement learning) environment where multiple agents trade against one another in a CDA (continuous double auction).
Tensorflow graphs in Tensorboard
This post demonstrate how setup & access Tensorflow graphs.
.bash_profile for Mac
This post demonstrates how to create customized functions to bundle commands in a .bash_profile file on Mac.
RND (Random Network Distillation) with Proximal Policy Optimization (PPO) Tensorflow
This post documents my implementation of the Random Network Distillation (RND) with Proximal Policy Optimization (PPO) algorithm. (continuous version)
DPPO distributed tensorflow
This post documents my implementation of the Distributed Proximal Policy Optimization (Distributed PPO or DPPO) algorithm. (Distributed continuous version)
A3C distributed tensorflow
This post documents my implementation of the A3C (Asynchronous Advantage Actor Critic) algorithm (Distributed discrete version).
A3C multi-threaded continuous version with N step targets
This post documents my implementation of the A3C (Asynchronous Advantage Actor Critic) algorithm. (multi-threaded continuous version)
A3C multi-threaded discrete version with N step targets
This post documents my implementation of the A3C (Asynchronous Advantage Actor Critic) algorithm (discrete). (multi-threaded discrete version)
Accumulate gradients with Tensorflow
This post demonstrates how to accumulate gradients with Tensorflow.
Distributed Tensorflow
This post demonstrates a simple usage example of distributed Tensorflow with Python multiprocessing package.
N-step targets
This post documents my implementation of the N-step Q-values estimation algorithm.
Python’s multiprocessing package
This post demonstrates how to use the Python’s multiprocessing package to achieve parallel data generation.
Numpy array manipulation
This post provides a simple usage examples for common Numpy array manipulation.
Dueling DDQN with PER
This post documents my implementation of the Dueling Double Deep Q Network with Priority Experience Replay (Duel DDQN with PER) algorithm.
Dueling DDQN
This post documents my implementation of the Dueling Double Deep Q Network (Dueling DDQN) algorithm.
DDQN
This post documents my implementation of the Double Deep Q Network (DDQN) algorithm.
DQN
This post documents my implementation of the Deep Q Network (DQN) algorithm.