Measurement & mixed states for quantum systems.
Notes on measurement for quantum systems.
Output dimension from convolution layer
How to calculate dimension of output from a convolution layer?
No padding (aka valid padding):
input = n x n = 6 x 6
kernel = f x f = 3 x 3
output = m x m = 4 x 4
How do we get output = 4 x 4 ?
Ans: Use the formula: (n - f + 1) x (n - f + 1)
With padding of size 1:
p = 1
input = n x n = 6 x 6
kernel = f x f = 3 x 3
output = m x m = 6 x 6
How do we get output = 6 x 6 ?
Ans: Use the formula: (n + 2p - f + 1) x (n + 2p - f + 1)
Meaning of valid padding & same padding:
1) No padding is also known as valid padding.
2) Same padding means pad input so that the resulting output dimension after convolution will be the same as input.
Size of padding needed to achieve same padding:
Size of padding needed to achieve same padding depends on the kernel size, f.
Using p = (f - 1) / 2 will produce output dimension = input dimension.
2020
Quantum States as a generalization of classical probabilities.
Notes on quantum states as a generalization of classical probabilities.
Finding the ray_results folder in colab
The location of ray_results folder in colab when using RLlib &/or tune.
PBT for MARL
My attempt to implement a water down version of PBT (Population based training) for MARL (Multi-agent reinforcement learning).
RLlib trainer common config
Ray (0.8.2) RLlib trainer common config from:
Output dimension from convolution layer
How to calculate dimension of output from a convolution layer?
Changing G drive directory in Colab
Changing Google drive directory in Colab.
Linear regression (Bayesian)
Notes on the probability for linear regression (Bayesian)
RNN backprop thru time(BPTT part 2) \(\frac{\delta h_{t}} {\delta h_{t-1}}\)
Notes on the math for RNN back propagation through time(BPTT), part 2. The 1st derivative of \(h_t\) with respect to \(h_{t-1}\).
RNN backprop thru time(BPTT)
Notes on the math for RNN back propagation through time(BPTT).
2019
Filter rows with same column values in Pandas dataframe
Filter rows with same column values in a Pandas dataframe.
Custom Sagemaker reinforcement learning container
Building & testing custom Sagemaker RL container.
Reinforcement learning custom environment in Sagemaker with Ray (RLlib)
Demo setup for simple (reinforcement learning) custom environment in Sagemaker. This example uses Proximal Policy Optimization with Ray (RLlib).
Django + Postgres + Travis CI + Heroku CD
Basic workflow of testing a Django & Postgres web app with Travis (continuous integration) & deployment to Heroku (continuous deployment).
Django + Postgres + Docker + Travis CI + Heroku CD
Basic workflow of testing a dockerized Django & Postgres web app with Travis (continuous integration) & deployment to Heroku (continuous deployment).
Dockerized Postgres connection with Django web app in Travis CI
Introducing a delay to allow proper connection between dockerized Postgres & Django web app in Travis CI.
Random policy in RLlib
Creating & seeding a random policy class in RLlib.
Custom MARL (multi-agent reinforcement learning) CDA (continuous double auction) environment
A custom MARL (multi-agent reinforcement learning) environment where multiple agents trade against one another in a CDA (continuous double auction).
Tensorflow graphs in Tensorboard
This post demonstrate how setup & access Tensorflow graphs.
.bash_profile for Mac
This post demonstrates how to create customized functions to bundle commands in a .bash_profile file on Mac.
RND (Random Network Distillation) with Proximal Policy Optimization (PPO) Tensorflow
This post documents my implementation of the Random Network Distillation (RND) with Proximal Policy Optimization (PPO) algorithm. (continuous version)
DPPO distributed tensorflow
This post documents my implementation of the Distributed Proximal Policy Optimization (Distributed PPO or DPPO) algorithm. (Distributed continuous version)
A3C distributed tensorflow
This post documents my implementation of the A3C (Asynchronous Advantage Actor Critic) algorithm (Distributed discrete version).
A3C multi-threaded continuous version with N step targets
This post documents my implementation of the A3C (Asynchronous Advantage Actor Critic) algorithm. (multi-threaded continuous version)
A3C multi-threaded discrete version with N step targets
This post documents my implementation of the A3C (Asynchronous Advantage Actor Critic) algorithm (discrete). (multi-threaded discrete version)
Accumulate gradients with Tensorflow
This post demonstrates how to accumulate gradients with Tensorflow.
Distributed Tensorflow
This post demonstrates a simple usage example of distributed Tensorflow with Python multiprocessing package.
N-step targets
This post documents my implementation of the N-step Q-values estimation algorithm.
Python’s multiprocessing package
This post demonstrates how to use the Python’s multiprocessing package to achieve parallel data generation.
Numpy array manipulation
This post provides a simple usage examples for common Numpy array manipulation.
Dueling DDQN with PER
This post documents my implementation of the Dueling Double Deep Q Network with Priority Experience Replay (Duel DDQN with PER) algorithm.
Dueling DDQN
This post documents my implementation of the Dueling Double Deep Q Network (Dueling DDQN) algorithm.
DDQN
This post documents my implementation of the Double Deep Q Network (DDQN) algorithm.
DQN
This post documents my implementation of the Deep Q Network (DQN) algorithm.