Measurement & mixed states for quantum systems.
Notes on measurement for quantum systems.
Tensorflow graphs in Tensorboard
This post demonstrate how setup & access Tensorflow graphs.
In order to access Tensorflow graphs, you need to use Tensorboard which comes will Tensorflow installed.
The following snippet shows how to setup a FileWriter with a Tensorflow
graph.
tf.reset_default_graph()
# Your Tensorflow graph goes here.
# ...
sess = tf.Session()
sess.run(tf.global_variables_initializer())
# Declare tf.summary.FileWriter where log is your output directory for
# Tensorboard & add the graph to the writer.
writer = tf.summary.FileWriter('log', sess.graph)
# Run your training loop
# sess.run(...)
writer.close()
Run this command in terminal to start tensorboard:
$ tensorboard --logdir log
$ tensorboard --logdir ~/ray_results
Where log is the log folder.
Navigate to http://127.0.0.1:6006 in your browser to access Tensorflow.
Your graph is in the graph tab.
2020
Quantum States as a generalization of classical probabilities.
Notes on quantum states as a generalization of classical probabilities.
Finding the ray_results folder in colab
The location of ray_results folder in colab when using RLlib &/or tune.
PBT for MARL
My attempt to implement a water down version of PBT (Population based training) for MARL (Multi-agent reinforcement learning).
RLlib trainer common config
Ray (0.8.2) RLlib trainer common config from:
Output dimension from convolution layer
How to calculate dimension of output from a convolution layer?
Changing G drive directory in Colab
Changing Google drive directory in Colab.
Linear regression (Bayesian)
Notes on the probability for linear regression (Bayesian)
RNN backprop thru time(BPTT part 2) \(\frac{\delta h_{t}} {\delta h_{t-1}}\)
Notes on the math for RNN back propagation through time(BPTT), part 2. The 1st derivative of \(h_t\) with respect to \(h_{t-1}\).
RNN backprop thru time(BPTT)
Notes on the math for RNN back propagation through time(BPTT).
2019
Filter rows with same column values in Pandas dataframe
Filter rows with same column values in a Pandas dataframe.
Custom Sagemaker reinforcement learning container
Building & testing custom Sagemaker RL container.
Reinforcement learning custom environment in Sagemaker with Ray (RLlib)
Demo setup for simple (reinforcement learning) custom environment in Sagemaker. This example uses Proximal Policy Optimization with Ray (RLlib).
Django + Postgres + Travis CI + Heroku CD
Basic workflow of testing a Django & Postgres web app with Travis (continuous integration) & deployment to Heroku (continuous deployment).
Django + Postgres + Docker + Travis CI + Heroku CD
Basic workflow of testing a dockerized Django & Postgres web app with Travis (continuous integration) & deployment to Heroku (continuous deployment).
Dockerized Postgres connection with Django web app in Travis CI
Introducing a delay to allow proper connection between dockerized Postgres & Django web app in Travis CI.
Random policy in RLlib
Creating & seeding a random policy class in RLlib.
Custom MARL (multi-agent reinforcement learning) CDA (continuous double auction) environment
A custom MARL (multi-agent reinforcement learning) environment where multiple agents trade against one another in a CDA (continuous double auction).
Tensorflow graphs in Tensorboard
This post demonstrate how setup & access Tensorflow graphs.
.bash_profile for Mac
This post demonstrates how to create customized functions to bundle commands in a .bash_profile file on Mac.
RND (Random Network Distillation) with Proximal Policy Optimization (PPO) Tensorflow
This post documents my implementation of the Random Network Distillation (RND) with Proximal Policy Optimization (PPO) algorithm. (continuous version)
DPPO distributed tensorflow
This post documents my implementation of the Distributed Proximal Policy Optimization (Distributed PPO or DPPO) algorithm. (Distributed continuous version)
A3C distributed tensorflow
This post documents my implementation of the A3C (Asynchronous Advantage Actor Critic) algorithm (Distributed discrete version).
A3C multi-threaded continuous version with N step targets
This post documents my implementation of the A3C (Asynchronous Advantage Actor Critic) algorithm. (multi-threaded continuous version)
A3C multi-threaded discrete version with N step targets
This post documents my implementation of the A3C (Asynchronous Advantage Actor Critic) algorithm (discrete). (multi-threaded discrete version)
Accumulate gradients with Tensorflow
This post demonstrates how to accumulate gradients with Tensorflow.
Distributed Tensorflow
This post demonstrates a simple usage example of distributed Tensorflow with Python multiprocessing package.
N-step targets
This post documents my implementation of the N-step Q-values estimation algorithm.
Python’s multiprocessing package
This post demonstrates how to use the Python’s multiprocessing package to achieve parallel data generation.
Numpy array manipulation
This post provides a simple usage examples for common Numpy array manipulation.
Dueling DDQN with PER
This post documents my implementation of the Dueling Double Deep Q Network with Priority Experience Replay (Duel DDQN with PER) algorithm.
Dueling DDQN
This post documents my implementation of the Dueling Double Deep Q Network (Dueling DDQN) algorithm.
DDQN
This post documents my implementation of the Double Deep Q Network (DDQN) algorithm.
DQN
This post documents my implementation of the Deep Q Network (DQN) algorithm.