Policy Gradient Algorithms for Full RL, Hierarchical RL,POMDPs,