Reviews recent successes of both model-free and model-based policy search in robot learning. This text classifies model-free methods based on their policy evaluation, policy update, and exploration strategies, and presents a unified view of existing algorithms.