Object handover is a basic and yet essential capability for robots interacting with humans in a broad range of applications, including caring for the elderly, assisting in a manufacturing workcell, etc. It appears deceptive simple, as humans perform object handover almost flawlessly. The success of humans, however, belies the complexity of object handover as a collaborative physical interaction between a robot and a human. This works addresses the problem of dynamic object handover, for example, when a robot hands over a water bottle to a marathon runner running by a water station. We formulate the problem as context-aware policy search, which enables the robot to learn dynamic object handover through interaction with the human. One key challenge here is to learn the latent reward of the handover task under noisy human feedback. Experiments show that the robot can learn to hand over an object with very high success rate. It can also adapt to the dynamics of human motion naturally.