July 22, 2019
Playing Atari Games in OCaml
At Jane Street, we enjoy using OCaml for lots of different things, from FPGA designs to web development. When it comes to Machine Learning, Python is one of the most commonly used languages. Machine learning frameworks such as TensorFlow or PyTorch wrap some highly efficient C++ and GPU implementations of tensor operations in easy to use Python apis. These frameworks also provide automatic differentiation functionalities which are commonly used to train deep learning models.
In this talk we will see how we can leverage TensorFlow or PyTorch directly from OCaml so that we can use our favorite programming language to build deep learning models and train them on GPUs. We will consider the Reinforcement Learning setting where an agent is trained to play Atari video games such as Space Invaders or Breakout. Our agents will be based on the Deep Q-Learning approach introduced in 2014.
Laurent first joined Jane Street as a developer in the London office back in 2013 working on trading systems. After a short stint at DeepMind in 2017/2018, he is now back at Jane Street as a researcher working on the equities desk in London. Laurent holds a PhD in theoretical computer science from Institut National Polytechnique de Grenoble.