Cooperating with Machines.

Since Alan Turing envisioned Artificial Intelligence (AI) [1], a majordriving force behind technical progress has been competition with humancognition. Historical milestones have been frequently associated with computersmatching or outperforming humans in difficult cognitive tasks (e.g. facerecognition [2], personality classification [3], driving cars [4], or playingvideo games [5]), or defeating humans in strategic zero-sum encounters (e.g.Chess [6], Checkers [7], Jeopardy! [8], Poker [9], or Go [10]). In contrast,less attention has been given to developing autonomous machines that establishmutually cooperative relationships with people who may not share the machine'spreferences. A main challenge has been that human cooperation does not requiresheer computational power, but rather relies on intuition [11], cultural norms[12], emotions and signals [13, 14, 15, 16], and pre-evolved dispositionstoward cooperation [17], common-sense mechanisms that are difficult to encodein machines for arbitrary contexts. Here, we combine a state-of-the-artmachine-learning algorithm with novel mechanisms for generating and acting onsignals to produce a new learning algorithm that cooperates with people andother machines at levels that rival human cooperation in a variety oftwo-player repeated stochastic games. This is the first general-purposealgorithm that is capable, given a description of a previously unseen gameenvironment, of learning to cooperate with people within short timescales inscenarios previously unanticipated by algorithm designers. This is achievedwithout complex opponent modeling or higher-order theories of mind, thusshowing that flexible, fast, and general human-machine cooperation iscomputationally achievable using a non-trivial, but ultimately simple, set ofalgorithmic mechanisms.