A 3D Game Theoretical Framework for the Evaluation of Unmanned Aircraft Systems Airspace Integration Concepts.

Predicting the outcomes of integrating Unmanned Aerial Systems (UAS) into theNational Airspace System (NAS) is a complex problem which is required to beaddressed by simulation studies before allowing the routine access of UAS intothe NAS. This paper focuses on providing a 3-dimensional (3D) simulationframework using a game theoretical methodology to evaluate integration conceptsusing scenarios where manned and unmanned air vehicles co-exist. In theproposed method, human pilot interactive decision making process isincorporated into airspace models which can fill the gap in the literaturewhere the pilot behavior is generally assumed to be known a priori. Theproposed human pilot behavior is modeled using dynamic level-k reasoningconcept and approximate reinforcement learning. The level-k reasoning conceptis a notion in game theory and is based on the assumption that humans havevarious levels of decision making. In the conventional "static" approach, eachagent makes assumptions about his or her opponents and chooses his or heractions accordingly. On the other hand, in the dynamic level-k reasoning,agents can update their beliefs about their opponents and revise their level-krule. In this study, Neural Fitted Q Iteration, which is an approximatereinforcement learning method, is used to model time-extended decisions ofpilots with 3D maneuvers. An analysis of UAS integration is conducted using anexample 3D scenario in the presence of manned aircraft and fully autonomous UASequipped with sense and avoid algorithms.