Zorua: Enhancing Programming Ease, Portability, and Performance in GPUs by Decoupling Programming Models from Resource Management.

The application resource specification--a static specification of severalparameters such as the number of threads and the scratchpad memory usage perthread block--forms a critical component of the existing GPU programmingmodels. This specification determines the performance of the application duringexecution because the corresponding on-chip hardware resources are allocatedand managed purely based on this specification. This tight coupling between thesoftware-provided resource specification and resource management in hardwareleads to significant challenges in programming ease, portability, andperformance, as we demonstrate in this work.

Our goal in this work is to reduce the dependence of performance on thesoftware-provided resource specification to simultaneously alleviate the abovechallenges. To this end, we introduce Zorua, a new resource virtualizationframework, that decouples the programmer-specified resource usage of a GPUapplication from the actual allocation in the on-chip hardware resources. Zoruaenables this decoupling by virtualizing each resource transparently to theprogrammer.

We demonstrate that by providing the illusion of more resources thanphysically available, Zorua offers several important benefits: (i) ProgrammingEase: Zorua eases the burden on the programmer to provide code that is tuned toefficiently utilize the physically available on-chip resources. (ii)Portability: Zorua alleviates the necessity of re-tuning an application'sresource usage when porting the application across GPU generations. (iii)Performance: By dynamically allocating resources and carefully oversubscribingthem when necessary, Zorua improves or retains the performance of applicationsthat are already highly tuned to best utilize the resources. The holisticvirtualization provided by Zorua has many other potential uses which wedescribe in this paper.