EXT_window_rectangles extends OpenGL with a new per-fragment test called the "window rectangles test" for use with FBOs that provides 8 or more inclusive or exclusive rectangles for rasterized fragments. Applications of this functionality include web browsers and virtual reality.

1. EXT_window_rectangles
Mark Kilgard
NVIDIA
June 9, 2016

2. What is EXT_window_rectangles?
• Provides new per-fragment pipeline stage:
window rectangles tests
– Provide an array of window-space rectangles
• Maintained as GL context state
– Rasterized fragments must be inclusive or exclusive
to the set of active window rectangles
– Limited to framebuffer objects (FBOs), not windows
– Fast because discards prior to shading & pixel
operations
• Specified as multi-vendor OpenGL extension

3. Standard OpenGL Per-fragment
Operations

4. NEW Window Rectangles Test in
Per-fragment Operations
Window
Rectangles
Test
NEW
stage

5. Rationale for
EXT_window_rectangles
• Often rendering is obscured by opaque regions
in window space
– Wasteful to render to obscured regions
– Discarding such rendering = easy performance
• Common applications
– Web page background often
obscured by overlapping
• Complication: modern web pages
use lots of rounded rectangle sections
– Windowed rendering in UI
– Virtual reality limited views

6. Rationale for FBO-only Behavior
• Window rectangles test hardware is used to aide
conventional window clip enforcement for
windows
– Called “pixel ownership test” in OpenGL specification
– Details of pixel ownership depend on underlying
window system’s clipping
• But FBO rendering is unclipped by window
system
– So window rectangles test functionality can be re-
tasked for FBO rendering under application control
• Reasonable because modern OpenGL rendering
is FBO-focused

7. All Rendering Subject to Window
Rectangles Test
• Affects
– point/line/polygon rasterizaton
– framebuffer clears (glClear, etc.)
– framebuffer blits (glBlitFramebuffer, etc.)
– image rectangles (glDrawPixels, glCopyPixels,
glDrawTextureNV)
– bitmaps (glBitmap)
• Means regions excluded or not inclusive simply
have no effort spent on shading or pixel
operations
– Excellent at avoiding wasted rasterization, shading,
and pixel update bandwidth & effort

8. Straightforward API
• glWindowRectanglesEXT(GLenum mode, GLsizei count,
const GLint rects[]);
– mode can be either GL_INCLUSIVE_EXT or
GL_EXCLUSIVE_EXT
– count can be from 0 to maximum number of supported window
rectangles
• Must be at least 8
– Rectangles allowed to overlap and/or disjoint
• Each rectangle is (x,y,width,height)
• width & height must be non-negative
• Initial state
– GL_EXCLUSIVE_NV with zero rectangles
– Excluding rendering from zero rectangles means nothing is
discarded by window rectangles test

9. API Queries
• State can be queried
– GL_WINDOW_RECTANGLE_MODE_EXT
• Use glGetIntegerv to query mode
– Either inclusive or exclusive
– GL_WINDOW_RECTANGLE_EXT
• Use indexed glGetIntegeri_v to query (x,y,width,height) of a
window rectangle by index
– GL_NUM_WINDOW_RECTANGLES_EXT
• Use glGetIntegerv to get number of active window rectangles
• Implementation-dependent limit query
– GL_MAX_WINDOW_RECTANGLES_EXT
• Number of window rectangles supported, at least 8

10. Window Rectangles Test
Pseudo-Code
• Given a fragment at window-space position (x,y)
• And winrect state structure with EXT_window_rectangles state
• Determine if this fragment should be discarded or not
const bool inclusive = (winrects.mode == GL_INCLUSIVE_EXT);
bool in_some_rect = false;
// For each window rectangle...
for (int r = 0; r < winrects.count; r++) {
const WinRect &rect = winrects.rect[r];
const int box_x0 = rect.x;
const int box_y0 = rect.y;
const int box_x1 = int(rect.x + rect.width);
const int box_y1 = int(rect.y + rect.height);
// Is this fragment [i,j] within this rectangle r?
if ((x >= box_x0) && (y >= box_y0) && (x < box_x1) && (y < box_y1)) {
in_some_rect = true;
}
}
if (inclusive ^ in_some_rect) {
// INCLUSIVE: Fragment not in any inclusive rectangle; discard.
// EXCLUSIVE: Fragment in one or more exclusive rectangles; discard.
discard;
} else {
// INCLUSIVE: Fragment included in at least one rectangle; accept.
// EXCLUSIVE: Fragment not within any exclusive rectangles; accept.
}

11. Rasterization Performance Benefits
• Being able to throwing away useless work quickly is
always a boon to performance
• Software rasterization details
– Scan-line rasterizers can quick “skip over” window rectangles
along scan-lines
– Happens before expensive shading & pixel operations
– MMX friendly
• Hardware rasterization details
– Today, GPUs can enforce rectangular windows
• Details vary but rectangular region discard commonly supported
• Example: OpenVG exposes at least 32 window rectangles
– Very fast discard since done in rasterizer and requires no reads
to video memory (e.g. stencil buffer reads)
– EXT_window_rectangles allows this same hardware to be re-
tasked for application use

12. Skia Support
• Skia has patches to support this
functionality
– Implemented by Chris Dalton
• For acceleration of Google’s Chrome
browser
– <div> sections in web pages provide
exploitable knowledge of opaque layered
occlusion
– Rounded rectangles can be approximated by
two overlapped rectangles

13. Skia Performance Gains
• Not drawing to obscured web page regions helps a lot
• Skia’s Nanobench running captured web page scenes at
1800x2560
4 sample 8 sample 16 sample
scene improvement improvement improvement
keymobi_wordpress.skp_1 143.3% 301.8% 606.6%
keymobi_blogger.skp_1 137.2% 341.2% 566.7%
keymobi_digg_com.skp_1 125.3% 308.0% 621.9%
keymobi_shop_mobileweb_ebay_com.skp_1 119.7% 289.3% 593.2%
keymobi_pinterest.skp_1 105.6% 272.0% 571.9%
desk_wikipedia.skp_1 103.6% 224.7% 490.1%
desk_jsfiddlebigcar.skp_1 102.3% 190.0% 231.7%
top25desk_wordpress.skp_1 102.2% 271.0% 564.8%
keymobi_linkedin.skp_1 97.2% 264.3% 525.6%
desk_pokemonwiki.skp_1 89.6% 254.3% 457.3%
keymobi_boingboing_net.skp_1 89.4% 182.1% 416.7%
keymobi_androidpolice_com_2012_.skp_1 77.0% 177.1% 189.0%
keymobi_bing_com_search_q_sloth.skp_1 63.9% 168.8% 354.3%
keymobi_amazon_com_gp_aw_s_ref_.skp_1 60.5% 135.6% 286.4%
keymobi_cnn_com.skp_1 59.8% 133.2% 299.4%
keymobi_ftw_usatoday_com_2014_0.skp_1 53.9% 68.3% 75.5%
top25desk_google_com_search_q_c.skp_1 49.3% 170.3% 401.5%
keymobi_gsp_ro.skp_1 46.0% 180.2% 376.8%
keymobi_online_wsj_com_home_pag.skp_1 39.9% 107.6% 309.9%

14. Example Scene
• Scenario: draw the green background to be overlapped
by 5 irregular regions
Waste of rendering
to rasterize green
into regions to be
subsequently
overdrawn

15. Window Rectangles to the Rescue
• Naively, just want to draw a big rectangles
• Use all 8 window rectangles to restrict rendering
– Using exclusive mode, GL_EXCLUSIVE_NV

16. Window Rectangles to the Rescue
• Checkerboard regions show discarded rendering that
gets excluded by GL_EXCLUSIVE_NV window
rectangles test

17. Visualization
• Rounded rectangles & non-pixel aligned rectangles still
need additional discarding
Shader discarded
regions shown in
gray checkerboard
Window rectangles
test regions shown
in orange/yellow
hash

18. Usage of Window Rectangles Test
• Shows rectangle drawn in 8 inclusive and 8 exclusive
situations
– Notice how window rectangles allowed to overlap

19. Real-world Web Pages that Benefit
89% improvement @ 4x 46% improvement @ 4x 59% improvement @ 4x

20. Consider a Standard Rectangular
Viewport
• Normal rendering

21. Restricted to a Non-rectangular
View
• Limit rasterization to approximately circular
region cheaply with window rectangles test

22. Restricted to a Non-rectangular
View
• Visualizing the 8 inclusive rectangles

23. Alternatively Exclusive
• Visualizing the 8 inclusive rectangles
– Overlap of two sets of window rectangles means exclusive &
inclusive scenes have overlap of pixels

24. Alternatively Exclusive
• Possible to render unfocused area with less
detail using exclusive window rectangles test

25. Specification, Driver, & Library
Status
• Specification public
– https://www.opengl.org/registry/specs/EXT/window_rectangles.txt
– Identical API for OpenGL 4.5, ES 2.0, and ES 3.2
• Patches in progress for Google’s Skia library
– 2D graphics library used by Chrome browser
• NVIDIA OpenGL drivers
– Fermi, Kepler, Maxwell, and Pascal
– So GeForce 4xx and up
– In drivers mid-2016
• Mesa3D
– Considered by Brian Paul

26. Thanks
• Brian Paul, VMware
• Brian Salomon, Google
• Chris Dalton, Google
• Jeff Bolz, NVIDIA
• Mark Callow, Khronos
• Walt Steiner, NVIDIA