Integrating GLPbuffer and Java Graphics2D

Advertisements

This article discusses the incorporation of OpenGL into an ongoing mathematics visualization project, 3D-XplorMath-J. 3D-XplorMath is a program to visualize mathematical objects and allow the user to interact with them, created for educational and research purposes. The object is to have a "virtual math museum," organized into "galleries" and "exhibits." The project was begun by Richard S. Palais as a program for Macintosh computers; it was written in the Pascal programming language, with contributions from other mathematicians.

A complete rewrite of the program in Java was begun in 2005, with primary design work by David J. Eck. The Java version is known as 3D-XplorMath-J. It is available as a standalone program and as a collection of applets on the website. The project is partially supported by The National Science Foundation (DUE Award #0514781).

Design Issues With OpenGL Integration

3D-XplorMath-J was initially designed to use Java Graphics2D, and so our goal was to find the simplest way to integrate OpenGL graphics. 3D-XplorMath-J uses renderers defined by an interface, Renderer3D, to draw its images. We created an additional renderer class to allow the program to perform OpenGL rendering in addition to the already existing Java Graphics2D rendering. The new renderer would set up the OpenGL lighting, projection, and viewing transformation and then render and draw using OpenGL commands.

OpenGL offers hardware-accelerated 3D drawing, with the possibility of improved graphics and a large speed-up compared to implementing 3D drawing with Graphics2D. OpenGL is implemented in Java using the JOGL API. The images in Figure 1 gives a visual impression of the improved graphics that we achieved by use of JOGL. From the top two images, you can see that JOGL gives a cleaner representation of the Snail Shell than Java Graphics2D does. However, this isn't as noticeable in less complicated objects. For objects that are being rotated, the difference becomes noticeable even with the Monkey Saddle, which can be seen from the bottom images.

Figure 1. Graphics for comparison. Top Left: Snail Shell at rest, drawn using JOGL. Top Right: Snail Shell at rest, drawn using Java Graphics2D. Bottom Left: Monkey Saddle in motion, drawn using JOGL. Bottom Right: Monkey Saddle in motion, drawn using Java Graphics2D.

There are three classes in JOGL that represent OpenGL "drawables," meaning places where 3D images can be drawn. They are GLCanvas, GLJPanel, and GLPbuffer. Of these, GLCanvas and GLJPanel are the two commonly used classes, but in projects of great size and complexity, it may be necessary to change large amounts of code before integration can be successful. GLPbuffer is the least known of the three, yet it possesses unique features that should cause Java programmers to take notice. In our case, using a pbuffer to render objects appeared to be simplest because it allowed us to use the same framework as the Java Graphics2D, just modified slightly to draw to a pbuffer (which stores pictures offscreen in memory). Such a picture can then be retrieved as a standard BufferedImage, which can be copied to the screen.

GLCanvas and GLJPanel are GUI components. GLCanvas is more compatible with AWT, GLJPanel with Swing. Drawing in a GLCanvas is a little faster, but it is a "heavyweight" component that is more difficult to use in Swing. These issues are discussed in articles by Amy Fowler and Chris Campbell. Using these classes in 3D-XplorMath-J would require replacing a JPanel with a GLCanvas or GLJPanel. Unfortunately, the JPanel is used for more than 3D drawing. It is used, for example, to display BufferedImages, and this is something that is not easy with a GLCanvas or GLJPanel. Note: This is true in Java 5.0 at least. At the end of the article we will briefly discuss some changes in Java 6.0.

A GLPbuffer, on the other hand, can be used without disrupting the Graphics2D drawing framework of the program. A GLPbuffer represents a region in memory that can be used as a drawing surface for OpenGL commands, much like a BufferedImage for Graphics2D. Once that is done, it is possible to obtain a copy of the image that was drawn in the form of a BufferedImage. The BufferedImage can then be copied to the screen using standard Graphics2D techniques. In this article, we explain how to do this, and we discuss the performance implications of doing so. Our greatest concern in using the pbuffer was the fact that it is still an experimental aspect of JOGL and therefore support on different platforms is hit or miss. The slow-down that could take place from drawing to a buffer rather than directly to a GLJPanel or GLCanvas was also a concern. After some testing in Linux, Mac, and Windows environments it was determined that the difference between the three was small enough to not deter pbuffer usage. Running the same animation, drawing 500 frames with 500 random spheres in each frame, on each setup resulted in these speeds:

Test Results in a Linux Environment (2GHz Intel 2 Duo, with Kubuntu Linux 8.04)

Class	Estimated Time of Full Animation	Estimated Time Per Frame
GLCanvas	39285ms=39.285sec	78.57ms=.07857sec
GLJPanel	42325ms=42.325sec	84.65ms=.08465sec
GLPbuffer	50945ms=50.945sec	101.89ms=.10189sec

Test Results in a Mac Environment (2GHz Intel Core 2 Duo, with Mac OS 10.5.)

Class	Estimated Time of Full Animation	Estimated Time Per Frame
GLCanvas	48795ms=48.795sec	97.59ms=.09759sec
GLJPanel	53161ms=53.161sec	106.322ms=.106322sec
GLPbuffer	82221ms=82.221sec	164.442ms=.164442sec

Test Results in a Windows Environment (Intel T2500 2.0 GHz Core Duo, with WindowsXP SP3)

Class	Estimated Time of Full Animation	Estimated Time Per Frame
GLCanvas	50813ms=50.813sec	101.626ms=.101626sec
GLJPanel	67704ms=67.704sec	135.408ms=.135.408sec
GLPbuffer	70703ms = 70.703sec	141.406ms=.141406sec

These tables show that the time penalty for using GLPbuffer is not huge, although it is more noticeable on Mac than on other platforms. So we decided to attempt integration using pbuffers and take a chance on its experimental nature. Code for the programs used to produce these tables can be found in the sample code download. Also included is an interactive program that allows you to compare the two techniques interactively.

The Pbuffer

Since we can get a BufferedImage from a pbuffer, then integration with the already existing math visualization framework was relatively simple. All we had to do was set up an OpenGL renderer to draw to the pbuffer, get the buffered image from the buffer, and then copy the BufferedImage to the screen. Here we explain how to do this.

To create the pbuffer, we used the following code, which our OpenGL renderer calls before drawing the picture (with the width and height being that of the window).

if (buf == null || bufw != width || bufh != height){     if (buf != null) {   // clean the old buffer       context.destroy();   //context is type GLContext       buf.destroy();    // buf is type GLPbuffer    }    GLDawableFactory fac = GLDrawableFactory.getFactory();    GLCapabilities glCap = new GLCapabilities();    // Without line below, there is an error on Windows.    glCap.setDoubleBuffered(false);    //makes a new buffer    buf = fac.createGLPbuffer(glCap, null, width, height, null);    //save size for later use in getting image    bufw = width;    bufh = height;    //required for drawing to the buffer    context =  buf.createContext(null);   }  

A context is an abstraction that is necessary for rendering to any OpenGL drawable. The JOGL event-driven drawing framework handles contexts automatically, but they are required for drawing outside that framework, which is the case with our pbuffer. Manually controlling a context involves getting a context from the drawable object and making it current when rendering to it.

We also discovered a buffer clean-up check was needed, as follows:

if (buf != null) {    context.destroy();    buf.destroy();  }  

We found that this is necessary to prevent slowdowns and crashes from the creation of too many pbuffers. The destroy() method of each frees up resources being held by the context and the pbuffer. This is apparently not done automatically in JOGL.

Another check that should also be made at the beginning of the program is to make sure that a GLPbuffer can be created:

if(!GLDrawableFactory.getFactory().canCreateGLPbuffer())    //Disable the using of OpenGL or at least by way of a Pbuffer  

This check is necessary because a pbuffer isn't necessarily supported on all computers, since it uses the hardware to accelerate rendering. The check looks to see if the graphics card on the computer supports the pbuffer but from our testing on our different lab machines, we have found that trying to create a GLPbuffer sometimes prduces an error even though this method returns true. Knowing this, it might be better to just try to create the pbuffer and catch any exception that occurs.

Now, to render to the pbuffer, we take the context created for the buffer and make it current. This assures that the OpenGL commands that follow will draw to the pbuffer and not elsewhere. In our OpenGL renderer, we call this at the start of the drawing method of the renderer:

context.makeCurrent();  

The next task is to is to get a GL object to draw with. This is done directly using the pbuffer and the rest follows the normal JOGL syntax.

GL gl = buf.getGL();  

Getting an image from the pbuffer is done by use of the JOGL class Screenshot, which is used for taking screen shots of OpenGL applications. In spite of the name, it only returns an image of the current OpenGL drawable, not of the entire screen. By calling Screenshot.readToBufferedImage(bufw,bufh), we take a screen shot of the current GL Drawable (our pbuffer) as a buffered image. Also note that the context for our pbuffer must be current. Now that we have the buffered image, we then can use Java Graphics2Dto draw the image to the screen.

context.makeCurrent();  BufferedImage img = Screenshot.readToBufferedImage(bufw,bufh);           context.release();  g.drawImage(img,0,0,null);  

Since the design of 3D-XplorMath-J was meant for use with Java Graphics2D, this is the most beneficial part of using a pbuffer in the project. Since we're able to get a buffered image from the pbuffer, no code changes were needed outside of the renderer except for creating an OpenGL renderer to do the 3D drawing instead of a Graphics2D renderer.

Speed Concerns and Some Solutions

Since mathematical objects can be complicated, rendering these objects can become quite slow, especially when rotating a surface and having it render in real time. Since being able to rotate and look at mathematical objects in as many ways as possible is largely a goal of the 3D-XplorMath project, we needed a way to increase the efficiency of real-time rendering of the math objects. Part of the rendering inefficiency could be attributed to the use of a pbuffer, but the drawing was still too slow given the capabilities of OpenGL. To start improving render time, we began looking at OpenGL display lists and how they are used and what kind of speed up we might achieve with them.

Display lists allow the storage of OpenGL commands in an optimized form for later execution, which is largely useful when used to store geometry or state changes that will be used multiple times, such as in the rotation of a mathematical object. This is useful in this case, since the same display list can be called each time the rotating object is drawn. While this is all well and good for code organization, does it actually give us the speed boost we want?

A display list is identified by a unique integer, which is created by using the gl.glGenList() command where gl is an object of type GL. Next we create a new list using the list identifier and the desired mode.

displayListID = gl.glGenLists(1); // type int   gl.glNewList(displayListID, GL.GL_COMPILE);  

The OpenGL commands that follow glNewList are added to the list, and when the commands to be stored in the list are complete, we simply end the list using

gl.glEndList();  

Finally, when we want to execute the cached code in the display list we just call the list using its identifier.

gl.glCallList(displayListID);  

After learning how to use a display list, we decided that we should write a test to see if it actually increases efficiency before we worked out how to add it to the 3D OpenGL renderer. Taking our test prgram that draws 500 spheres in 500 frames from before, we implemented the same test with the addition of a display list to store the commands to draw a sphere, and we called the list to draw each sphere to the screen. What we found was a surprisingly huge speed-up.

The simple sphere drawing program using just a pbuffer and no display list for drawing 500 different frames of 500 spheres ran in about 50 seconds in a Linux environment. The same program implemented using a display list ran in an impressive 12 seconds, making the decision to incorporate this into the project an easy one.

Adding display lists to 3D-XplorMath-J was a relatively simple venture. The idea is that code for rendering the object is put into a display list and when the renderer needs to draw to the screen, it does so by calling the display list. This means that when the object just needs to be transformed, such as in rotating the object, we can just apply the transformations to the display list rather than re-rendering the object with its new orientation.

Use of a display list gave us a huge speed increase for rotation of surfaces, but the original production of the display list was still rather slow. We discovered that this was because we were drawing individual polygons using glBegin(GL_POLYGON). We got a speed up by using glBegin(GL_QUAD_STRIP) to draw our surfaces. A GL_QUAD_STRIP is a geometric primitive that is a linked strip of quadrilaterals defined by two vertexes at a time. This slight change turned out to be a significant speed improvement in comparison to drawing the individual polygons.

When we look at the speedup achieved by implementing these two techniques to the project, the penalty for using a pbuffer becomes small in comparison. We now have a renderer that performs real-time rendering well for even fairly complicated geometric objects.

Quick Look at Java 6 and OpenGL

In Java SE 6, JOGL and Java 2D can be used together directly. Java 6 allows Swing objects to now overlay OpenGL rendering. Now rendering of 3D OpenGL graphics can be done directly onto Java 2D graphics, and similarly, Java 2D graphics can be drawn directly onto 3D OpenGL graphics. For now, the 3D-XplorMath-J project will continue to use Java 5, but the future switch to Java 6 gives many design options for integration of OpenGL rendering that are more reliable and more efficient. Java 6 makes using either a GLJPanel or a GLCanvas fairly painless, which implies a small speedup from no longer having to get buffered images from a pbuffer. Eliminating the use of the pbuffer would avoid the complications with not being able to create pbuffers on some computers, making the OpenGL rendering version available to more people. Java 6 is definitely a great advancement for graphics and the Java programming language and should be looked into for the use of OpenGL with Java. But for programs that still need Java 5, a pbuffer is one option for using OpenGL and Java 2D until the switch is made.