Subject: An enhanced OpenGL renderer for Unreal Tournament
From: <Lagret av Microsoft Internet Explorer 5>
Date: Fri, 15 Jul 2005 10:19:33 +0200

An enhanced OpenGL renderer for
Unreal Tournament

Last updated July 11, 2005

I added some enhancements to the OpenGL Renderer for Unreal Tournament. A
Win32/x86 binary and the source code are available on this page.

The settings page
documents some of the options.

Latest news
I built an updated D3D8 renderer. It has the
experimental ZRangeHack option, which is designed to avoid problems with
flickering decals in the distance. Like with the OpenGL renderer, it would have
been better if this issue were dealt with in higher level engine code since it
could be done more efficiently and with less risk of side effects. ZRangeHack
had to be implemented a little differently in this renderer. I ran some tests,
but there may still be some risk of it failing in certain cases. There are also
various other mostly minor updates in this build. The file is utd3d8r11.zip.

7-3-2005
Version
3.1 is released. I added a function that uses SSE for buffering fogged actor
vertices. There's a fix for a minor detail that probably never breaks anything,
but is still good to take care of anyway. Some very minor optimizations were
added in a few places.

6-12-2005

Rune renderer
I
built a new renderer for Rune. It's the 3.0 code built to work with Rune. It
also contains minor additional optimizations to the Rune specific fog code. The
file is runeglr10.zip
(works with Rune version 1.07).

Deus Ex renderer
I built a new
renderer for Deus Ex. It's the 3.0 code built to work with Deus Ex. The file is
dxglr14.zip
(works with Deus Ex version 1112fm). OneXBlending is enabled by default in this
renderer, but if the brightness looks off, in addition to GammaOffset
adjustments, also make sure OneXBlending=True in the
[OpenGLDrv.OpenGLRenderDevice] section of your DeusEx.ini file. A new option
called SceneNodeHack was added (in the previous version). Enabling this may work
around some minor problems, though it wasn't tested extensively, so there's a
chance it might cause other problems.

Old files
I removed a few
old files. This includes old versions of the Deus Ex renderer dxglr10.zip and
dxglr11.zip, and the one renderer u1glr10.zip that I built for Unreal Gold. For
Unreal Gold, and other versions of Unreal, use the newer renderer from
OldUnreal.

6-4-2005
Version 3.0 is released. It adds a few new
major features.

Single pass fog
Support for single pass fog
mode is now present in the OpenGL renderer. It requires at least 3 texture units
and support for the GL_ATI_texture_env_combine3 extension or the
GL_NV_texture_env_combine4 extension. The bad news is that if you don't have an
ATI or NVIDIA video card, you probably can't use single pass fog mode since
there is no standard extension that provides access to the required texture
blending mode (for fixed function), and it looks like other vendors decided not
to support either of these extensions. The SinglePassFog option enables single
pass fog mode if supported.

DetailMax
Support for an extra
detail texture layer was added. This feature is already present in some OpenGL
renderers from Oldunreal and some Direct3D renderers. The extra detail texture
layer, which adds even more detail at short distances, can be enabled by
settings DetailMax to 2. If set to 0 or 1, standard one layer detail texturing
will be used if detail textures are enabled. I limited DetailMax to 2 because my
testing showed that the third layer only starts to blend in when right up
against a wall, at which point it makes almost no difference. Also, the second
detail texture layer will not show up unless SinglePassDetail is
disabled.

Masked S3TC texture support
Masked S3TC textures are
given an RGBA format that supports single bit masking instead of an RGB format
that does not like in previous renderers. This is enabled by default and
hopefully does not break any existing uses of textures. The new debug option
NoMaskedS3TC can be used to disable this feature (though it really should be
considered a fix for a bug that's always been there). If creating new masked
S3TC textures, this option can also be used to see how they would look with an
older renderer that does not support masked S3TC textures. Also note that this
change most likely will not fix existing problems with black areas in older
original S3TC skybox textures since all of these do not appear to have been
encoded with proper masking information.

ZRangeHack
I activated
the experimental ZRangeHack code, fixed a few things, and adjusted it a little
bit for safety. There's still some risk, so no guarantee it works in all cases.
This hack fix isn't enough to make a 16-bit z-buffer work. It can however fix
the problem with decals flickering in the distance with 24-bit z-buffers. It
will also fix the issue with the Redeemer covering up part of the HUD. It may
have the same effect on other similar weapon models used in various mods, though
I didn't test any of these potential cases, so I can't say for sure, or if it
might cause problems in these cases. Set ZRangeHack to True to enable this
experimental option.

Notes and other minor changes
The single
pass fog support was difficult to add for the OpenGL renderer. NVIDIA, ATI, and
others involved with OpenGL in this area really made a mess out of things here.
NVIDIA has one extension, GL_NV_texture_env_combine4, that provides access to
great functionality that should have always been the standard at this level, but
does a lot of bad things in the way the extension was implemented. The ATI
extension, GL_ATI_texture_env_combine3, doesn't seem too bad, though there are
still things I don't like about this and how tex env stuff in general ended up
in OpenGL. Unfortunately, no other vendors appear to support the simpler ATI
extension that provides minor additional functionality, which their hardware
most likely supports. So, for myself and other developers interested in using
this functionality, we get to write and test two different code paths to support
NVIDIA and ATI, and end up without access to certain features on other video
cards unless using fragment programs or Direct3D. I pulled some previous
optimization to implement single pass fog mode, though the overall effect should
be very minor. This could be fixed, but it's even more extra work, including
testing the updates on two different video cards.

Enabling ZRangeHack
will slow things down a little bit, but the difference will probably still be
very small. It would be better if the issues this hack fix addresses could be
dealt with in higher level code, but it has been a while, and with not much else
to add, I decided to try adding this one, mainly to get rid of the flickering
decals with 24-bit z-buffers that will be the maximum available on most video
cards for quite a while it seems.

I added some minor optimizations and
one or two minor fixes. There's some new inline assembly for the various gouraud
polygon buffering functions. It will help fogged triangles on all systems, the
colored triangles path on systems without SSE support, and the buffer additional
triangles path on all systems. I also ran some benchmarks on the new code that
buffers 3 vertices on my P4 2.8C. For colored, execution time is reduced by
7.5%, and it's 15% for fogged. The existing SSE colored function reduces
execution time by an additional 25% compared to the new colored function with
the improved inline assembly. But, the architecture of the renderer that I can't
change isn't really designed for moving lots of polygons quickly, so due to high
overhead, these changes will make very little difference
overall.

5-22-2005
I built an OpenGL renderer for Rune, which
is based on the 2.9 code. Consider it a beta since I didn't run extensive tests
on all areas of it. There is a chance that some things might have been missed
since I didn't merge all modifications into this one. It's difficult to tell if
all of the modifications are necessary, though I'm fairly sure that some are old
stuff that isn't used. The file is runeglr09.zip.
If you notice anything that might be broken, make sure it isn't the same in the
original OpenGL renderer. Although it still could be a bug, if it's the same in
the original OpenGL renderer, it may not be a bug that I can
fix.

5-15-2005
I ran a few tests on the D3D8 renderer built for
D3D9, which only required minor modifications. With D3D9, V Sync in a window
control is available and it has access to a more rational z-bias implementation.
It also tends to run slightly slower.

5-7-2005
I built a new
version of the D3D8 renderer. This one is a bit faster with interleaved
vertex/color data, larger vertex buffers, and the BufferTileQuads code added.
BufferTileQuads is enabled by default in this renderer since not having it hurts
D3D a lot more than OpenGL, and I don't have to be concerned about any backwards
compatibility issues. I also added a few more features and some minor
optimizations. The file is utd3d8r10.zip.

I
didn't add paletted texture support to this renderer, so if you have a
GeForce1-4 series video card, you should make sure to use the OpenGL renderer
and enable the settings that tell it to use paletted textures (these are
disabled by default). Also, on other video cards with good enough OpenGL driver
support, the OpenGL renderer may be better.

Performance differences
between this renderer and the OpenGL renderer are fairly small on my system,
though it does tend to be a little slower. It may be possible to improve this in
some cases by interleaving the texture arrays, but this is a lot of extra work,
so I may not try it. It doesn't help that D3D seems to have poor small batch
performance in general due to intrinsic design/implementation characteristics.
There's no avoiding this after a certain point since UT has fairly low geometric
complexity.

So, D3D is far simpler compared to OpenGL in the feature set
it supports on the API side and yet ends up with far worse small batch
performance. In various places in the renderer, it's possible to get moderate
performance with a minor amount of work using OpenGL, but with D3D, it requires
extra work just to make it work at all and end up with poor performance. With
either API, it's possible to get higher performance by adding more advanced
buffering schemes such as actor triangle buffering, clipped actor triangle
buffering, BufferTileQuads, etc. This D3D renderer will be far slower than the
OpenGL renderer for line drawing since it lacks advanced buffering in this area.
This shouldn't be a problem with the editor because I don't support it with this
renderer anyway since selection support is not implemented. Hopefully line
drawing isn't used too heavily, or at all, outside the
editor.

z-buffer issues
Like the OpenGL renderer, this D3D
renderer may have problems with far away decals flickering due to z-buffer
precision issues if only a 24-bit z-buffer is available. It doesn't support
w-buffering either, though it looks like a lot of newer video cards don't
support this feature anyway. It's probably possible to work around this problem
in the renderer, though it may not be anything I'll add. Of course if all these
new GPUs/VPUs didn't drop support for 32-bit z-buffers, this wouldn't be a
problem.

4-26-2005
I finally decided to learn Direct3D in case
knowing it would be good for a future job. Porting the renderer only added a few
days, with a lot of that time spent dealing with things D3D makes difficult, so
I tried building one that uses D3D. D3D has gotten better in recent versions,
but some areas are still problematic. I'm sure glad I never used D3D7 or
earlier.

This renderer will most likely be slower than the OpenGL one on
ATI, NVIDIA, or other graphics cards that at least have reasonably good OpenGL
drivers. I also left out a few likely significant optimizations in the current
build that may limit its performance. I guess I'll find out later if fixing
these can bring it up to the speed of the OpenGL renderer on my system. It uses
D3D8 and since it uses certain advanced features, it will not function on
various older video cards. Also, due to certain SDK complications, I think it
ends up requiring at least DirectX 8.1, which I believe means it will not
support Win95. You can download the initial beta renderer here. It
comes with a .int file that also goes in the UnrealTournament\System directory.
To use it, change video drivers in UT, select show all devices, and then select
the new entry that says Experimental Direct3D8 Support. Remember, it probably
isn't worth trying this beta build if the OpenGL renderer runs okay on your
system.

I added single pass fog mode to this one, since it happened to be
easy with D3D. The required blend mode on the OpenGL side requires one extension
for NVIDIA, another extension for ATI, and probably just isn't there for various
other video cards since providing a standard way to access it on the fixed
function side seems to have been forgotten about. It's too bad some of the other
vendors didn't at least add support for the ATI version of the extension since
it doesn't really add much and their hardware probably supports it all. I'll
check the standard extensions again sometime, but I don't think the
functionality required for single pass fog in UT is there.

I'm checking a
large number of caps bits/values in this build, but a few checks are still
missing. I'll probably fix a few of these later, but may leave a few of the more
complicated ones out.

Windowed mode, windowed mode resizing, and
surviving through various mode switches should work, but some things in this
area get awfully difficult to support and test when using D3D. Windowed mode
screen shots hopefully work okay, including without crashing in various special
cases when the window isn't fully within the screen. D3D still makes something
basic like grabbing a copy of what got rendered far too difficult in cases like
this.

This initial build of this renderer supports a large number of
features, but some are missing at this time.
- Selection support for UnrealEd
isn't there. I may never add it, so don't use it with the editor (other
functionality should work, but it's not really usable there without this
feature).
- S3TC support is there.
- 16 bit texture support is there, but
I did the conversions using simple clipping instead of proper rounding.
- Not
checking texture aspect ratio restrictions yet, so if any specific requirements
here, it may just crash when trying to load certain textures (good chance this
may not be an issue on any new enough cards to run this renderer though).
- V
Sync on or off request only works full screen. D3D8 doesn't allow something
basic like V Sync on or off to be requested when in windowed mode. I believe
this got fixed in D3D9.
- All the texture filtering modes and LOD bias should
work.
- No paletted texture support, and I'm not sure I'll ever add it to
this one.
- Lots of other features are supported, but a few others are
not.

4-20-2005
Version 2.9 is released. It has a new option
called BufferTileQuads, which should speed up text rendering. It was easy to add
and seems to work well in various cases. It's disabled by default because it's
new, but it should be fairly safe to enable. UseSSE was changed to a True/False
option and it now supports dynamic updates.

A few other changes are also
present:
- Improved alignment on main buffering arrays.
- Minor
optimizations in multipass detail texture code.
- Removed a few instructions
from 3 vertex gouraud polygon SSE fastpath.
- Minor optimizations in buffer
additional clipped vertices function.
- Fixed bug in buffer size remaining
calculation for clipped gouraud polygons. Note that this could not actually
cause a failure in version 2.8 because of other factors.
- Other minor
optimizations.
- Other minor changes.

3-16-2005
Various
things.

Broken TruForm support in the renderer
TruForm support
in the renderer is broken for a few reasons. Consider it an experimental and
incomplete feature right now. There is no easy fix for the problem with player
models where it doesn't look good when enabled. There are two other fixes that
could be made to other parts of the game engine code that could correct two
other outstanding issues.

Higher level rendering code clips actor
polygons to the edge of the screen. This destroys information contained in
normals needed to implement TruForm correctly. This will lead to polygons that
cross the edge of the screen having minor to potentially severe graphical
corruption. This is trivial to fix, but the code that does it isn't in the part
of the renderer that was open sourced, so it's nothing I can fix right now.
Also, with many video card/driver combinations, letting the driver or hardware
deal with clipping polygons that are partially clipped by the edge of the screen
should speed things up a little.

Once actor triangles make it to
DrawGouraudPolygon in the renderer, there's no good way to tell if they're from
a player actor that should have TruForm applied or some other actor that should
not have TruForm applied. It would probably be fairly easy for higher level
rendering code to use a new PolyFlags bit to tag triangles from objects that
should have TruForm applied if enabled. This could reliably eliminate problems
with weapons and other objects that look bad with TruForm applied. Note that the
TruFormMinVertices setting attempts to solve these problems, but cannot do so
reliably, and while it can fix some cases, it will also break
others.

Linux builds
I never seem to hear any good news about
attempts to build the updated renderer on Linux. Unfortunately, I can only
provide limited help in this area. I do try to keep the code cross platform
friendly, but it's unlikely that I will be able to attempt to build it on other
platforms anytime in the near future. I know the current code won't compile as
is with gcc, but I expect that only minor syntax fixups and basic non-essential
feature removal will be able to make the updates I added both compile cleanly
and work correctly.

The first major step in attempting to build the
updated renderer on Linux is to make sure you can build the original renderer
code before I added any updates to it. This will ensure that there are not any
major existing issues before going forward and attempting to use the new code.
If any problems are encountered in this stage, it's not really anything I can
help with much because I don't have a local build environment for this platform,
didn't write this code, etc. If there's some problem like maybe the ut432 header
files don't quite match the current Linux version of UT and cause problems
because of this, that falls into the I didn't break it and I can't fix it
category.

You'll need to use a compatible version of gcc. Unfortunately,
ABI changes mean you will almost certainly not be able to use a newer version of
gcc (unless the rest of the game were compiled with it of course).

You
can easily ifdef out the SSE code I added because whatever compiler you use
probably won't support it. This is not a major loss since the SSE code I added
only provides very minor speedups.

There's a chance there will be
problems with the sstream code I used for the debug stream when using an older
gcc and/or older libraries. Although it requires numerous changes to remove it,
this code is non-essential, and the changes should be simple.

There are
good reasons to try to get an updated renderer working on Linux if running UT
natively here. Besides just being very obsolete at this point, the original
OpenGL renderer code does contain a couple of more major design/implementation
issues that would be good to have fixed.

Other news (technical
details)
I may not release any updates to the renderer beyond utglr 2.8
anytime soon. It's getting difficult to find places that can be improved much at
all, can be easily improved, or can perhaps be significantly improved without a
lot of extra work.

For a long time now, a lot of the updated renderer
code is much more optimized than various similar bits of code in the rest of the
game engine. This leads to diminishing returns kicking in sooner when attempting
to apply various general optimizations. I've still found a few simple things and
maybe one or two more complicated things I might experiment with if I have the
time.

9-15-2004
I may have fixed the problem with UnrealEd not
restoring gamma on exit, but I still need to review the changes to make sure
they have little risk of causing new problems. It doesn't help that ATI's
drivers still do odd stuff with gamma correction. They're getting close to a
year of breaking things in this area to various degrees. There's something odd
about their installer for 4.9 too, as I had to temporarily rename my bin
directory to prevent it from failing.

8-15-2004
I don't think
there's any easy way to fix the 16-bit z-buffer problems without using a
w-buffer. I can sort of half fix it, but it's not really good enough to be of
much use, so I'll either leave the new code ifdef'ed out for a bit or just
delete it. W-buffers are supported through D3D on some cards, but I've never
seen them supported through OpenGL.

I found the cause of the excess code
padding in all of my renderer builds. Incremental linking was turned on for
release builds in the project file. Getting rid of this cut the uncompressed DLL
size by over 40 KB. Although it won't significantly speed things up, it'll still
save a little memory and perhaps increase code cache and TLB hit rates by a very
small amount.

Based on looking at a few other files, it looks like all of
436 and 451 on Win32 might have also been built with incremental linking
enabled. Although unfortunate, it's still one of those things where the
difference might be too small to show up in typical benchmarks by
itself.

5-3-2004
I built a new version of SetGamma that fixes
various minor problems. It's a simple command line utility program that adjusts
the hardware gamma ramp on the primary display adapter. A shortcut that sends it
the -reset option can be used to reset the hardware gamma ramp to 1.0 after a
crash that prevents it from being restored.

Some of the old news gets moved to the News
Archive page.

Notes
- Additional options are documented in the [New
options] section.

Things that could be added
-
Patching the ComputeOutcode function with some SSE code.

OpenGLDrv.dll for Win32/x86: utglr31.zip (48
KB)

OpenGLDrv.dll for Win32/x86: utglr30.zip (48
KB)
OpenGLDrv.dll for Win32/x86: utglr29.zip (48
KB)
OpenGLDrv.dll for Win32/x86: utglr28.zip (46
KB)
OpenGLDrv.dll for Win32/x86: utglr27.zip (46
KB)
OpenGLDrv.dll for Win32/x86: utglr26.zip (48
KB)
OpenGLDrv.dll for Win32/x86: utglr25.zip (50
KB)

Installation instructions
Go to your
UnrealTournament\System directory. Make a backup of your old OpenGLDrv.dll in
case the new one doesn't work. Then put the new OpenGLDrv.dll in your
UnrealTournament\System directory. This one contains a number of optimizations
that should improve performance over the base UT 4.36 OpenGL renderer. It also
contains a number of new options, which are described further down on this
page.

Source code: utglr31src.zip
(110 KB)

Source code: utglr30src.zip
(110 KB)
Source code: utglr29src.zip
(95 KB)
Source code: utglr28src.zip
(94 KB)
Source code: utglr27src.zip
(93 KB)
Source code: utglr26src.zip
(92 KB)
Source code: utglr25src.zip
(88 KB)

Notes about the source code
The source code has
been modified extensively. Although I did not try to break Linux support
completely, I did add some Windows specific code. Feel free to email me at [email-masked] if you need any
help getting it to build on Linux. Make sure to add the NO_UNICODE_OS_SUPPORT
define when building it on Win32.

The source code package only contains
.cpp and .h files from the OpenGL\Src subdirectory, which is where my changes
are. You will need to get the 432 headers from Epic to be able to build it. You
can download these from the Unreal Technology Downloads
page.

For version 1.2 and up, I had to remove the operator new and delete
overrides to make the new C++ debug functions work. I included a copy of the
modified UnFile.h with the proper ifdefs. I just have it pass things through to
malloc and free instead. I believe the problem may be with the overrides not
handling 0 byte allocations as malloc and new
do.

Feedback
Email: [email-masked]

New options
This enhanced UT OpenGL
renderer supports some new options. They go in the
[OpenGLDrv.OpenGLRenderDevice] section of your UnrealTournament.ini file. Most
options are documented on the settings
page.

Credits
I'd like to thank Epic Games for releasing the source code to
the UT OpenGL renderer, which made adding these updates to it
possible.

NitroGL for the original TruForm renderer modification. Initial
experimental TruForm code is based on these modifications.

Leonhard
Gruenschloss for help with implementing and testing additional TruForm related
updates and new Deus Ex specific code.

Copyright 2002-2005 Chris Dohnal