Thursday, March 15, 2012

Blur effect


What I said in my previous post about shaders in D is sadly not up-to-date, so I want to bring something new. Here is a little project testing
  • Derelict3
  • OpenGL 3+ API
  • Multipass post-processing effect via shaders and render-to-texture
  • D builder for SCons
  • SCons targets

Eventually, it results as a program blurring a restricted zone of an image.

Blurred cow on a sharp photo.

Get the project on GitHub.

Environment:

Derelict3

Derelict3 is the new version of Derelict, a project providing wrappers in D for C libraries. It is though an alpha version, so consider using Derelict2 if you really want to pacify your boss.

The hearth of this project is DerelictGL3, which wraps OpenGL up to the latest version 4.2. I'll take about OpenGL in the next section.

In order to use OpenGL, we have to create a window and its OpenGL context. Derelict3 provides two libraries: SDL2 or GLFW3. Both are new versions in development, so once again, the code may break up at any moment. As I used DerelictSDL last time, I tried to stay on this library. That was a bad idea: SDL2 is not able to open a OpenGL 3.2+ context for now. So I give DerelictGLFW3 a try. This time it works up to OpenGL 3.3, my graphics card doesn't seem to accept any higher version. Both libraries have somewhat similar concepts, so it was not a big deal.

I finally used DerelictIL, a wrapper of the image library DevIL, to load a picture as OpenGL texture. Loading is then really easy, it only consists of a function call. Again, this may not fit to production environments, as the textures should be compressed on some low-level format which are much more efficient. But for test purposes, it comes in handy.

The wrapper code is very lightweight, it's easy to dig into files to find functions' prototype or constants. It also seems easy to make a wrapper over any library, next time I'll try that. However I regret that the files contain no documentation, and even no name for the function parameters. As they wrap libraries in development, prototypes may change compared with the libraries' official documentation. Then don't be surprise if it doesn't match.

OpenGL 3+

I was playing with OpenGL at a time where shaders didn't exist. As new versions come, the API changes, sometimes radically. As such, glBegin and glEnd are now deprecated, view and projection matrices are now handled by shaders... well, let's learn the modern OpenGL from scratch. Sadly, a lot of resources on Internet still use old methods, or a mix of old an new ones...

I don't plan to deliver for the mass market, so I target version 3.3 (but I'll understand if you target a lower version). Shaders have also to declare GLSL version 330.

My goal was only to blur parts of an image, but this can be extended to simulate depth of field for example. This is entirely a post-processing effects, so it involves shaders. The basic idea for the effect is to get a blurry version of the image, and to mix it with the original, sharp version, with a blending factor depending on the position of the pixels (in my case, it is related to the mouse cursor position). This is obtained in one pass, but getting a blurry photo itself involves two passes (explanations), so we end up with three passes: horizontal blur, vertical blur, and mix.


When involving several passes, the output of a pass shall be the input of the next pass, except for the last pass which can render directly on the screen. This is known as Render to Texture (RTT for short), indeed the output of a pass is written to a texture, which in turn will feed the next pass. RTT requires a framebuffer defining the output textures. In this case, there is only a single output, the color of each pixel, but one may also define the depth buffer or render on several textures. I don't want to explain further, a lot of tutorials already exist.

Feeding a pass with a texture eventually means simply apply the texture on a quad covering the whole screen space. I used to draw quad with four vertices, but a friend told me a simple triangle large enough could cover the screen as well, even being better for the graphical pipeline (not verified). To render faces using the new API, we need to set a vertex buffer object, an UV buffer object and an index buffer object. On top of that, a vertex array object retains the states, so it will be easier to restore them in the rendering loop. Again, I don't explain in detail, lots of sites already do. The loop is really straightforward: bind a framebuffer, the correct texture, and a shader program, render the quad and voila the texture contains data for the next pass.

Shaders are not complicated. Vertex shaders don't do anything related to the post-processing effect in itself, everything is in the fragment shaders. Below is the code for the horizontal blur, inspired from an article on GameRendering.com. The shaders in that article have a little flaw: the sum of all coefficients is 0.98, resulting in a small but noticeable darkening. The following code fixes this problem by adding two new samples with coefficient 0.01.
#version 330

in vec2 texCoord;

out vec4 fragColor;

uniform sampler2D texture;
uniform vec2 resolution;

void main() {
    float blurSize = 1.0 / resolution.x;
    
    vec4 sum = vec4(0.0);
    sum += texture2D(texture, vec2(texCoord.x - 5.0 * blurSize, texCoord.y)) * 0.01;
    sum += texture2D(texture, vec2(texCoord.x - 4.0 * blurSize, texCoord.y)) * 0.05;
    sum += texture2D(texture, vec2(texCoord.x - 3.0 * blurSize, texCoord.y)) * 0.09;
    sum += texture2D(texture, vec2(texCoord.x - 2.0 * blurSize, texCoord.y)) * 0.12;
    sum += texture2D(texture, vec2(texCoord.x -       blurSize, texCoord.y)) * 0.15;
    sum += texture2D(texture, vec2(texCoord.x                 , texCoord.y)) * 0.16;
    sum += texture2D(texture, vec2(texCoord.x +       blurSize, texCoord.y)) * 0.15;
    sum += texture2D(texture, vec2(texCoord.x + 2.0 * blurSize, texCoord.y)) * 0.12;
    sum += texture2D(texture, vec2(texCoord.x + 3.0 * blurSize, texCoord.y)) * 0.09;
    sum += texture2D(texture, vec2(texCoord.x + 4.0 * blurSize, texCoord.y)) * 0.05;
    sum += texture2D(texture, vec2(texCoord.x + 5.0 * blurSize, texCoord.y)) * 0.01;
    
    fragColor = sum;
}
As I write, I remark that blurSize is constant for all fragments so that I could have made the calculation in the vertex shader. Anyway it's not very optimized.

SCons

I used DSSS last time. This project hasn't been updated for years, still compiles using D1 (discontinued by end 2012)... so let's try something else. As I use SCons for my C++ projects, I've though it would be great to use the same system. Luckily, it has built-in support for D!

Building a D program is the same as for a C++ program:
env.Program('bin/program', Glob('src/*.d'))
Really easy. The compilation flags for DMD are however quite different than GCC, pay attention.

Another thing I've never done is to specify targets, that is, being able to run 'scons release' or 'scons debug'. This is well-handled with Alias, the trick is to retrieve the return value of Program or any other builder. I've separated each target in its own SConscript, partly because of the VariantDir. To pass a value through a SConscript, use Return. Besides, 'scons .' builds everything, so 'all' is a simple alias to '.'.
debug = SConscript('SConscript.debug', variant_dir='build/debug', duplicate=0)
release = SConscript('SConscript.release', variant_dir='build/release', duplicate=0)

Alias('all', '.')
Alias('debug', debug)
Alias('release', release)
Default(debug)

That's all for this project!

No comments:

Post a Comment