Introduction
Rendering realistic ocean surfaces in computer graphics is a challenging yet essential task for creating immersive virtual environments. From video games to movies, accurately simulating the dynamic nature of ocean waves is crucial. However, traditional methods often fall short in avoiding repetitive patterns, known as tiling artifacts, especially when dealing with large ocean surfaces as shown in Figure 1. Furthermore, in a real-time context, memory consumption and performance are critical, which limits the range of feasible solutions to address this issue.
Figure 1: Traditional methods fall short in avoiding repetitive patterns.
In comparison, our method efficiently produces aperiodic oceans.
In our recent publication presented at HPG 2024, we introduced a novel approach that leverages a texture synthesis algorithm to create an aperiodic, realistic ocean surface efficiently. The goal of this blog post is to make more accessible the key aspects of our method, its advantages, and how it compares to traditional techniques.
Traditional Ocean Simulation: The Challenges
The conventional approach for simulating ocean surfaces relies on the Tessendorf model [1], which uses an Inverse Fast Fourier Transform (IFFT) to compute realistic wave displacements as a seamless texture each frame. This approach is illustrated in Figure 2. An additional periodic normal map is often computed using IFFTs for rendering.
Figure 2: Conventional approach for simulating ocean surfaces based on the Tessendorf model
The resulting maps are periodic, enabling us to cover an entire ocean surface by tiling the generated patch. When tiled over a large area and seen from afar, the tiling becomes visually apparent, compromising the plausibility of the generated ocean.
Figure 3: Periodic ocean generated using Tessendorf model
Previous Approaches
To tackle this issue, one common solution is to blend the periodic displacement map with aperiodic noise, such as Perlin noise [2]. In our experiment, adding too little noise fails to hide the periodicity, while adding too much noise can distort the appearance of the ocean, making it look unrealistic.
Another approach involves subdividing the ocean spectrum into several sub-spectra, each representing different frequency ranges. These sub-spectra are then used to compute multiple IFFTs, which are combined to create a final displacement map. This approach is illustrated in Figure 4.
Figure 4: Tessendorf model used with 3 sub-displacement maps
This method yields periodic ocean displacements with a much greater period; however, repetitions are still very visible for low frequency waves, which often requires to tone them down. Additionally, this method requires computing several Inverse Fast Fourier Transforms, which can increase computation time if the resolution of each sub-displacement remains high.
Our Approach: Tiling and Blending
To overcome these limitations, we propose a new method that synthesizes an ocean surface using the tiling and blending algorithm [3]. This technique, originally developed for real-time texture synthesis, is adapted to generate an aperiodic ocean displacement map efficiently as is illustrated in Figure 5. The algorithm works by blending the content of multiple hexagonal tilings on a regular grid while preserving statistical properties of the original maps. Using hexagonal tiles allows to blend the content of exactly three hexagonal contents per texel on the texture. The blending weights are equal to 1 at the center and converge to 0 at the edges to smoothly transition from the content of one hexagon to the other. The content of each hexagon is chosen randomly in the exemplar. This makes the output texture fully aperiodic. Also note that the blending is covariance-preserving rather than linear, meaning that synthesized vectors remain coherent with the exemplar.
Figure 5: Tiling and Blending
To use tiling in blending in the context of ocean generation, we start with a periodic displacement map and normal map generated using the traditional Tessendorf model. These maps serve as the input for our synthesis algorithm. As such, any existing implementation can be enhanced from this point without changing the generation algorithm.
Our method then applies the tiling and blending algorithm to the input displacement map and normal map, as shown in Figure 6 with the displacement map. The result is an unbounded, aperiodic displacement and normal map that preserves the realistic aspect of ocean waves.
Figure 6: Our Approach
Flow map
We introduce a flow map that allows users to dictate the direction of the waves with a map of directions. In practice, it consists in sampling a vector of the flow map in the center of each hexagon, turning it into a rotation angle, rotating the full hexagon by that angle, and then correcting each sampled displacement or normal with another rotation by the same angle around the up vector. This strategy is depicted in Figure 7.
Figure 7: Flow map strategy implementation
The smooth blending from one hexagon content to another also enables to smoothly shift from one ocean wave direction to another. This feature can be particularly useful for creating waves that respond to environmental factors, enhancing the overall authoring control. We show different instances of using a flow map with our ocean in Figure 8. Note that the flow map can be modified in real time since no pre-computations are needed.
Figure 8: Our approach enables to author the orientation of the waves using a flow map
LEAN mapping
We employ LEAN mapping [4] to filter the normals of the ocean surface, ensuring that specular highlights are rendered accurately at any viewing distance as show in Figure 9. It consists in turning the normal map into a BRDF as the resolution decreases to accurately represent footprint of each screen pixel over the shaded surface. This is done by pre-computing the means and covariance functions of the BRDF for each resolution, requiring the computation of an additional texture. It was previously known to be compatible with tiling and blending, so we showed how to implement it directly in our paper. Additionally, we showed how to implement it when using flow maps, correcting the specular lobe without additional pre-computations.
Figure 9: Comparison between normal mapping, ground truth and LEAN mapping
Implementation
Our method can be seamlessly integrated into existing rendering pipelines. It is compatible with any model that generates periodic ocean displacement maps, making it a versatile solution for various applications.
We integrated our algorithm into the shader pipeline to ensure real-time performance. The displacement and normal maps are computed on the GPU, allowing for fast updates and dynamic wave interactions. The displacement map is synthesized in the vertex shader, while the normal map and LEAN map (if necessary) are synthesized in the fragment shader. Our implementation is also compatible with existing systems that generate derivative maps instead of normal maps. It also supports real-time lighting adjustments and environmental reflections.
To handle different levels of detail efficiently, our method dynamically adjusts the resolution of the displacement map based on the viewer's distance through texture filtering and geometry tessellation. This ensures that distant ocean surfaces are rendered with fewer details, while near surfaces maintain high fidelity.
Results
To validate this approach, we conducted extensive side-by-side comparisons with traditional methods. These comparisons highlight the absence of tiling artifacts and the natural appearance of the synthesized waves. To quantify the quality of our results we rely on the autocorrelation function. This mathematical tool helps us measure correlations between observations of synthesized displacement maps that are separated by varying distances. A low autocorrelation value at high distances indicates reduced periodicity. This side-by-side comparison demonstrates the superior visual quality / performance trade-off of our approach and is illustrated in Figure 10. We also showed that the autocorrelation function of our ocean is closer to that of a real ocean for the same reason.
Figure 10: Side-by-side comparison with traditional methods. From top to bottom: ocean rendering (top), displacement map over 4 periods and corresponding (middle) and positive autocorrelation (bottom)
Our approach is faster than traditional methods that combine multiple sub-displacements. It also requires less memory, as it does not necessitate storing and computing multiple IFFTs. It requires an equivalent amount of memory / GPU time compared to ocean generated using Perlin noise, but produces a truly aperiodic ocean, resulting in more realistic open seas.
Figure 11: GPU times
Conclusion
Our research presents a significant advancement in the field of ocean surface simulation. By leveraging a texture synthesis algorithm, we can create realistic, aperiodic ocean surfaces efficiently. This method not only addresses the limitations of traditional techniques but also offers enhanced control and visual quality.
Whether you're a game developer seeking to create immersive environments or a researcher exploring new ways to simulate natural phenomena, our approach provides a robust and versatile solution. We look forward to seeing how this method will be adopted and further refined in future applications.
References
[1]: Simulating Ocean Water, SIGGRAPH 2001 Course notes, Tessendorf, Jerry
[2]: Ocean Surface Simulation, NVIDIA Graphics SDK 11 Direct3D, NVIDIA
[3]: Procedural Stochastic Textures by Tiling and Blending, GPU Zen 2: Advanced Rendering Techniques, Deliot, Thomas and Heitz, Eric
[4]: LEAN mapping, Proceedings of the 2010 ACM SIGGRAPH symposium on Interactive 3D Graphics and Games, Olano, Marc and Baker, Dan