Texturing component

This component provides extensive textures support. 2D textures may be loaded from image files (ImageTexture), movie files (MovieTexture) or encoded directly in VRML/X3D files (PixelTexture, also ImageTexture with data urls). Multiple textures may be overlayed on a single polygon in a variety of ways. Texture coordinates may be explicitly provided or automatically calculated, and may be transformed.

See also X3D specification of the Texturing component.

Contents:

See also Castle Game Engine (and view3dscene) extensions related to texturing.

1. Demos

For demos and tests of these features, see the texturing_advanced and movie_texture and multi_texturing subdirectories inside our VRML/X3D demo models.

2. Supported nodes

  • ImageTexture(Pascal API: TImageTextureNode),
    PixelTexture(Pascal API: TPixelTextureNode).

    These nodes define a texture, from an external file (in the usual case of ImageTexture) or embedded inside the X3D file (in case of PixelTexture, or ImageTexture using data URI).

    ImageTexture allows various texture formats, including JPEG, PNG, GIF, BMP, PPM, RGBE, KTX, DDS. See castle-view-image documentation for a detailed list.

    Note about alpha channel: alpha channel of the textures is fully supported, both a simple yes-no transparency (done by alpha_test in OpenGL) and full range transparency (done by blending in OpenGL, just like partially transparent materials). See "override alpha channel detection" extension description for details. The bottom line is: everything will magically work fast and look perfect.

  • TextureCoordinate(Pascal API: TTextureCoordinateNode),
    TextureTransform(Pascal API: TTextureTransformNode).

    These nodes specify how the texture is mapped onto the shape. All X3D nodes have some "default" texture mapping, if you don't provide explicit texture coordinates.

  • MovieTexture(Pascal API: TMovieTextureNode)

    This node defines a texture that is actually a (playing) movie file.

    TODO: for now, the sound of the movie is not played.

    Notes:

    • Current implementation keeps the whole encoded video in memory (images may be discarded after loading (by TCastleSceneCore.FreeResources feature), but still the textures for all frames are kept in memory). The disadvantage is that this makes it impractical to load "real" movies, normal 2-hour movie will most usually eat all of your memory. The advantage is that once the movie is loaded, the playback is super-fast, just like you would display normal nodes with static textures. Since there's no streaming, decoding etc. in the background while you browse your models.

      In other words, this is quite perfect for movie textures with game effects, like smoke or flame. But it's not a substitute for your "real" multimedia movie player.

    • ffmpeg must be installed and available on $PATH to actually open any movie format. See instructions for installing ffmpeg in view3dscene docs. Thanks to ffmpeg, we can handle probably any movie format you will ever need to open.

    • We can also open movies from images sequence. This doesn't require ffmpeg, and allows for some tricks (movie texture with alpha channel). See "Movies from images sequence" extension description.

  • MultiTexture(Pascal API: TMultiTextureNode),
    MultiTextureCoordinate(Pascal API: TMultiTextureCoordinateNode),
    MultiTextureTransform(Pascal API: TMultiTextureTransformNode)

    Multi-texturing means that you can use multiple textures on a single polygon (face). A number of textures can be mixed, in a way resembling the "layers" concept you from image editing applications (like GIMP or Photoshop). For each layer, you specify which texture to use, the "mode" (how it is mixed), also each layer may have it's own texture coordinates.

    Some examples are present in the demo-models, in particular in multi_texturing subdirectory.

    We implement various modes (in MultiTexture.mode), sources (MultiTexture.source) and functions (in MultiTexture.function). As an extension, we also allow separate mode and sources for RGB and alpha, as documented in Proposed improved MultiTexture.mode specification.

    TODO: Modes not implemented yet: MODULATEALPHA_ADDCOLOR, MODULATEINVALPHA_ADDCOLOR, MODULATEINVCOLOR_ADDALPHA.

    MultiTexture.source values "DIFFUSE" and "SPECULAR" are treated the same. They don't make much sense in the context of PBR material parameters anyway.

    Note that the multi-texturing specification of X3D has unfortunately some issues. Clarifications and our extensions are documented here.

    Note that in case of 2D games, when the camera cannot move freely, you can often use multiple polygons (each with a single texture) rendered on top of each other (positioned with slight shift), instead of using a single polygon with multi-texturing. Both choices are fine, in principle: use whichever is more comfortable for your case.

  • TextureCoordinateGenerator(Pascal API: TTextureCoordinateGeneratorNode)

    This node can be used instead of TextureCoordinate(Pascal API: TTextureCoordinateNode) to automatically calculate the coordinates based on some algorithm.

    Supported modes are now "SPHERE", "COORD", "COORD-EYE", "CAMERASPACEPOSITION", "CAMERASPACENORMAL", "CAMERASPACEREFLECTIONVECTOR". We also allow some extensions: "WORLDSPACEREFLECTIONVECTOR" and "WORLDSPACENORMAL", "BOUNDS", "BOUNDS2D" and "BOUNDS3D".

    Note that "CAMERASPACEPOSITION" and "COORD-EYE" are exactly the same thing. Google confirms it (e.g. this source code also treats them as equal and in this old bitmanagement spec they mention they are equal).

    TODO: not implemented modes: "SPHERE-LOCAL", "NOISE", "NOISE-EYE", "SPHERE-REFLECT", "SPHERE-REFLECT-LOCAL".

  • TextureProperties(Pascal API: TTexturePropertiesNode)

    Adjust various texture filtering/wrapping properties. Supported fields now are:

    • minificationFilter, magnificationFilter,
    • anisotropicDegree,
    • boundaryModeS, boundaryModeT, boundaryModeR.

    Some details about supported and unsupported fields:

    anisotropicDegree (fully supported)

    See examples/viewport_and_scenes/anisotropic_filtering/ for example Pascal code that creates TextureProperties nodes and requests anisotropic filtering for specific textures. We plan to enable adjusting this in CGE editor at some point too.

    textureCompression (we do not support compressing texture data at runtime)

    We don't plan to support the this field now.

    Reason: If you're looking for a way to use GPU compressed textures, simply place the GPU compressed texture data in DDS or KTX files. Using GPU compressed textures in DDS and KTX is fully supported, and always will be a more efficient solution anyway.

    boundaryModeS/T/R overrides texture repeatS/T/R

    Be aware that using TextureProperties means that texture fields repeatS/T/R is ignored.

    E.g. ImageTexture.repeatS is ignored, instead we'll use TextureProperties.boundaryModeS to determine whether texture repeats or not in S coordinate.

    You should instead adjust boundaryModeS/T/R, which is by default "REPEAT" (regardless of the texture type, 2D or 3D). You get more options for it now: "CLAMP_TO_EDGE", "REPEAT" and "MIRRORED_REPEAT".

    This is a particular trap for 3D texture nodes, like ImageTexture3D, that have by default repeatS/T/R = FALSE (unlike 2D textures like ImageTexture that have by default repeatS/T = TRUE). So merely adding a TextureProperties node to 3D texture, with everything left as default, changes the default (for 3D textures) "clamp" mode into "repeat". Adjust the boundaryModeS/T/R to "CLAMP_TO_EDGE" to restore "clamp" mode.

    clamp to border (unsupported)

    We don't plan to support the "clamp to border" wrapping modes for boundaryModeXxx fields. For the same reason, we don't plan to support related borderColor and borderWidth fields, that only make sense for "clamp to border" mode. The idea of "clamping to border" is from OpenGL but it does not have much actual usage and it was removed/rejected from various other modern APIs:

    For this reason, "CLAMP", "CLAMP_TO_EDGE", "CLAMP_TO_BOUNDARY" are all equivalent and work like "CLAMP_TO_EDGE".

    meaning of "DEFAULT" minification / magnification filters

    The default value of magnificationFilter and minificationFilter is "DEFAULT". Which means to use the texture filtering mode specified by TCastleRenderOptions.MinificationFilter, TCastleRenderOptions.MagnificationFilter (by default nice minLinearMipmapLinear, magLinear), matching behavior when TextureProperties was not specified.

    So this behavior is most natural.

    (It was a bit more tricky in older CGE versions <= 7.0-alpha.1, before 2022-03-09.)

3. Supported image file formats

See castle-view-image features for the full list of 2D image formats we can handle.

See also lower on this page for details about DDS format support and KTX format support.

4. Clarifications to X3D multi-texturing specification

Comments about X3D MultiTexturing problems and solutions (used in our engine, and proposed for future X3D spec) are here.

5. DDS (Texture format) support details

DirectDraw Surface (DDS) image format is supported. This format allows to store textures compressed for GPU (S3TC), storing mipmaps, cube maps, and volume (3D) textures. A number of technical details about DDS implementation are below, but in short: we try to support all formats and all options of DDS in a standard way. You may find example DDS images inside our VRML/X3D demo models (look inside textures subdirectory for *.dds files). You can open them directly with our image viewer castle-view-image.

Implementation history:

  • DDS support is natively built into the engine. Since I knew that I want to use many of DDS features, like cube maps, 3D textures, mipmaps, compression, I decided the best way to go will be to create my own reader, instead of relying on external tools.
  • Other notable open-source implementations of DDS are by GIMP-DDS plugin and new ImageMagick (since 6.3.9).
  • While implementing, I was looking at GIMP DDS source code (it's on GNU GPL >= 2 :) ) and MS documentation for DDS.

Cube maps in DDS are supposed to be oriented as usual for DDS:

  1. Which means that they match Direct X "positive/negative x/y/z". For OpenGL rendering we swap positive/negative Y faces (because Direct X has left-handed coordinate system, see here for drawing of DirectX cube map images orientation and compare with OpenGL cube map orientation).

  2. It's also a different orientation then the one of X3D ComposedCubeMap specification (left/right, bottom/top, front/back, with bottom/top on Y axis; X3D orientation needs rotating left,right,front,back images by 180 degrees for OpenGL orientation).

Images in DDS are supposed to be written from top to bottom row, as is the standard in DDS. (One particular tool, AMD CubeMapGen, allows to invert rows of the DDS images to match OpenGL bottom-to-top ordering; don't use this — we expect rows ordered as is standard in DDS, top-to-bottom.) Internally, our engine just inverts the rows for OpenGL (yes, this is doable also for S3TC compressed images.)

Pixel formats supported:

  1. Absolutely all uncompressed non-float pixel formats are supported.

    Details:

    The formats that are currently loaded optimally are ABGR8, BGR8, AL8, L8. They translate to RGBA8, RGB8 etc. OpenGL formats (reversed order, as DDS color masks are little-endian). Popular ARGB8 and RGB8 are also loaded very fast.

    Grayscale (luminance) images are allowed. AL8 and L8 are optimized. Note that grayscale images aren't officially allowed by DDS docs, but at least GIMP-DDS plugin can write it (just sets all R, G and B masks equal, and doesn't set any of DDPF_RGB, DDPF_FOURCC, DDPF_PALETTEINDEXED8).

    Also only-alpha images are allowed (another undocumented DDS feature, GIMP-DDS can write it, for now they will result in grayscale(white) with alpha image).

  2. Compressed texture formats handled: DXT1, DXT3, DXT5 are supported. Texture with DXT1 is always treated like a texture with simple (yes/no) alpha channel (so it will be rendered with alpha testing) and DXT3 / DXT5 are always treated like a texture with full range alpha channel (so they will be rendered with blending).

    Texture compression formats commonly used on mobile platforms are supported as well: ATITC, PVRTC, ETC1.

    Both normal (2D) textures and cube maps may be compressed. (There is no compression possible for 3D textures — neither DDS format allows it, nor do common graphic cards.)

  3. Reading float textures from DDS is for now not supported. Our engine supports float textures (see TRGBFloatImage class), but DDS reader doesn't support them yet. Please submit a feature request (through forum or ticket system), preferably with some test images, if you need them.

If DDS file includes mipmaps, and mipmaps are required for texture minification filter, we will use DDS mipmaps (instead of generating mipmaps automatically). Works for all 2D, 3D, cubemap DDS files.

6. KTX (Khronos Texture format) support details

Since Castle Game Engine 6.3 we support Khronos KTX texture format. KTX is in many ways an alternative to DDS, with a clean specification, supporting the same features (GPU compression, 2D and 3D textures...). The KTX Khronos page has links to various tools that can create KTX files, and to the KTX specification.

The features of KTX that we support:

  • 1D, 2D and 3D images (textures).

  • Various uncompressed texture formats. RGB + alpha, RGB, luminance, luminance + alpha, luminance 16-bit, luminance float32 texture... RGB can be swapped as BGR (it doesn't matter for performance in practice, either way).

    More uncompressed formats can be trivially added, just submit a bugreport with a sample KTX file.

  • Texture arrays, mipmaps. Engine programmers: look at internal TCompositeImage to access all mipmaps, and for each mipmap: the items of the array.

  • Configurable orientation. KTX data may be specified in top-to-bottom or bottom-to-top order. For 3D images, the slices can additionally be in front-to-back or back-to-front order. This is specified by a special KTXorientation field inside the KTX file (see the KTX specification for details) and we support it fully.

  • GPU-compressed data, like

    • S3TC (DXT* formats, often available on desktops),
    • ATITC (ATI compression, available on some mobile devices),
    • PVRTC (PowerVR compression, available on some mobile devices),
    • ETC1 (Ericsson compression, ofen available on mobile devices, but without alpha support).
    • More compresssed formats can be trivially added, just submit a bugreport with a sample KTX file.

TODO: We don't support (yet) cubemaps in KTX. Report if you need it, it's easy to add.