Lightning 2: Ability to create multiple RTT textures of Elements that do not update with changes

Get Help
1

Problem:

We have surfaces that we want to animate, crossfade. Surfaces are Lighning Elements that group together elements that make up the content of one screen/page. In some instances different surfaces also reuse the same Element and switch up the sub elements to represent a different Surface.

We are trying to animate Surface switching. Currently it looks bad when faded in/out without rasterising, as individual elements looked as if they faded one after the other. Overlapping elements revealed elements hidden behind them (Ex. tile image fading out revealed fallback text behind), making the transition undesirable.

To fix that we can use the renderToTexture method that Lightning provides, but in doing so we suffer from lower FPS as Lightning updates this texture after any change within the surface (triggering another renderToTexture). Additionally in WebKit it also creates flickers.

Current Solution:

We call renderToTexture for one frame and use that first instance of the texture throughout the animation (Since we don’t care about seeing updates in the surface during the animation, we only care about animation being fluid). Once the animation is finished, we replace the texture with the actual surface.

Recipe to create this texture:

  1. 2x Surfaces to render to texture and crossfade
  2. 2x Instances of a custom texture that hold the frozen textures (extends lng.Texture and implements getSourceLoader)
  3. Our “Neuralyzer” (Reference to Men In Black) for wiping Lightning’s memory
  4. A method to get all elements in the right state and then capture texture at the right time

Since we don’t have a way of freezing rtt-ed textures, I created a custom texture. This does rely on having rtt: true on the surface for at least one frame. I then grab the WebGLTexture created by Lightning’s texturizer and apply it to my custom texture.

However, I don’t know what Lightning will do first in its render loop, draw my textures or renderToTexture the surface. So I assume that the safest time to create my custom texture is in the next frame using requestAnimationFrame to give the surface element a chance to have finished rendering to texture.

So the contents of getSourceLoader are like this:

source = element.core._texturizer.getRenderTexture() // at this point we should be in the immediate next frame of setting rtt:true, so this texture should exist
element.core.ctx._renderTexturePool.splice(element.core.ctx._renderTexturePool.indexOf(source), 1); // we need lightning to forget about this texture so it doesn't overwrite it
element.core._texturizer = null; // safe to reset the texturizer, lightning will create a new one when it needs to

return function(cb) {
    cb(null, {source: source, permanent: false});
}

Delaying one frame is a bit problematic, we have to be very careful not to switch surface until the first texture is created and completed, especially if the surface reuses the same element. Then we have to create another texture and make sure rtt was set to false after the first one, and then we have to “Neuralyze” Lightning so it forgets about the last texture it created (to avoid reusing it). If we don’t, our frozen textures get replaced because Lightning still owns and reuses those textures.

Additionally, if rtt stays true for more than a frame and we have taken its texture, then it breaks with exceptions about texturizer being undefined on the Surface element.

Things that would make life better:

  • Not having to rely on rtt and swap render targets of the surface ourselves
  • Own the WebGLTexture so it is not reused/overwritten unexpectedly.
  • In the current approach, it would be useful if we had a hook when the render loop finishes its call to renderToTexture, so we don’t have to wait another frame.
  • Being able to restore render method back to that with rtt:false in the same frame.
  • Being able have multiple rendered textures for the same element representing different screens.
  • Lastly having a way to manage GPU memory so these WebGLTexture instances don’t pile on and being able delete them on gc, currently Lightning doesn’t do that for TextureSource with nativeTexture of WebGLTexture type.