Lightning 3.0 discussion thread!

Lightning 3.0 discussion thread

What is Lightning 3.0?
Lightning 3.0 is the next major release of the Lightning project (current version is 2.6) and has focus areas on:

  • Developer Experience
  • Performance

It is a complete refactor of the Lightning codebase and comes with lots of lessons learned from Lightning 1.x/2.x. With Lightning 3, we expect to have a smoother developer experience while creating apps that perform better with less resources required.

Developer Experience
The Lightning 1.x/2.x syntax is very imperative and might have quite a steep learning curve for some. Lightning 3.0 aims to lower that learning curve by providing a more declarative syntax. This should create an easier transition from other frameworks to Lightning. We also plan to provide more tools and functionality. All this is aimed to empower the Lightning developers; give an easier starting point with Lightning and provide higher control to the more advanced users.

Performance enhancements
We have spent a lot of time looking at gaming engines, asking ourselves ‘what does the current engine do’ and ‘what can we improve’. Lightning 3.0 will come with a new render engine that is built from the ground up. We plan to add features such as: Dynamic Shader Generation and Batched Rendering. This should reduce the number of draw calls on the GPU, which benefit the overall FPS and pushing new boundaries on embedded GPUs.

Lightning 3.0 planning
The target for Lightning 3.0 is to have a beta version in Q4 2022. We’ll start publishing alpha releases during the summer and everyone is invited to join!

What does it mean for my current Lightning project?
Nothing, unless you want to! Lightning 2.x is not going away and will continue to be supported. We will keep you posted on the timeline of how long Lightning 2.x. will be supported.

That said moving to Lightning 3.0 does come with small changes. We’re working on a transition guide and tools to make that process as smooth as possible. You do not need to rewrite your application, but there will be impact on the “view” portion that interacts with the Lightning templates. As we get closer to a beta state of Lightning 3.0 we’ll be able to provide more details of the impact.

Have questions? Feel free to drop them in this thread!

4 Likes

Hi :wave:

I would like to highlight a few key points our team stressed during development of a few projects on Lightning SDK.

  1. No code splitting support. Real apps are using a lot of analytics systems and other third-parties, most of which are usually required at the moment when a playback is started. Same applies to third-party player libraries. Splitting player’s code into a dedicated JS chunk will help a lot to reduce an app start time. Alas, this issue is deeply tied to Metrological store which is currently a main target environment for the Lightning apps.

  2. No possibility to configure the bundler (part of lng-cli). We have some third-parties that are using some modern/experimental ES features, like decorators. While esbuild and rollup could support those, we are unable to tweak their configuration as it’s hard-coded inside lng-cli.

  3. No video streaming solution available out of the box. It’s quite strange that the framework designated for development of OTT apps doesn’t have any VOD/Live streaming capabilities except a very basic player for progressive video only which lacks of support of DASH, Smooth, DRM, audio tracks, subtitles tracks, stream quality management, and other important features. Instead, it proposes RTT functionality which is useless on STBs since those are not performant enough to render video to texture in real time.

  4. No basic components for building of layouts which are “industry defaults” for OTT apps. Take a look at Netflix, Amazon Prime, Apple TV, etc. All those channels are presenting nearly the same layout: vertical list of rows with addition of full-screen Hero/Jumbotron components. While it’s easy to create such a list with basic functionality, adding things like virtualization may take a lot of time instead of development business-related functionality. So we need to re-invent the wheel which is already in place in core libraries when we are talking about Android or tvOS, or even regular HTML-based apps (ex. react-virtualized). I’m aware that Lightning UI is a thing, yet it introduces very basic solution for lists and grids which lacks of performance improvements that are crucial for embedded devices.

  5. Bitmap fonts support may come in handy and boost performance a lot.

On a behalf of our team I would say we’ll be super-happy if some of those points will be improved in Lightning 3.0.

1 Like

Hi Serhii! :wave:

Many thanks for the feedback, always love to get input from the community. I’d like to address a few of your concerns, here goes:

  1. This is a very specific artifact to the Metrological App Store. However Lightning is not just for the Metrological store, the Metrological store is one of many distribution platforms we support. There are quite some self hosted apps outside of the Metro solution and additional distribution platforms that we support. Such as Firebolt and Smart TV distribution platforms (e.g. Tizen, WebOS, etc). By any means Lightning serves a broader set of distribution channels and especially L3.0 is going to be fully agnostic to however you like to chose to release and distribute your application. This indeed adds the ability to do different ways of bundling & code splitting and definitely something we’re going to do differently in the next release.

  2. Lightning today already supports ESBuild and Rollup, there are also plenty of Lightning apps released in Webpack. In the future we’re heavily looking towards ESBuild just because we like it so much, it’s blazing fast! Whether or not support will come directly through the CLI or directly is of discretion of the developers at this point.

  3. Strongly disagree. Our competitors are major app dev frameworks like Pixi, React, Vue, Next, etc. And if you look at those frameworks they also do not provide a baked in video player. The main reason for this is video players are hard and would turn the project into the a competitor of HLS.js, Shaka, etc. Those are completely different scopes with a different vision and goal. The amount of work to create a video player is massive and would come at the cost of advancing Lightning as app development framework. Instead we’re fully agnostic to the video player of choice, just like other major app frameworks, allowing developers to take what they need or feel comfortable with. We’ve noticed that major content distributors typically already have a very bespoke video player solution tied into their video delivery pipeline. We can provide small guides or tutorials to help Lightning developers get started with a video playback solution if they are new to the subject. But long story short, Lightning is not in the business of creating a complete video player solution, not will it ever be. There are plenty really good paid and open source solutions out there.

  4. This is being worked on, we currently have repositories like Lightning-UI-Components that provide a set of very elaborate and easy to use components. These will continue to grow as the Lightning standard components of choice. We’re looking into a “boilerplate” generator that can generate different types of screens that would be very typical to the designs you mention. Basically with a little bit of input get a generated boilerplate screen, such as a poster gallery, download the code and use that as a basis to start your own screens on. Though this is work in progress and in early concept phases, though decoupled from Lightning 3.0 work and does not have a hard deadline right now. We’ll have to finish the L3.0 efforts first before we can circle back to the “screen generator”.

  5. Yes! We agree, currently we are investing much into multithreading, dynamic shaders and batched rendering to improve performance in general. Once we got those lined up we are going to look into improving font rendering!

2 Likes

Hi there!

Couple questions from me and my team:

  1. Is there any plan for Typescript definitions to be available in some fashion? We’ve created our own up to now, but it would make our lives a bit easier if we didn’t have to update them each time we do an upgrade.

  2. Will there be some improved eventing? eg. Using event emitters so multiple listeners can be attached, as well as new events like childListChanged for component.

  3. I know it’s early now, but what is the idea for font rendering changes? We’ve worked on a custom text renderer and would be interested in helping out.

Very excited to see 3.0 in the pipeline!

1 Like

Hey @bix, happy to hear you’re excited for Lightning 3.0! And thanks for your questions! Let me answer them one by one:

  1. Yes! For Lightning 3.0 we’ll be making typescript a first class citizen. Which means that we’ll have Typescript definitions, but will also support Typescript in the app code with the CLI out of the box.

  2. This part of Lightning 3 hasn’t been fully scoped out, so I don’t have any exact examples. But I can say that we are bringing several improvements to the way we’ll do inter component communication, as well as handling lifecycle events (created, attached, destroyed etc) from the Lightning renderer to the App development framework. I like the idea of multiple listeners, if you have any real life examples of when you would use multiple listeners I’d be interested to learn more!

  3. The way font rendering is handled in Lightning 2.x is far from ideal, so you can expect a completely new implementation in Lightning 3.0. One of the idea’s we’ve been working on is using a font-atlases for font rendering. @erikhaandrikman will be able to elaborate more on this!

2 Likes

Hi Wouter,

Thank you for the response! Just to advocate the video player point: we are mostly working with Lightning on RDK platforms and we are very frustrated how those are handling video playback. Usually there is no documentation on supported video formats, players, codecs, MSE/EME support, etc. It’s always a guess game to pick a correct MSE/EME wrapper (ex. Shaka, dash.js, rx-player) or even use some undocumented “native” solutions.
Taking into account Metrological’s Lightning was initially developed as a part of Comcast’s RDK initiative (i.e. new MAF) it was expected to see some support when it comes to video playback (which again is a key point of the whole story).

Some extra pain comes again from RDK platforms that forbids HTML/CSS support in their embedded browser so Lightning becomes an only option which makes impossible to use subtitles rendering (as it utilizes DOM and CSS) which comes out of the box in any modern player library, so we need to write custom subtitles renderers re-implementing TTML spec, and we are very lucky if a player library supports plugging-in a custom renderer.

Those platforms also incapable to handle 2 different players working together (to render ads which usually are presented in a form of un-protected plain mp4) and we need to create some custom solutions to parse VAST and VMAP and swap two different players to support CSAI. Major players like Google TV, tvOS and Roku providing such solutions boosting development on those devices significantly. It’s also not a thing with modern Tizen and webOS, they got rid from those problems and currently supporting MSE and EME on a same level as modern desktop browsers.

Sorry for an off-topic, I understand it’s not a Lightning problem and it’s totally fine if this post will be banned, but if there is a slightest chance I may somehow point Metrological developers attention to this problem, it will help a lot of companies which considering RDK as a target platform.

Totally get it! And we should touch base outside of this topic to engage with RDK and see if we can make a difference.

The problem with RDK and embedded devices is the compatibility from the lowest layer up to the browser + how well it is integrated. If you’re comparing Google TV/tvOS and Roku to RDK it’s much like comparing early Windows to early Apple computers. RDK is used by many different devices with many different “flavors” of integration just like Windows whereas Apple always had full control from hardware to software. For the RDK they all have different SOCs, different drivers and thus different levels of support for video playback capabilities. Add in different ways of integrating, different versions and video works slightly different each time. Ultimately the browser will rely on the SOC drivers (their ability to implement Gstreamer plugins) to do video playback. If the GST plugins are not properly integrated, you’ll get different results/mileage. I do think the RDK is working really hard on trying to unify that and MSE/EME is quite well supported by most modern RDK integrations. The browser in a way already provides that abstraction, best approach at its current stage is definitely EME/MSE and use an open source (HLS/Shaka) player or a closed source player like THEOplayer. We’ve been working with THEO to see if they can provide a reference Lightning video player and I hope to bring good news in the future.

I’ll gladly take your feedback and connect you to the RDK. It’s always good to receive feedback from the community and maybe that will help drive better documentation/guides or even a preferred way of doing video in a browser on RDK devices.

1 Like

Hi @Wouter ,

Hope you are well.

Currently we use the lightning-inspect.js for our automation with Cypress. It’s been super useful as without it, we’d struggle to do much. But there’s a lot of pain points with using the generated DOM. Eg we’ve had to add a patch to deduce the focus state of a lightning component and have had to do other workarounds.

Couple of questions on this:

  1. Will there still be support for the lightning inspector going forward or is there a replacement? Any plans on maintaining it?
  2. Have there been any improvements or support added in the testability of lightning ? Does 3.0 having anything to support automating on devices?
  3. Are there any additional debug tools to monitor the performance/resource usage of lightning apps as part of 3.0? Performance benchmarking
  4. What’s the right way to suggest improvements to the inspector or testing improvements in general?
  5. Curious how the lightning team automate their tests. What tools do you use if you are able to share.

Have a great weekend.

Thank you in advance
Richard

Hi @rforjoe !

Appreciate the trigger! I honestly personally haven’t thought much about the lightning inspector for L3.0 yet, since we’ve been so focussed on the core functionality of it. But there certainly will be an equivalent for L3.0 and its great timing to start scoping that out.

I think the best way to suggest improvements for the next version of this is through here for now. I’ll try to collect them and see if we can incorporate them into the overall plan. I assume you’re looking at this from an App automation / Visual Regression type of testing am I right?

If so help us understand what would make it easier for you to achieve those with regards to Lightning, namely:

What are debug tools/APIs you miss in the current setup that you get in a traditional HTML5 app? What do we miss (other then the obvious DOM tree) compared to a HTML5 app?

What kind of workarounds / patches are you using today, can we upstream those so that everyone can benefit and see? It’s hard for us to understand something that we haven’t seen so this would be a tremendous help. Just fire off a pull request to the main repository and one of us will take a look.

On top of that we’ll start looking into similar WebGL based projects and see if we can get any additional learnings from there.

Thnx
-Wouter

Can you elaborate a little bit on “providing a more declarative syntax”?

I second the need to show in the inspector the focus state.

We’re looking to support RTL (right-to-left) layouts and fonts.

In terms of patches workaround:

  • we use JSX for the template as syntaxic sugar,
  • we’re in the process of removing all the StateMachine machinery from the Component as it has a significant CPU impact and we don’t use it.
1 Like

Hi @elsssph,

Lightning 2.x is fully imperative in the sense that you define a template object and perform patches on the render tree object whenever you want to make changes to the rendered state

For Lightning 3.0 we are working on a different (and more clear) way to define the templates. We’re also adding reactivity for template properties out of the box. This promotes a more declarative way of programming. In Lightning 3 you’ll be able to simply update the values of certain properties in the component state, and if they are reactively bound to the template definition, it will automatically render accordingly on the screen. Without the need for a manual patch (which requires you to have knowledge on the template structure in your business logic).

On the state machine, I can say that (in retrospect) we’re also not the biggest fans of it, and it will not be part of L3.0.

3 Likes

Looking forward to see more of the next-gen templating!

Following up from what @rforjoe said above re lightning inspector, we’ve been patching lightning-inspect.js like this, to have the currently-focused element indicated. We’ve been patching it for a long time now, so perhaps this isn’t necessary any more? Either way, for L3.0 it would be great if inspector could pinpoint the focused element for us.

diff --git a/node_modules/@lightningjs/core/devtools/lightning-inspect.js b/node_modules/@lightningjs/core/devtools/lightning-inspect.js
index 47aea7c..78fa3f4 100644
--- a/node_modules/@lightningjs/core/devtools/lightning-inspect.js
+++ b/node_modules/@lightningjs/core/devtools/lightning-inspect.js
@@ -305,6 +305,20 @@ window.attachInspector = function({Application, Element, ElementCore, Stage, Com
         });
     }

+    if (typeof Application !== "undefined") {
+        var oUpdateFocusRec = Application.prototype.__updateFocusRec;
+        Application.prototype.__updateFocusRec = function() {
+            const oldFocusLeaf = this._focusPath && this._focusPath.length > 0 ? this._focusPath[this._focusPath.length - 1] : undefined;
+            oUpdateFocusRec.apply(this, arguments);
+            const newFocusLeaf = this._focusPath && this._focusPath.length > 0 ? this._focusPath[this._focusPath.length - 1] : undefined;
+
+            if (oldFocusLeaf !== newFocusLeaf) {
+                oldFocusLeaf && val(oldFocusLeaf, 'focused', false, false);
+                newFocusLeaf && val(newFocusLeaf, 'focused', true, false);
+            }
+        }
+    }
+
     Element.prototype.$ref = Element.prototype.__ref;
     Object.defineProperty(Element.prototype, '__ref', {
         get: function() {
1 Like

A separate request (something to consider, anyway) for L3.0, is whether we can get better visibility at runtime when textures can’t be created - eg due to GPU resource exhaustion. I feel like I should say that I’m a big fan of L2.0, so I’m having to dig quite deep to find things that are annoying :slight_smile: but it seems 2.0 fails (gracefully, but silently) in this case, and the result is black rectangles or missing portions of rendered UI.

In many situations we’d just pick this up during development, and we’re careful about GPU texture creation in general, manage garbage collection carefully, profile our target device classes, inspect and graph GPU usage etc. However, getting an alert when a texture can’t be created would give us another way to manage this - we can monitor for occurrences of this in production (where the UI is very data driven, so patterns texture generation is harder to predict), and also flag them to developers and QAs more forcefully during development. We could patch L2.0 for this, but I’d like to avoid patching as much as possible.

2 Likes

Hi @Wouter ,

I’m super into video player tech and have worked with Video.js for over a year. I am currently working with Lighting and am also frustrated with how the SDK player is.

Given what you have said, do you think it would be worth investing time trying to create a bespoke player for the Lighting ecosystem?

I am not trying to downplay how complicated such a project would be just curious to know your thoughts on whether it would be more efficient to try and fit an existing player into the ecosystem.

Hey @bix,

Do you guys have these TS definitions open sourced somewhere? Would be nice to have a look!

Hey @imtiaz101325, looks like 2.7.0 now includes TS definitions, so they got to it before we could :heart:

Separately; I was wondering if it would be possible to allow setting custom layouts. We have some custom layouts that we use that work similar to the flex layout, but different enough that we’ve had to put some hackery in place to make it work. Ideally, there would be an interface that we could implement, then pass our own layout system into the ElementCore as an alternative to flex.

In the meantime, adding gap support to flex would also be helpful :slight_smile:

1 Like

Any ETA for the Lightning 3.0 Beta release?