Porting a Large ES5 JavaScript Library to ES6 Modules and Rollup

Since 2011, the PlayCanvas engine sourcebase has adhered strictly to the ES5 JavaScript specification. Since then, the JavaScript language and the surrounding tools ecosystem has moved on considerably. But PlayCanvas has steadfastly stuck to ES5. Why? Internet Explorer 11.

IE11 was released on October 17, 2013. But even today, StatCounter reports that IE11 has 2.43% of the global desktop browser market. Since PlayCanvas content is viewed by 100s of millions of end users, this is a pretty big deal.

Over time, the engine codebase has grown significantly. It’s now nearly 100,000 lines long. Maintaining and building such a large codebase can be problematic. To help bring some level of consistency and structure, we imposed the following pattern:

Object.assign(pc, (function () {
    var SomeClass = function () {
        this.other = new pc.OtherClass();
    };

    Object.assign(SomeClass.prototype, {
        someFunction: function () {}
    });

    return {
        SomeClass: SomeClass
    };
}()));

pc is the PlayCanvas library namespace. So a developer would create instances of this class as follows:

var thing = new pc.SomeClass();

To build the engine, we wrote a node.js script which would parse a list of dependencies (JavaScript filenames) and concatenate them. There were several problems:

  1. The pattern above is overly verbose making it harder to inspect the code.
  2. Object.assign needs to be called 250 times when the library is initially executed by the browser, once for each module. This increases app start-up times.
  3. In internal engine code, all class names and constants need to be accessed via the pc namespace. This is because the internals of the pattern above cannot see the internals of other modules and vice versa. This bloats the engine code and slows things down.
  4. The dependencies file had to be carefully manually ordered to ensure things were declared in the right order.
  5. The build script itself was about 1000 lines of JavaScript which carried its own maintenance overhead.
  6. Unused code was being included in the published engine.

We believed the solution to these problems was to migrate the engine codebase from vanilla ES5 to ES6 modules. This would transform the original module pattern to:

import { OtherClass } from './other-class.js';

var SomeClass = function () {
    this.other = new OtherClass();
};

Object.assign(SomeClass.prototype, {
    someFunction: function () {}
});

export { SomeClass };

Much better!

  • No more needless calls to Object.assign to the pc namespace.
  • No further need to use the pc namespace within modules.
  • Dependencies are explicitly described in each JS file.
  • Only what is necessary is exported from modules (and the overall engine).

But what about IE11? It doesn’t understand ES6 module syntax! 😱 PlayCanvas still needs to ship as a strictly ES5 library. To transform the entire codebase from ES6 module format to ES5 UMD format (PlayCanvas is used in both the browser and Node), you simply need to leverage a JavaScript bundler.

There are many options for selecting a JS bundler. In the end, 3 options were tested: Rollup, Parcel and ESBuild. You can see the PRs for each here:

In the end, Rollup was selected over Parcel and ESBuild. In testing, Parcel needed up to 16s for an initial build, while Rollup only required 3s. And we were reticent to adopt ESBuild since it was far less established and battle tested compared to Rollup. But we may review this decision in the future. Now that the engine is ported to ES6 modules, switching bundler is fairly straightforward.

With Rollup, the build script reduced from ~1000 lines to ~100 lines. Quite a saving. Rollup’s plugin system made it incredibly easy to write custom handling for GLSL files and also run a C-like preprocessor in order to build debug, release and profile versions of the engine.

What’s Next?

Now that we have merged the ES6 Module port, where do we go from here?

First up, Rollup is kindly informing us that circular dependencies exist in the PlayCanvas codebase.

We want to clean things up and eliminate them. What’s the motivation for that? It makes it easier for the bundler to employ tree-shaking to remove unreferenced code from the engine. At the moment, the engine’s Application class imports pretty much everything. And many classes import the Application. This makes it hard to build a version of the library which doesn’t include the particle engine, say. Or maybe the physics engine.

Beyond that, we are keen to explore leaving ES5 behind and fully embracing ‘modern’ JavaScript, or maybe Typescript. Rollup can trivially run Babel or the Typescript compiler as plugins, thereby ensuring we can still ship an ES5 library. We’re looking forward to making that decision based on your feedback. So feel free to let us know what you think on the forum!

TANX lives! Popular WebGL online game returns

In the early days of PlayCanvas, we published a game called TANX, an online multiplayer tank battle game.

To us, it was the perfect illustration of why the web makes such a great platform for gaming.

  • Bite-sized fun with drop-in/drop-out gameplay.
  • Nothing to download – just follow a hyperlink to stream the game in a couple of seconds.
  • True device independence – play on your phone, tablet or desktop.
  • Easy social sharing – get your friends into matches by posting your personal game URL to Twitter in real-time!

So we decided to keep developing the game. The first iteration above just had primitive tanks driving around an empty environment. v2.0 brought upgraded tank models, a new UI and a proper level:

And the game’s popularity continued to grow. It even hit #1 on Hacker News one lazy Friday afternoon!

By now, the PlayCanvas engine was rapidly evolving and had recently gained support for physically based rendering (PBR). We were desperate to upgrade the game to show off these new capabilities. So not long afterwards, we developed v3.0:

The game also leveraged PlayCanvas’ run-time lightmapper to generate the level’s shadows.

Not long after the upgrade, we decided to focus all of our resources on developing the core PlayCanvas engine. In the end, we didn’t have enough time to maintain the game and so, with heavy hearts, we took it offline.

Until today.

We’re really excited to announce that TANX is back! You can play it at:

https://tanx.io

Or alternatively, you can find the game hosted on Miniclip and Itch.io.

Keep your eyes peeled for a follow up article outlining how we rebuilt and deployed the game’s back-end!

Implementing Cloth Simulation in WebGL

The PlayCanvas WebGL game engine integrates with ammo.js – a JavaScript/WebAssembly port of the powerful Bullet physics engine – to enable rigid body physics simulation. We have recently been working out how to extend PlayCanvas’ capabilities by using soft body simulation. The aim is to allow developers to easily set up characters to use soft body dynamics.

Here is an example of a character with and without soft body cloth simulation running in PlayCanvas:

CLICK TO PLAY DEMO

Want to know how it was done? Read on!

Step 1: Create a soft body dynamics world

By default, PlayCanvas’ rigid body component system creates an ammo.js dynamics world that only supports generic rigid bodies. Cloth simulation requires a soft body dynamics world (btSoftRigidDynamicsWorld). Currently, there’s no easy way to override this, so for the purpose of these experiments, a new, parallel soft body dynamics world is created and managed by the application itself. Eventually, we may make the type of the internal dynamics world selectable, or maybe even allow multiple worlds to be created, but for now, this is how the demo was structured.

Step 2: Implement CPU skinning

PlayCanvas performs all skinning on the GPU. However we need skinned positions on CPU to update the soft body anchors (btSoftBody::Anchor) to match the character’s animation. CPU skinning may be supported in future PlayCanvas releases.

Step 3: Patch shaders to support composite simulated and non-simulated mesh rendering

Soft body meshes will generate vertex positions and normal data in world space, so in order to render the dynamically simulated (cloth) parts of character meshes correctly, we have to patch in support by overriding the current PlayCanvas vertex transform shader chunk. In a final implementation, no patching should be necessary, as we would probably add in-built support for composite simulated and non-simulated mesh rendering.

Step 4: Implement render meshes to soft body meshes conversion

PlayCanvas character meshes cannot be used directly by the soft body mesh creation functions (btSoftBodyHelpers::CreateFromTriMesh) and so require some conversion, so the PlayCanvas vertex iterator was used to access and convert the mesh data. Eventually this conversion could be done on asset import into the PlayCanvas editor.

Step 5: Implement per-bone attachments

PlayCanvas currently doesn’t have a way to attach objects to specific character bones via the Editor (it’s on our roadmap for the coming months!). Therefore, per-bone attachments was implemented in order to attach simplified rigid body colliders to different parts of the character to prevent the cloth from intersecting the character mesh. We are using simplified colliders instead of trying to use the full skinned character mesh because it runs much faster.

If you are feeling adventurous, you can find the prototype source code for the example above in this PlayCanvas project:

https://playcanvas.com/project/691109/overview/cloth-simulation-demo

It is a prototype implementation and so expect many changes (some of which are mentioned above) in a final implementation.

Want to try soft body dynamics on your own character? Here’s how:

Step 1: Fork the PlayCanvas prototype project.

Step 2: Mark out what parts of the character you want simulated:

This is done by painting colors into the character mesh vertices – the example character was downloaded from Mixamo, and imported into Blender:

Black = fully simulated, White = not simulated

Step 3: Import the character into the PlayCanvas editor and set up collision: 

On this character, only colliders for the legs and body are needed.

What’s Next

We are really excited about developing this technology in the coming months. We will focus on these areas:

  • Take learnings from the prototype and add good support for soft body dynamics in PlayCanvas
  • Create easy to use tools for PlayCanvas developers to import and setup characters with soft body dynamics
  • Further optimize and improve quality

We would love to get your thoughts and feedback so come join the conversation on the PlayCanvas forum!

A New UI for the PlayCanvas Editor

The PlayCanvas Editor has been around since 2011. Way back then, it was called the PlayCanvas Designer. It was built on Sencha’s ExtJS front end framework and looked like this:

As you can see, it sort of looks like a Windows XP application. If you think it looks retro now, it actually looked retro then! But you can more or less recognize it as PlayCanvas. The Pack Explorer is now the Hierarchy panel on the left. The Attribute Editor is now the Inspector. You can even see the ‘Who’s Online’ bar to the bottom right, which is still around today. We decided quite quickly to drop the WinXP aesthetic by adopting a darker theme in September 2013:

This refresh was definitely an improvement and it was the start of the Designer finding a personality of its own. But after this, the team was struggling to iterate quickly. We took the very difficult decision to throw away the entire ExtJS-based Designer application and rewrite it from scratch using the incredible Vanilla JS framework. And thus, by early 2015, the PlayCanvas Editor was born:

This initial version of the Editor should be much more recognizable. The toolbar and panels are all still in the same location. But today, the Editor packs in far more functionality and is much more powerful.

It was originally written in ES5 and a subset of CSS that would enable the Editor to work in browsers all the way back to IE11. Late last year, we embarked on the next major overhaul for the Editor, with the following goals:

  • Create a clean, structured underlying Editor API (more on this later)
  • Adopt ES6 (let’s get with the times!)
  • Adopt CSS grid to make managing the Editor’s layout easier
  • Refresh and refine the front-end design

We’ve slowly been phasing in the new front-end over the last several months, first deploying an updated Hierarchy Panel and then a new Inspector Panel. Today, we’re proud to announce the final step in the rollout – the new Asset Panel:

To the end user, the most obvious change is the addition of the Asset Panel’s Details View. As an alternative to thumbnails, you can now see a linear list of assets and even sort them on size and type.

We expect that this will rapidly become the default view for most PlayCanvas developers. No more mouse hovering to read long filenames!

So what’s next? Earlier, I mentioned that this completely rebuilt Editor front-end is built on a new, clean, object-oriented API. It is our plan to release this API publicly, once it is ready. This will enable you to write extensions and customizations to the Editor that will culminate in a full plugin system. Stay tuned for further details of that. But in the meantime, let us know what you think by joining the conversation on the forum!

Introducing the PlayCanvas Localization System

Are you shipping your PlayCanvas app or game in just one language? You may be preventing international users from enjoying it! Today, we are happy to announce the arrival of a localization system built right into the PlayCanvas Editor!

PlayCanvas-powered Bitmoji Party localized into English, Spanish and French

The system works in tandem with PlayCanvas’ text element component and it’s super-easy to use. The text element interface now provides a ‘Localized’ property and when checked, you can enter a Key instead of a Text string.

New text element properties for enabling localized text

The Key is the string used to look up a localized string based on the user’s currently selected locale. The localized string data is stored in JSON assets and is documented, along with the rest of the system, here. You can even preview your localized User Interface by choosing a locale in the Editor Settings panel:

Locale setting in the Scene Settings panel

We look forward to playing your newly localized games!

Arm and PlayCanvas Open Source Seemore WebGL Demo

Cambridge/Santa Monica, August 1 2019 – Arm and PlayCanvas are announcing the open sourcing of the renowned Seemore WebGL demo. First published in 2016, the graphical technical demo has been completely rebuilt from the ground up to deliver even more incredible performance and visuals. With it, developers are empowered to build their projects in the most optimised way, as well as using it to incorporate some of its performant features and components into their own projects.

PLAY NOW

EXPLORE PROJECT

“I’m so excited to be relaunching the Seemore demo. Open sourcing it in partnership with Arm will bring a host of benefits to the WebGL development community,” said Will Eastcott, CEO of PlayCanvas. “It’s an incredible learning resource that provides many clean, easy to follow examples of some very advanced graphical techniques. Sharing this project publicly is going to help move web graphics forwards for everybody.”

“PlayCanvas and Arm have always strived to push the boundaries of graphics and the original demo is a testament to that,” said Pablo Fraile, Director of Developer Ecosystems at Arm. “It’s encouraging to see how PlayCanvas have advanced mobile web rendering performance since the original demo. This re-release provides a unique resource into graphics for the mobile web that is both easy to follow and incorporate into your own projects.”

The Seemore demo was originally created as a graphical showcase for the mobile browser and takes full advantage of Arm Mali GPUs. It has been upgraded to utilize the full capabilities of WebGL 2, the latest iteration of the web graphics API. Some of the main features of the demo include:

  • Physical shading with image based lighting and box-projected cube maps.
  • Stunning refraction effects.
  • HDR lightmaps.
  • Interpolated pre-baked shadow maps as a fast substitute for real time shadow-mapping.
  • ETC2 texture compression to ensure that highly detailed textures occupy only a small amount of system memory.
  • Draw call batching.
  • Asynchronous streaming of assets to ensure the demo loads in mere seconds (approximately 5x faster than the original version).
  • Fully GPU-driven mesh animation.

Version Control in PlayCanvas

One of our most requested features has always been for more advanced version control features. We’re very pleased to announce that from today we now have built in version control throughout the PlayCanvas Editor. Integrated support for branches, merging and checkpoints brings a host of new workflow options for your team and we’re confident that it’s going to be a huge productivity multiplier for your HTML5 games and 3D applications.

How does it work?

Checkpoints

Checkpoints take a snapshot of your project at a moment in time. This lets you restore previous versions or just see a timeline of what changes are being made by each member of your team.

Branches

Like other version control systems with PlayCanvas you can create independent lines of development by creating branches. Branches let you or your team work on changes and features that don’t affect your main product development.

 

Merging

Once you’ve finished work in your branch, you’ll want to merge you branch back into your production development. We’ve got a sophisticated merging interface that let’s you merge your code, scenes and assets and resolve any conflicting changes in your scenes and code.

We’ve been testing the version control features over the last few months and we know you’re going to love them. Read more about how you can use branches in your project in our developer docs.

What’s next?

We know that our users have specific needs and want to customize their workflows. With branches now available to isolate development, we’ve unlocked a host of new opportunities that you can try via our API. For starters it’s now possible to synchronize your script assets from your PlayCanvas branch into an external source control system like Github. Try this yourself via our Asset REST API, but we’ll be building on these features in the future.

Scene Launching Supercharged!

Iterating on your PlayCanvas game just got a whole lot faster!

When you launch a scene from the PlayCanvas Editor, a set of assets has to be loaded. Non-script assets are loaded from the browser’s memory cache. However, script assets each generate a round trip request to the server. For projects with a lot of scripts, the load time can be long and your ability to iterate is going to suffer.

Today, we’re pleased to announce that we have deployed an update that introduces a lightning fast build step when you launch your scene that concatenates scripts into a single file. This means that only one HTTP request is made for scripts regardless of how many you have.

How To Enable The Concatenation Goodness

In the Launch button sub-menu, there is a new options called ‘Concatenate Scripts’. Check this to enable the feature:

What Difference Does it Make?

Your mileage will vary depending on how many scripts your project has and what your network conditions are. But if you have a lot of scripts, and you are subject to an internet connection with high latency (ping), then the speed up will be much more noticeable.

Here are the launch times for a scene referencing 125 scripts on low-latency WiFi (150Mbps down with 7ms ping):

  • Concatenate off: 24 seconds
  • Concatenate on: 2 seconds (!)

Wait! Won’t this Screw Up My Debugging?

In a word: nope! We use source maps to refer back to your individual scripts. So you can find your scripts in the Sources tab and set breakpoints as usual instead of having to navigate the super-long concatenated script file.

So we hope you like this latest improvement and that it makes your iteration time faster than ever. Enjoy!

New Feature: 2D Sprites and 9-slicing

PlayCanvas is one of the most popular ways to build 3D interactive web content today. But before 3D graphics was a thing, there was 2D graphics!

Today we’re excited to launch the first part of our 2D graphics support. Great for building classic 2D games.

There are 5 great new features which will help you build 2D games using PlayCanvas.

Texture Atlas Asset

 

The new Texture Atlas asset is an enhanced texture asset which includes additional frame data for splitting your texture into multiple Sprites.

Sprite Asset

The Sprite Asset is a set of frames from a texture atlas. This can be a single button or a complete flip-book style animation.

Sprite Component

The Sprite Component is a new Component that you can add to your Entities. The Sprite Component let’s you play back sprite animation clips and build up your new 2D world.

Sprite Editor

The Sprite Editor is a new tool inside the PlayCanvas Editor to create and edit Texture Atlases. The Sprite Editor lets you quickly define frame regions in your texture and it’s also used to define the 9-slicing region.

9-Slicing

9-Slicing is a very useful technique for creating scalable user interface elements from 2D textures. Using the Sprite Editor to define a border on an image, you can now use Sprites in your Element components to build your UIs. Watch this space for more User Interface features coming soon.

All these features are available today inside the PlayCanvas Editor. Take a look at the documentation and let us know what you think on the forum.

Prehistoric graphics by Pixel-boy

PlayCanvas Engine reaches 1.0.0!

PlayCanvas was born 7 years ago, way back on 9th May 2011. In the early days, we were essentially prototyping, seeing what this amazing new WebGL API could do. By October 2011, we set up a source code repository and committed our first engine prototype. Right at the beginning, we adopted semantic versioning for naming our releases. Our initial commit generated engine v0.10.0. From that point onwards, we adopted a rapid release cadence, often publishing more than one release a week. The months and years passed, our team grew and feature after feature was integrated into the codebase. But through all that time, we never incremented the major version number. Why? Well, there were several reasons:

  1. Our rapid deployment meant we never delivered a monster release that seemed to warrant going to 1.0.0.
  2. We always made a huge effort to maintain backwards compatiblity. Projects like the inane Doom3: Gangnam Style created in December 2011 still work fine today! So we never (intentionally) broke your projects.
  3. We, uh, just never got around to it!

The semantic versioning site says:

How do I know when to release 1.0.0?
If your software is being used in production, it should probably already be 1.0.0. If you have a stable API on which users have come to depend, you should be 1.0.0. If you’re worrying a lot about backwards compatibility, you should probably already be 1.0.0.

The PlayCanvas API is now very stable, mature and battle-hardened. Backwards compatibility is something we take very seriously indeed. And today, PlayCanvas is used in production by thousands of developers.

Indian Motorcycle’s PlayCanvas-powered configurator

And so, it gives me great pleasure to announce PlayCanvas Engine v1.0.0. I want to give my sincere thanks to all of the truly exceptional, hugely talented contributors who helped make this possible.

Here’s to the next 7 years!