Cryo Engine

Cryo Engine is an experimental game engine. It is not meant to be used for real game productions: it is a testing ground for a new kind of asset registry that doubles as a version control system.

The idea came from the GDC 2023 talk "Managing Source Content for 'Overwatch 2'", given by Rowan Hamilton from Blizzard. It explains how the Overwatch engine relies on an integrated version control system to manage its assets.

Why build this?

I have been using Perforce for several years, both professionally and on personal projects. I even built a package to make it easier to install on my home Synology NAS. Yet every time I administer my Perforce servers, the same frustration comes back: it is software that works, but whose design no longer matches today's standards.

Storage and scalability

For binary files, which make up most game assets, Perforce stores every revision in full, simply compressed. Two versions of a file that are 99% identical therefore take up twice the disk space. Modern techniques such as deduplication could drastically reduce that footprint, yet Perforce has not adopted them.

Perforce was also never designed to run as a distributed, cloud-based service. It is possible to build large multi-server architectures, but they feel more like pre-cloud workarounds than a genuinely scalable solution.

To be fair, I don't criticize the choice itself. Perforce is software that requires rock-solid stability, and completely changing how it stores files would be an extremely risky move. My point is only that it has fallen behind the current ecosystem.

This is precisely one of the directions taken by Lore, the open-source version control system developed by Epic Games, which positions itself as a direct competitor to Perforce. Lore is built to scale, relies on deduplication to optimize storage, and uses standard protocols (such as HTTP and gRPC) that make it easy to integrate.

Engine integration

The other frustration comes from how Perforce is used inside game engines, especially Unreal. Even though Unreal was designed with Perforce as its primary supported VCS, the integration remains shallow: one asset equals one file, and Perforce has, by design, no idea what that file contains, nor how it relates to other assets.

For example, a user can delete a file in Perforce even though it is still used by other assets in the Unreal project. Perforce will accept the deletion without any warning, even if it breaks the project for the whole team.

The core idea: a versioned asset registry

Cryo Engine is built around an asset registry that is also the version control system.

The architecture is fairly simple: clients (the editor, tools, etc.) talk to a central server, a setup similar to centralized VCSs like Perforce. The difference is that they don't manage files, they manage asset versions.

Content and metadata

An asset is made of two parts:

• Content: the actual data, whether XML, text, binary or any other format.
• Metadata: everything that describes the asset, such as its identifier, its "path", its dependencies, its size, its type, and so on.

Every version of an asset is made of both, and the server manages both. As a result, the server doesn't just store versions: it understands the assets it holds.

Reference integrity

The server also manages references between assets, and acts as the source of truth for resolving them. To be submitted, an asset must only reference assets that exist. There can be no missing assets, which keeps every workspace consistent at all times.

This fixes a classic weakness of traditional VCSs like Perforce: since they don't track links between files, an asset can easily end up depending on another asset that has been deleted.

Storage

For content storage, Cryo Engine builds on the work I started with Atlas. Uploads and downloads work the same way: content-defined chunking, bundling, and downloads through HTTP range requests. This approach greatly reduces storage without hurting performance, since identical data is only stored once, even across versions.

This split mirrors the Overwatch engine, which is divided into two servers: one that manages versions, and another that manages content storage.

Asset identity

Each asset is identified by a GUID. Unlike the rest of the metadata, this identifier is immutable, and it stays the same across every version of the asset: whatever changes are made to its content or metadata, it remains the same asset. The ID is the central element that links assets together, and every reference between assets goes through it.

The asset's path, on the other hand, is ordinary editable metadata. It is only a visual hint for developers, for example to decide where the asset appears in the editor's file browser. The ID's job is to be unique, not descriptive; the path's job is to provide a human-readable name. Since references never rely on paths, moving or renaming an asset never breaks anything.

Developer workflow

From a developer's point of view, the system works much like Perforce, with a workspace to check out, edit and submit assets. The main difference is that assets must be handled through dedicated tools, such as the editor, since they are no longer plain files.

Virtual file system

Because the system handles assets with IDs rather than files with paths, it becomes possible to build a virtual file system: assets appear as regular files in Windows Explorer without their content being downloaded. Content is only fetched on demand, when it is actually needed.

This virtual file system is only a view for developers: a hierarchical representation of the assets. The actual content lives in a hidden local cache, for example a simple folder where each asset's content is stored in a file named after its ID.

Packaged builds

All of this only applies to development and production. Once the game is packaged, it uses a different asset registry, which reads data from archives rather than from the server.

Advantages

• Asset information without downloading: Since the server is a versioned asset registry, the details of any asset can be queried without downloading or loading it into memory, including the details of any other version of it.
• No more sync time: Since content is only downloaded when needed, the goal is to open the editor and start working right away, without fetching the whole project first.
• An always-consistent project: References are validated by the server, so a workspace can never contain broken dependencies.
• Efficient storage: Chunking and deduplication avoid storing the same data several times.

Drawbacks

• High complexity: The whole system is significantly more complex than a traditional VCS.
• Dependence on dedicated tools: Assets can't simply be opened and edited with any external software, since everything goes through tools built for the system.
• Reliance on the server: With on-demand downloading, working with assets that aren't cached yet requires access to the server.

Current status

The project is progressing slowly: it is a large undertaking, and I often have other priorities. That said, I already have a simple working prototype that supports:

• asset versioning
• branching
• locks

The engine only contains the bare minimum needed to manage a few assets. It has nothing of a real game engine yet: no physics, no rendering, not even an actual gameplay loop. The same goes for tooling: there are only a few essential tools for testing versioning, with no level editor or anything similar.