Cocos Creator Session Replay (Experimental)¶

This document describes Cocos Creator Session Replay initialization, Camera selection, performance parameters, touch privacy, and node privacy rules.

Prerequisites¶

Session Replay requires:

1. An Android or iOS native build;
2. RUM initialized;
3. An active RUM View;
4. The Cocos SDK and native dependencies from the current SDK version combinations.

If there is no valid RUM Context, the SDK skips the current frame and does not generate replay data independent of RUM. We recommend enabling autoTrack.scenes, or manually calling guanceSdk.rum.startView() before frame capture.

Installing the Standalone Replay Package¶

Starting with Cocos SDK 0.1.0-alpha.6, Session Replay is provided separately by @cloudcare/cocos-session-replay. Install both packages at exactly the same version in the Cocos project root directory:

npm install @cloudcare/cocos-sdk@0.1.0-alpha.6 @cloudcare/cocos-session-replay@0.1.0-alpha.6
npx --no-install guance-cocos install --project . --replay

Reopen Creator, generate, and compile the native project. iOS projects using CocoaPods must also run pod install; see Application Access for SPM configuration and flag retention rules.

The base package does not include the Replay API, frame capture implementation, or Replay native dependencies. The following example saves the composed instance as guanceSdk exported from observability.ts, and other modules reuse this instance.

Initialization¶

import { guanceSdk as baseSdk } from '@cloudcare/cocos-sdk/creator3';
import { withSessionReplay } from '@cloudcare/cocos-session-replay/creator3';

export const guanceSdk = withSessionReplay(baseSdk);


guanceSdk.start({
  ...,
  rum: {
    androidAppId: 'android-rum-app-id',
    iosAppId: 'ios-rum-app-id',
  },
  replay: {
    sampleRate: 1,
    sessionOnErrorSampleRate: 0,
    captureFps: 2,
    maxImageDimension: 720,
    imagePolicy: {
      quality: 'medium',
    },
    touchPrivacy: 'show',
  },
  autoTrack: {
    scenes: true,
  },
});

For Creator 2, change the import entries of both packages to /creator2. Composition must happen before SDK initialization or the Hybrid attach(); subsequent .replay, start({ replay }), and attach({ replay }) calls are all made on the composed instance.

Invalid sample rates, FPS, image dimensions, or traffic policies throw a TypeError or RangeError at the TypeScript layer.

Image Traffic Policy¶

imagePolicy limits the image Resource traffic generated by recorded sessions. You only need to configure a quality tier to use its preset:

replay: {
  captureFps: 2,
  imagePolicy: {
    quality: 'medium',
  },
}

imagePolicy supports the following fields:

Explicitly set maxImageDimension, maxFrameBytes, and maxBytesPerMinute override the tier presets. captureFps is always configured independently; when increased to 2 fps or higher, the budget controller limits the actual output, so traffic does not grow linearly with the nominal FPS.

When adaptiveCapture is enabled, the SDK adjusts capture based on the size of images accepted in the last 60 seconds:

The first frame of a new View or an orientation change can use a separate burst allowance of up to 100 KiB, ensuring a complete frame appears as soon as possible after page switching. To avoid frequent quality oscillation, a lower recovery threshold is used after the budget drops to gradually return to normal capture.

iOS uses JPEG and Android uses WebP encoding, and both the per-frame limit and the minute budget are calculated by the actual encoded size. If the encoded result exceeds maxFrameBytes, quality is lowered first and then dimensions are reduced; if it still exceeds the limit, the Resource is not written.

Camera Selection¶

The first Camera found in the current scene is used by default. Multi-Camera projects should explicitly specify the Camera to replay:

import { guanceSdk } from './observability';

export function selectReplayCamera(camera: unknown): void {
  guanceSdk.setReplayCamera(camera);
}

setReplayCamera() stores only one Camera at a time. When called multiple times, the Camera passed in the latest call overrides the previous setting; the SDK does not capture or composite multiple Cameras simultaneously. If the previous frame is still being captured when it is called, the new Camera takes effect from the next capture.

Call setReplayCamera() again after switching the main Camera. If there is no available Camera in the scene, the current frame is skipped.

Touch Privacy¶

touchPrivacy controls whether Session Replay records touch positions:

Touch capture belongs to Session Replay and is independent of autoTrack.actions: even if autoTrack.actions is not enabled, touchPrivacy: 'show' still records Replay touches; conversely, if automatic Action capture is enabled while touchPrivacy: 'hide' is kept, touch positions still do not appear in the Replay. Even if two consecutive frames are visually identical, pending touch actions are saved to the Replay separately.

The current Cocos API only supports a global touch privacy setting and does not support per-node touch privacy overrides. guanceSdk.replay.setPrivacy(node, 'hide') only hides the node's visuals and does not hide touch records at that position; sensitive pages should use touchPrivacy: 'hide' when starting Session Replay.

Node Privacy¶

All EditBox nodes use mask by default. You can also set rules for specific nodes:

guanceSdk.replay.setPrivacy(accountNode, 'mask');
guanceSdk.replay.setPrivacy(secretPanelNode, 'hide');
guanceSdk.replay.setPrivacy(publicNode, 'unmask');

unmask only removes custom rules. If the node is still an EditBox, the default mask continues to apply.

Privacy areas are calculated based on the node's rectangle in world coordinates. Custom rendering, particles, Shaders, RenderTextures, or visual content that extends beyond the node's bounding box are not automatically inferred as privacy areas; they must be checked scene by scene in integration testing.

Managing Masks per Page¶

Rules set by setPrivacy() are bound to the node instance passed in and do not switch automatically by page name or RUM View. When different pages need to mask different content, the entry and exit logic of each business page should manage the rules for its own nodes.

The following example switches between the "Diagnostics" page and the "Actions" page within the same scene: diagnosticsPage and motionPage are the root nodes of the two pages, and privateTokenNode is a regular node in the Diagnostics page that needs masking. After SDK initialization or the Hybrid attach() completes, call the corresponding functions when switching pages:

function showDiagnostics() {
  // Reapply on every entry to ensure the sensitive node is already masked before it is shown.
  guanceSdk.replay.setPrivacy(privateTokenNode, 'mask');
  motionPage.active = false;
  diagnosticsPage.active = true;
}

function showMotion() {
  // Hide the sensitive page first, then remove the custom rule for that page's node.
  diagnosticsPage.active = false;
  guanceSdk.replay.setPrivacy(privateTokenNode, 'unmask');
  motionPage.active = true;
}

If the Actions page also has sensitive nodes, set mask or hide for each of these nodes before they are shown, and remove the corresponding rules when leaving. When a page contains multiple custom-masked nodes, manage them one by one.

Also note the following in the page lifecycle:

• Setting only the page or its parent node to active = false does not remove the registered custom rules; hiding a page does not replace calling unmask, otherwise the masked areas of the old page may affect subsequent frames.
• Before a page is left or a node is destroyed, remove the custom rules registered for that page; you can place this logic in the project's page manager, or in the component's enable, disable, and destroy callbacks.
• Reapply the rules when entering the page again; if the page is destroyed and rebuilt, you must pass in the newly created node instance.
• Switching the RUM View name does not automatically set or remove node rules; the Hybrid enterCocos() / leaveCocos() manage capture ownership, and pages still need to manage node masks themselves.
• unmask only removes the custom rule of the specified node; it does not disable the global privacy protection or cancel the EditBox default mask.

These rules affect node visuals in Cocos frame captures. Hybrid native pages still use the host Native SDK's Session Replay privacy configuration, while touch positions are controlled separately by touchPrivacy.

During verification, cover "enter page → switch to another page → re-enter", and confirm in the Replay that sensitive content remains masked throughout and no masks linger on other pages.

For more privacy recommendations, see Data and Privacy.

Runtime Behavior¶

• Captures RGBA frames using Cocos RenderTexture.
• When imagePolicy is not configured, the legacy image path is retained; when explicitly configured, Android uses WebP V2 and iOS uses JPEG V2, and accounting is based on the actual encoded size.
• Privacy masks are applied before image compression.
• Frames with identical or nearly static content are skipped; changes to the View, frame dimensions, or privacy rules force the generation of a change frame.
• When an image is skipped due to deduplication, budget, or encoded size, pending touch records are still saved separately.
• A new frame is not processed concurrently while the previous frame is still being processed.
• A single frame capture or encoding failure only drops the current frame and does not stop subsequent Session Replay capture.
• Temporary images are deleted from the application's temporary directory after being written to the Native SDK.

Traffic Estimation¶

The traffic budget only counts image Resources; it does not include Replay Segments, touch records, or upload protocol overhead. Highly dynamic scenes such as continuous combat, camera movement, or particles usually approach the tier limit; menus, static backgrounds, and a small amount of UI animation are affected by identical-frame and near-static detection, so actual consumption is usually lower.

The table below is estimated for a single stable View based on V2 encoding and the default configuration of the corresponding tier. It is intended for capacity planning and is not measured data or a network billing guarantee; the ranges and limits change accordingly after explicitly modifying maxImageDimension, maxFrameBytes, or maxBytesPerMinute.

The casual scene range assumes that the frame continuously has a small amount of change; if the frame is completely static, traffic after deduplication may be even lower. The high-dynamic image value is the rolling 60-second budget limit, not a fixed consumption. The total reference adds Replay Segments, touch metadata, and application-layer upload overhead at low touch density on top of image Resources, and does not include TLS, TCP/IP, cellular, or Wi-Fi link overhead in production networks.

Android V2 uses WebP and iOS V2 uses JPEG. Image content, platform differences in compressors, the actual number of changed frames, the number of View or orientation switches, touch density, and upload retries all affect the final traffic; measure it on the target platform and in real business scenarios before launch.

Estimate the image traffic of a single recorded session as follows:

Image traffic per minute ≈ min(frames actually changed per minute × average encoded size, image rolling 60-second budget)

For example, with captureFps: 2 and quality: 'medium', dynamic scenes automatically reduce the capture frequency after the budget reaches 75%, and lower quality and dimensions after 90%, so image Resources are constrained by the 1.5 MiB rolling 60-second budget instead of growing continuously at 2 × 60 × 40 KiB to about 4.7 MiB/min. The first frame of each new View or orientation switch can still use a separate burst allowance of up to 100 KiB, so the corresponding window may temporarily be higher than the regular image budget.

The budget is enforced per recorded session and cannot replace session sampling. In production, you can first set sampleRate for regular sessions to 0.01–0.05, then configure sessionOnErrorSampleRate according to troubleshooting needs; in diagnostic environments, you can temporarily use 100% sampling. Estimate the overall image traffic as follows; error session supplemental sampling, Segments, and network protocol overhead must be counted separately:

Overall image traffic ≈ total session minutes × sampleRate × estimated image traffic per minute per recorded session

Migrating from the Bundled Package¶

When upgrading from 0.1.0-alpha.5 or earlier:

1. Upgrade the base package to 0.1.0-alpha.6, install the Replay package at the same version, and rerun the installer with --replay.
2. Call withSessionReplay(baseSdk) before the first start() / attach() so that business code reuses the composed instance.
3. Change the previously standalone setReplayCamera(camera) to guanceSdk.setReplayCamera(camera).
4. Move the imports of Replay types such as FTSessionReplayConfig, FTHybridSessionReplayConfig, and FTReplayImagePolicy to @cloudcare/cocos-session-replay/creator2 or /creator3. For overall configurations that include Replay, use FTCocosReplayConfig and FTCocosHybridReplayConfig respectively.
5. Regenerate and compile the native project; replacing only JavaScript does not install the new Replay Bridge.

If you no longer use Replay, remove the composition and the Replay configuration, run the installer with --no-replay, and rebuild. This flag removes the native integration managed by the installer; to pause frame capture at runtime, still use replay.stop() below, and use leaveCocos() in Hybrid scenarios.

Start and Stop¶

If the Replay package is installed and composed and you have not passed replay to guanceSdk.start(), you can also start it separately after RUM is initialized and a View has started:

guanceSdk.replay.start({
  sampleRate: 1,
  captureFps: 1,
  maxImageDimension: 720,
  touchPrivacy: 'show',
});

Stop frame capture:

guanceSdk.replay.stop();

guanceSdk.shutdown() also stops frame capture. Do not start twice via both guanceSdk.start({ replay: ... }) and guanceSdk.replay.start() at the same time.

Native host Hybrid mode does not use the start/stop interfaces above. Session Replay is initialized by the native host in native recorder mode, and Cocos automatically switches to the external capture source between enterCocos() and leaveCocos(). See Native Host Hybrid Integration.

Performance Recommendations¶

• Start by validating with captureFps: 1 and imagePolicy: { quality: 'medium' }; increase to 2 fps only when a smoother picture is needed.
• Enable sampling only in business environments that need replay.
• When low-end devices show GPU, memory, or disk pressure, switch to low first, then lower the longest-edge dimension and the sample rate.
• Verify image orientation and mask positions in scenes with orientation changes, multiple Cameras, complex UI, and low frame rates.