Skip to content

bootstrap: support configure-time user-land snapshot#42466

Closed
joyeecheung wants to merge 8 commits intonodejs:masterfrom
joyeecheung:snapshot-main-configure
Closed

bootstrap: support configure-time user-land snapshot#42466
joyeecheung wants to merge 8 commits intonodejs:masterfrom
joyeecheung:snapshot-main-configure

Conversation

@joyeecheung
Copy link
Copy Markdown
Member

@joyeecheung joyeecheung commented Mar 25, 2022

This is a less-ambitious version of #38905 which only supports user-land snapshot at configure time. I've made the last two commits separate in case they cause any regressions and it would be easier to revert them to test things out.

A quick example of the feature:

typescript.js: A modified version of https://github.com/microsoft/TypeScript/blob/main/lib/typescript.js which stores the initialized ts namespace in globalThis, see #38905

// typescript.js
// Initializes the TypeScript compiler into ts...
globalThis.ts = ts;
// check-typescript.js
// Reads a file specified by the command line and compiles it
// as TypeScript, prints the compiled result to stdout.
// Use the snapshotted ts by grabbing it from globalThis if
// NODE_TEST_USE_SNAPSHOT is true.
'use strict';

let ts;
if (process.env.NODE_TEST_USE_SNAPSHOT === 'true') {
  console.error('NODE_TEST_USE_SNAPSHOT true');
  ts = globalThis.ts;
} else {
  console.error('NODE_TEST_USE_SNAPSHOT false');
  ts = require('./typescript');
}

const source = require('fs').readFileSync(process.argv[2], 'utf-8');

let result = ts.transpileModule(
  source,
  {
    compilerOptions: {
      module: ts.ModuleKind.CommonJS
    }
  });

console.log(result.outputText);
// file.ts
class VirtualPoint {
  x: number;
  y: number;

  constructor(x: number, y: number) {
    this.x = x;
    this.y = y;
  }
}

const newVPoint = new VirtualPoint(13, 56);
$ cd /path/to/node/source
$ ./configure --node-snapshot-main=typescript.js
$ make node
$ NODE_TEST_USE_SNAPSHOT=true out/Release/node check-typescript.js file.ts

NODE_TEST_USE_SNAPSHOT true
var VirtualPoint = /** @class */ (function () {
    function VirtualPoint(x, y) {
        this.x = x;
        this.y = y;
    }
    return VirtualPoint;
}());
var newVPoint = new VirtualPoint(13, 56);

For now the user can initialize the data/apps to be snapshotted in the entry point specified by --node-snapshot-main and hang this somewhere they can retrieve back after the snapshot is deserialized e.g. globalThis, we can invent better APIs in subsequent PRs (e.g. allowing the user to add a module into the module graph at build time that can be requireed back at runtime). Similar to #38905 the user has to bundle their app into one script to be built into the binary, as user-land modules are not yet supported (we'll need to do more refactoring like #42191 to support user modules in the snapshot).

bootstrap: refresh options in pre-execution

Refresh the options map during pre-execution to pave the way for
user land snapshots which may need to access run-time options at
snapshot-building time. The default embedded bootstrap snapshot
is still prevented from accessing them at snapshot building time
since it serves a wider audience and is ignorant of application
states, while the user-land snapshots are meant to be
application-specific and so it makes sense for them to access
runtime states as long as the snapshotted-code
works with re-initialized runtime states.

build: add --node-snapshot-main configure option

This adds a --build-snapshot runtime option which is currently only
supported by the node_mksnapshot binary, and a --node-snapshot-main
configure option that makes use it to run a custom script when
building the embedded snapshot. The idea is to have this experimental
feature in core as a configure-time feature for now, and investigate
the renaming V8 bugs before we make it available to more users via
making it a runtime option.

bootstrap: make I/O streams work with user-land snapshot

Use the mksnapshot cleanup hooks to release the references
to the native streams so that they can be destroyed right
before the snapshot is taken, and move the initialization
of the global console to pre-execution so that they can be
re-initialized during snapshot dehydration. This makes
it possible for user-land snapshots to use the I/O during
snapshot building.

bootstrap: run inspector and event loop in snapshot builder

This makes --inspect and stdin/out functional in the embedded snapshot,
currently it's not guaranteed that these work perfectly, the
plan is to investigate any out-of-sync states that might appear
in user land snapshots with this while the it is a configure-time
feature.

Refs: #35711

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

author ready PRs that have at least one approval, no pending requests for changes, and a CI started. lib / src Issues and PRs related to general changes in the lib or src directory.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants