@sleep/magic-wreq (0.1.6)
Installation
@sleep:registry=https://git.teawin.ru/api/packages/moonlight/npm/npm install @sleep/magic-wreq@0.1.6"@sleep/magic-wreq": "0.1.6"About this package
Magic — experimental native transport
Node.js / Deno transport with a default AzureTLS/Go backend and an opt-in Rust backend behind the same C ABI and JavaScript API. Targets: Windows x64 and Linux x64 (glibc and musl/Alpine; see backend evidence below). Not a conformant drop-in replacement for browser Fetch or WebSocket. Gitea npm publication is configured for version-tag pushes; a successful registry release has not been verified here.
Experimental Rust backend
A second, opt-in backend is available in native/rust, using pinned wreq 0.16.1,
wreq-util 0.2.0 and Tokio behind the existing C ABI. Go remains the default. Rust
adds pinned browser mappings (native/rust/src/profile.rs), headerOrder, a
session cookie jar shared by HTTP/WS, and dynamically sized WebSocket messages with cancellable
connects, a two-second graceful-close deadline and session-wide admission limits.
Cookie limits are Rust-specific: 1 MiB command input, 4 KiB per cookie, 3000
stored cookies. Rust supports HTTP/HTTPS proxies using CONNECT for all HTTP/WS
targets and SOCKS5 with remote DNS and optional authentication, through a
credential-protected loopback bridge. HTTPS-proxy certificate verification is
independent of the origin's insecureSkipVerify; proxy failure never falls back
to a direct connection.
forceHTTP3: true uses the implemented standard QUIC/HTTP/3 transport, strictly
for direct HTTPS: configured proxies fail with ERR_MAGIC_HTTP3_PROXY, and
non-HTTPS URLs with ERR_MAGIC_HTTP3_SCHEME. It does not emulate browser JA3/H2
fingerprints or fall back to H2; normal requests do not upgrade via Alt-Svc.
Nonempty ja3/http2 overrides intentionally remain ERR_MAGIC_UNSUPPORTED
because upstream cannot express their exact wire contract. This is not Go
semantic or fingerprint parity. See native/rust/README.md
for prerequisites and details.
rustup toolchain install 1.98.0 --profile minimal
npm run build:rust
npm run test:rust:node
npm run test:rust:deno
node scripts/test-rust-docker.mjs # Or npm run test:rust:docker (Linux x64 glibc)
Rust source-build assets are separate under build/rust/. For source entrypoints,
use libraryPath and Node's copied addonPath; Deno needs only libraryPath.
For installed packages, import @sleep/magic-wreq instead: it bundles the same
shared JS API independently, depends only on its own exact-version optional native
packages, and resolves its own addon/library automatically. @sleep/magic remains
Go; there is no backend-selector option and neither package depends on the other.
Both can coexist in one Node process because their addons have distinct physical
paths. Existing explicit path overrides remain available.
Standalone installed tarballs passed on glibc Node+Deno and native Windows x64 Node 26.7.0 / Deno 2.9.6, including Go/wreq co-installation. Wreq-alone musl Node 22.23.4 / native Alpine Deno 2.3.1 also passed in toolchain-free runtimes. Wreq now advertises all three validated x64 platform dependencies: Windows, Linux/glibc and Linux/musl. Go's independent release remains version-tag driven; wreq publication runs on a matching version-tag push or manual dispatch on that tag, after Linux runtime validation, Linux Docker/xwin Windows build/static inspection and identical main archives across all targets. No Windows runner or Wine is required; CI does not execute Windows binaries. See cross-build evidence and limits. See platform evidence for commands, counts and limits. Earlier source-level Docker validation passed 36/36 integration tests each on glibc Node 22.23.4 / Deno 2.9.6 and 36/36 on musl Node 24.22.1, including public proxy tests. Focused Rust checks also passed on both libc variants, including six HTTP/3 tests covering TLS, streaming, cancellation, informational responses and stopped uploads. Broader Windows feature coverage and Go Deno/musl remain unverified. The Docker runners use Linux named volumes, not host build caches; see Docker setup.
# Native matching host only; use Alpine for linux-x64-musl.
node scripts/build-rust.mjs --target linux-x64
node scripts/npm-package.mjs --backend wreq --target linux-x64
node scripts/npm-smoke.mjs --backend wreq --target linux-x64 --deno
# End-to-end Docker builds, offline tarball installs and toolchain-free runtime checks:
npm run test:npm:split:docker
Wreq archives live under dist/npm/wreq/TARGET/ (Docker validation copies them
under dist/split-validation/npm/wreq/). Its Linux platform packages contain only
magic.node and libmagic.so, never the Go musl helper. Install
@sleep/magic-wreq with optional dependencies enabled; /node, /deno and
/package.json exports mirror the Go package. See distribution.
Measured Go/Rust benchmark results report medians of three alternating rounds through the same Node API/addon, with separate loopback server and fresh client processes. Results are workload-dependent, not evidence that Rust is universally faster: Go led small HTTP throughput, while Rust led some bulk transfers; CPU and memory tradeoffs also vary. These are local API measurements, not WAN, proxy, H3 or fingerprint-parity results. Reproduce from the repository root with Docker Linux containers:
node scripts/benchmark-docker.mjs
Build and load
The following build and runtime details describe the default Go backend unless otherwise noted; Rust-specific behavior and limits are documented above and in its runtime README.
Requires Go 1.24+, Node >=22.18.0 with TypeScript stripping, Deno 2, and GCC/G++
(MinGW-w64 and dlltool on Windows). Musl builds also require python3 for the
IPC wrapper generator and a C++17 compiler. The addon build downloads Node headers from
nodejs.org when suitable local headers are unavailable. No runtime npm dependencies.
npm run build
# Explicit musl target (native Alpine or a configured musl cross toolchain):
node scripts/build.mjs --target linux-x64-musl
node scripts/npm-package.mjs --target linux-x64-musl
node scripts/npm-smoke.mjs --target linux-x64-musl
Host-build outputs: build/magic.dll or build/libmagic.so, and
native/node/magic.node. Explicit-target outputs are under build/targets/<target>/.
Musl also produces an executable magic-helper, which must remain beside
libmagic.so. Windows and glibc keep the in-process Go shared-library backend.
Musl cannot dlopen the Go c-shared library because of its TLS requirements:
its libmagic.so is instead a C++ IPC shim that uses posix_spawn to launch the
sibling regular Go executable and communicates over a Unix socket. The public
JavaScript API and C ABI are unchanged.
The recorded local Windows environment is Node 26.7, Deno 2.9.6, Go 1.24.
The actual musl workflow step passed locally in Docker Ubuntu 24.04 using an
Alpine 3.22.2 chroot with no /proc mount, Node 22.23.2 and Go 1.24.13:
build, packaging, offline npm installation, TypeScript consumer compilation,
Node smoke, targeted Go Helper race tests and Python RPC/fork/SIGKILL checks.
This does not establish broader runtime coverage, Deno/musl support or a
successful Gitea runner execution.
import { createClient, MagicError } from './src/node.ts';
// Deno: import { createClient, MagicError } from './src/deno.ts';
const client = await createClient({
browser: 'chrome', // Transport profile, not an automatic User-Agent change.
maxConnections: 16, // HTTP transport leases; integer 1..8192.
userAgent: 'Magic example/1.0',
headerOrder: ['user-agent', 'accept'],
});
try {
await client.cookies.set('https://example.com/', {
name: 'session', value: 'example', secure: true, httpOnly: true,
sameSite: 'lax', path: '/',
});
const response = await client.fetch(new Request('https://example.com/', {
headers: { accept: 'text/plain' },
}));
console.log(response.status, await response.text());
const saved = await client.cookies.export(); // Sensitive session data.
console.log(saved.length);
} catch (error) {
if (error instanceof MagicError) console.error(error.code, error.operation);
throw error;
} finally {
await client.close();
}
src/node.ts and src/deno.ts are the stable source entrypoints. Public release
packaging compiles src/ to lib/*.js; Node imports @sleep/magic and
@sleep/magic/node execute compiled JS without TS stripping in node_modules.
Deno also uses compiled JS with generated declaration types. The main
package selects exact-version optional native packages for Windows x64, Linux
x64/glibc (@sleep/magic-linux-x64-gnu) or Linux x64/musl
(@sleep/magic-linux-x64-musl), with Node-API 8, Node >=22.18.0 and no install
scripts or downloads. The musl native package also includes the executable
magic-helper; keep it beside libmagic.so with its execute permissions intact.
Go Deno on musl remains unverified; standalone wreq is verified as described above.
Public npm payloads contain compiled JS and generated declarations, not TypeScript
implementations, native sources or build scripts.
Configure the consumer's .npmrc before npm install @sleep/magic:
@sleep:registry=https://gitea.example.com/api/packages/moonlight/npm/
Replace the example host and owner with your Gitea destination; the registry
owner can differ from the @sleep scope. Public packages can be hosted under a
public owner while the source repository remains private. See
distribution for release setup and prerelease installation.
The local musl package validation above is not a registry release or Gitea CI
pass. The earlier full suite failed before fixes; only targeted reruns passed
afterward, and no new full suite was run. See validation.
Implemented
- Immutable per-session browser/TLS/H2/header options and a transport-client pool (default 16 leases, configurable 1..64), with a shared session cookie jar.
- Async HTTP streaming upload/download, Request input, request headers, standard runtime body encoding (including Blob, FormData/multipart, URLSearchParams), AbortSignal through body completion, redirects and lazy decompression.
- Session-isolated PSL-aware cookies shared by HTTP and WS, with get/set/delete/clear/import/export and canonical persistence records.
- Low-level
openWebSocket(url, { protocols?, headers?, signal? }): subprotocols, custom handshake headers, cancellable connects, close metadata and bounded graceful shutdown; an experimental EventTarget class facade supplies states/events, binaryType and bufferedAmount.createWebSocketClassis exported by both stable top-level entrypoints; see usage. - Structured
MagicError; Node environment-owned handles and async Worker teardown, bounded admission and two independent cleanup executors.
The low-level WebSocket API accepts only an optional settings object; old
protocol-string/array overloads and a third argument are removed. Reserved
handshake headers reject before native calls. The standard facade constructor
remains new WebSocket(url, protocols). Matching native components must be
rebuilt for the new {protocols:[],headers:[pairs]} JSON payload; see
WebSocket usage.
Ownership and limits
Read or cancel every response body. Explicitly close sockets and
await client.close(); the class facade's close() returns void and completes
via its close event. Deno requires FFI permission; native network access is not
restricted by --allow-net. Shared libraries remain process-pinned in normal use.
Musl additionally requires permission to execute the helper, subprocess/lifecycle
syscalls, and Unix-domain socket operations, plus writable /tmp for private
temporary socket directories. Sandboxes/seccomp policies must allow these;
the helper adds process startup, memory and IPC overhead versus the in-process
Windows/glibc backend.
maxConnections bounds exclusive HTTP transport-client leases (default 16,
range 1..8192), not physical TCP/TLS sockets. Each exchange holds its lease
through body EOF/cancel. WebSockets use dedicated clients and independent limits:
maxWebSockets defaults to 8192; maxConnectingWebSockets defaults to 128.
Both accept 1..8192 and null resets. Waiting acquisition is abortable.
See runtime limits.
Profiles are pinned to the native dependency versions, not exact browser-version
impersonation. browser does not automatically set a matching User-Agent;
ja3 takes a full string, and http2 a four-field fingerprint. Proxy options are
supported (HTTP/HTTPS CONNECT and SOCKS5). See profile configuration.
Network waits run in Go goroutines. Node uses one completion waiter per environment
and batched TSFN delivery, not a thread per operation. Deno uses one active FFI
wait per dispatcher, not per connection. Data admission is bounded to 32768 jobs
with a standard 1 GiB dispatcher byte budget; the Go musl helper's effective
budget is 768 MiB - 4148 bytes (reservations are not eager allocations).
Node permits 65536 resource reservations and at most 128 sessions; cleanup has
independent lanes. WS messages have no fixed 64 KiB limit and no user-configurable
message-size option; signed 32-bit ABI lengths (including framing), admission
budgets and available memory still constrain them. Each pending async WS read
reserves 1064 bytes of completion/error overhead; actual tagged payload storage is
charged before enqueueing the completion and retained in the budget until drain.
Completion accounting failure produces ERR_MAGIC_ASYNC_LIMIT and is terminal
for the affected WS; it is not a retryable read. The WS bridge transfers whole
messages, not streaming/chunked payloads. Queued/in-flight messages and temporary
copies created before completion accounting are outside this budget. It is not a
hard RSS cap; application queues, stream tee/clone buffers and total RSS are not
globally bounded.
Full Fetch/WebSocket conformance, credentials policy, caching and integrity are
unfinished. This is a server context: no browser CORS enforcement; manual
redirects expose the real response, not an opaque redirect. Autobahn is not done.
ABI stays at version 1, but current adapters require additive symbols absent
from original v1 libraries, including required magic_session_create_options:
rebuild matching components rather than mix versions.
Gitea is the only configured CI. Its sequential Linux-host workflow checks Linux
offline npm installation and Node/Deno runtime on glibc, builds musl natively in
a checksum-verified Alpine chroot with Node runtime smoke, and inspects/extracts
Windows cross-built npm archives without installing or executing them. All three
targets and identical main tarballs are required before publication of the three
platform packages followed by the main package on matching version-tag pushes.
For native Alpine builds and glibc-to-musl cross toolchains, see
Musl build toolchain.
See distribution
for the current single workflow; execution on a Gitea runner remains unverified.
Documentation
- Usage: cookies, Fetch and WebSocket
- Session options and transport profiles
- Implemented vs remaining stages
- Compatibility matrix
- Runtime ownership and limits
- C ABI and additive compatibility
- Distribution and workflows
- Validation record
Sources are separated into src/shared/, src/node/, src/deno/, native/go/,
native/musl/ and native/node/. The older native adapter README
contains useful low-level details but some API/compatibility descriptions lag
implementation; the source-checked docs above describe the current behavior.
Dependencies
Optional Dependencies
| ID | Version |
|---|---|
| @sleep/magic-wreq-linux-x64-gnu | 0.1.6 |
| @sleep/magic-wreq-linux-x64-musl | 0.1.6 |
| @sleep/magic-wreq-win32-x64 | 0.1.6 |