Python API

The application programming interface (API) below is the set of Python objects KCoral exposes to callers. Install the client for program construction and submission; install the server extra when using create_app.

Client

class kcoral.Client(base_url, *, headers=None, connect_timeout_seconds=10.0)

Synchronous HTTP client for a running KCoral server.

Use as a context manager to close connections automatically. The client requires no GPU libraries; computation happens on the remote worker.

__init__(base_url, *, headers=None, connect_timeout_seconds=10.0)

Create a client without contacting the server.

Parameters:

  • base_url (str) – Server address, such as http://localhost:8000.

  • headers (dict[str, str] | None) – Optional HTTP headers sent with every request.

  • connect_timeout_seconds (float) – Limit for establishing a connection.

Response reading has no client-side timeout. Pass timeout_seconds to execute() to request a server-side execution limit.

__enter__()

Return this client for use in a with block.

__exit__(*args)

Close connections when leaving a with block.

close()

Release the underlying HTTP client’s connections.

function(*, timeout=None, output_limit_bytes=None, cpu_only=False)

Decorate a self-contained Python function for this server.

Parameters:

  • timeout (float | None) – Server execution limit in seconds, subject to its maximum.

  • output_limit_bytes (int | None) – Captured output limit per stream.

  • cpu_only (bool) – Whether the function touches no GPU.

Returns:

A decorator producing a RemoteFunction. Its remote() method returns the decoded value; execute() returns the full ProgramResult. Ordinary calls execute locally.

Return type:

Callable[[Callable[_Parameters, _ReturnType]], RemoteFunction[_Parameters, _ReturnType]]

The function must have available Python source, no captured variables, and no external globals. Import dependencies inside the function. Calls reuse this client’s server address, headers and connections. Keep it open for the duration of remote calls.

execute(program, *, timeout_seconds=None, output_limit_bytes=None)

Submit a program and decode the values it explicitly returns.

Parameters:

  • program (Program) – Program built with the client-side Program.

  • timeout_seconds (float | None) – Requested execution limit in seconds; None uses the server default. The server clamps it to its configured maximum.

  • output_limit_bytes (int | None) – Requested captured output limit per stream; None uses the server default, subject to the server maximum.

Returns:

A completed or failed program outcome. Instruction failures do not raise an exception; inspect status and error.

Raises:

  • TypeError – If program is not a client-side program.

  • KCoralError – If the server returns an HTTP request error.

  • TransportError – If no HTTP response can be obtained.

  • ProtocolError – If the response is malformed or cache recovery fails.

Return type:

ProgramResult

The first submission omits binary blobs. A cache miss retries with the requested blobs; a second miss triggers one final submission with every local blob. Returned tensors are CPU NumPy arrays, including extended element types provided by ml_dtypes.

health()

Read endpoint status, load, and compilation environment.

load reports capacity (occupied + free), assigned requests (including GPU waiting), and requests awaiting assignment.

Returns:

The server’s health response with status == "ok".

Raises:
Return type:

dict[str, Any]

target()

Read the GPU architecture an uploaded library must be built for.

Returns:

Target metadata, for example {"arch": "sm_100a"}.

Raises:

ProtocolError – If the health response has no compilation target.

Return type:

dict[str, str]

Query the GPU server, not a CPU compilation server. This calls health() and can raise the same request and transport exceptions.

Remote functions

class kcoral.RemoteFunction(fn, *, client, timeout=None, output_limit_bytes=None, cpu_only=False)

A Python function with explicit remote execution methods.

Construct with Client.function(). Calling the decorated function normally still executes the original locally. Each remote invocation builds a self-contained program; no remote state survives between invocations. The function’s source is captured at decoration.

__call__(*args, **kwargs)

Call the original function locally, without contacting the server.

build_program(*args, **kwargs)

Bind arguments and build a program without contacting the server.

Parameters:

  • args (~_Parameters) – Positional function arguments.

  • kwargs (~_Parameters) – Keyword function arguments.

Returns:

An ordinary Program returning the key output.

Raises:

  • TypeError – If binding fails or an argument type is unsupported.

  • ValueError – If an argument cannot be serialized.

Return type:

Program

Defaults are bound locally and sent explicitly. Arguments may be JSON values, bytes-like objects, NumPy arrays, or DLPack-compatible tensors. Binary values must be whole arguments, not nested inside JSON containers. No pickle deserialization is used.

execute(*args, **kwargs)

Run remotely and return the full execution outcome, including logs.

Parameters:

  • args (~_Parameters) – Positional function arguments.

  • kwargs (~_Parameters) – Keyword function arguments.

Returns:

A ProgramResult; instruction failures remain data.

Return type:

ProgramResult

Uses the same cache negotiation and error handling as Client.execute. Reuses the client supplied by Client.function without closing it.

remote(*args, **kwargs)

Run remotely and return the decoded function value.

Parameters:

  • args (~_Parameters) – Positional function arguments.

  • kwargs (~_Parameters) – Keyword function arguments.

Returns:

The decoded value; tensors become local NumPy arrays.

Raises:

RemoteExecutionError – If the program failed. Its result preserves the error, traceback, request identifier and captured output.

Return type:

Any

Request and transport exceptions propagate unchanged. Use execute to inspect successful execution metadata and captured output as well.

Program construction

class kcoral.Program

Build an ordered, self-contained sequence of remote instructions.

Construction does not contact a server or compile a kernel. Uploads snapshot local input, and Client.execute() sends the instructions and required binary data. Each instruction identifier and return key must be unique within this program. Only values selected by return_() appear in the response.

Each value-producing instruction automatically receives an ID such as upload_0, get_function_1, or run_2. Use the optional id field to customize it.

property instructions: list[dict[str, Any]]

Return a shallow copy of the ordered wire instruction list.

The contained dictionaries are shared with this program. Treat them as read-only; use the builder methods to add instructions.

upload_file(*, blob, path)

Snapshot bytes-like data as a file in the request workspace.

The destination must be a relative POSIX path without .. components and cannot conflict with another file upload. Returns no register.

Parameters:

  • blob (Any) – Bytes-like content, copied when this method is called.

  • path (str) – Destination relative to the request’s working directory.

Raises:

  • TypeError – If the content is not bytes-like.

  • ValueError – If the destination is invalid or conflicts with a file.

Add this instruction before any code that reads the destination. Files are removed when execution ends; cached content may persist.

upload_folder(folder, *, path)

Snapshot a directory as ordinary file uploads at this program position.

Includes hidden files; empty directories and original file metadata are not uploaded. Symbolic links, special files, and repeated directories are rejected. Failed calls leave the program unchanged.

Parameters:

  • folder (str | PathLike[str]) – Local directory whose contents should be uploaded.

  • path (str) – Relative destination directory in the request workspace.

Raises:

  • ValueError – If traversal or destination validation fails.

  • OSError – If the local directory cannot be read.

For example, uploading assets with path="inputs" maps assets/a.bin to inputs/a.bin. An empty folder adds no instructions.

upload(*, id=None, kind, source=None, language='python', value=None, dtype=None, shape=None)

Upload source or binary content and return its request-local register.

Parameters:

  • kind (str) – One of module, tensor, bytes or library.

  • source (str | None) – Source text for a module upload.

  • language (str) – Module language, python or cuda.

  • value (Any) – Tensor input or bytes-like data for a binary upload. Tensors accept NumPy arrays, PyTorch tensors, objects implementing the DLPack tensor exchange protocol, or raw bytes.

  • dtype (str | None) – Element type for a tensor supplied as raw bytes.

  • shape (list[int] | None) – Dimensions for a tensor supplied as raw bytes.

  • id (str | None) – Optional custom instruction identifier. Must be nonempty and unique.

Returns:

A register usable by later instructions.

Raises:

  • TypeError – If the input does not match the upload kind.

  • ValueError – If the kind, identifier or tensor metadata is invalid.

Return type:

Register

A library is a compiled shared object; a module contains source. Use upload_file() or upload_folder() for filesystem uploads. kind="file" is a wire-protocol option, not accepted by this method.

get_function(*, id=None, module, name, cpu_only=False)

Select a named function from an earlier module or library upload.

Parameters:

  • module (Register | dict[str, str]) – An earlier upload register or {"$ref": "id"} reference.

  • name (str) – Nonempty function name exported by the module or library.

  • cpu_only (bool) – Declare that the function does not access the GPU.

  • id (str | None) – Optional custom instruction identifier. Must be nonempty and unique.

Returns:

A function register for a later run() instruction.

Raises:

  • TypeError – If the reference or flag has an invalid type.

  • ValueError – If an identifier, reference or function name is invalid.

Return type:

Register

Running a cpu_only function as its own instruction releases the exclusive GPU lease while it executes. This declaration is checked on a best-effort basis; it does not make GPU operations safe to call.

run(*, id=None, fn, args=None)

Append a function call and return a register for its result.

Parameters:

  • fn (Register) – The Register returned by get_function(), or by an earlier run() that returned a callable.

  • args (list[Any] | None) – Positional arguments; omitted or None means no arguments. Top-level registers are encoded automatically. Nested lists and dictionaries remain JSON literals, including reference-shaped objects.

  • id (str | None) – Optional custom instruction identifier. Must be nonempty and unique.

Returns:

A register, without automatically returning the value to the client.

Raises:
Return type:

Register

return_file(*, key, path)

Select a regular file at this instruction, relative to the request workspace.

return_folder(*, key, path)

Select a complete folder, including hidden files and empty directories.

return_(*, key, value)

Select an earlier value for the response’s results mapping.

Parameters:

  • key (str) – Unique, nonempty response key.

  • value (Register | dict[str, str]) – An earlier register or {"$ref": "id"} reference.

Raises:

  • TypeError – If the value is not a valid reference.

  • ValueError – If the key or referenced identifier is invalid.

A return instruction that runs before a later failure preserves its entry in the partial result. Only serializable values can be returned; compiled modules and function handles cannot be sent back.

class kcoral.Register(id)

Reference to a value produced by an earlier instruction in one request.

Parameters:

id (str) – The producing instruction’s unique identifier.

Pass the register to later instructions in the same Program. Registers do not refer to persistent server-side objects.

Results

class kcoral.ProgramResult(status, request_id, queue_ms, elapsed_ms, lease_wait_ms, lease_held_ms, results, stdout, stderr, stdout_truncated, stderr_truncated, error=None)

Decoded execution outcome, including any results returned before failure.

Parameters:

  • status (str) – COMPLETED or FAILED. The client handles cache misses internally before producing an outcome.

  • request_id (str) – Server-generated request identifier for log correlation.

  • queue_ms (float) – Milliseconds waiting for a worker.

  • elapsed_ms (float) – Execution elapsed time in milliseconds.

  • lease_wait_ms (float) – Execution time spent waiting for the exclusive GPU lease.

  • lease_held_ms (float) – Execution time holding the exclusive GPU lease.

  • results (dict[str, Any]) – Explicitly returned values, keyed by the program’s return keys. Binary values decode to bytes and tensors to CPU NumPy arrays.

  • stdout (str) – Captured standard output.

  • stderr (str) – Captured standard error.

  • stdout_truncated (bool) – Whether standard output exceeded the capture limit.

  • stderr_truncated (bool) – Whether standard error exceeded the capture limit.

  • error (dict[str, Any] | None) – Structured instruction failure, or None on success.

All parameters are available as attributes. When present, error includes the kind, message, instruction index and identifier, and traceback.

property completed: bool

Whether every instruction completed successfully.

__getitem__(key)

Read an explicitly returned value by key; raise KeyError if absent.

class kcoral.ReturnedFile(_data)

An in-memory snapshot, independent of the client and remote workspace.

save(destination, *, overwrite=False)

Save to an exact path with an existing parent; refuse symlink traversal.

class kcoral.ReturnedFolder(files, directories=())

A validated tree; paths are relative to the selected root, which is implicit.

save(destination)

Create a new folder, visible while writing; remove partial output on failure.

Errors

Client.execute and RemoteFunction.execute represent instruction failures as a ProgramResult with status FAILED. RemoteFunction.remote instead raises RemoteExecutionError retaining that outcome. The other exceptions below describe failures to obtain a valid program outcome. See KCoral Protocol for server error codes.

class kcoral.RemoteExecutionError(result)

A decorated function’s program failed on the server.

Parameters:

result (ProgramResult) – Failed execution outcome, retained as result with the request identifier, error details, remote traceback and captured output.

HTTP and transport failures continue to use the ordinary client exceptions.

class kcoral.KCoralError(status_code, message, *, kind=None, request_id=None)

A server response with an HTTP status other than 200.

Parameters:

  • status_code (int) – HTTP response status, such as 503 for a busy server.

  • message (str) – The server’s error message.

  • kind (str | None) – Structured error kind, when provided by the server.

  • request_id (str | None) – Request identifier, when provided by the server.

These parameters are also available as attributes. An instruction failure with HTTP 200 is instead represented by ProgramResult.

class kcoral.TransportError

The request did not produce an HTTP response.

class kcoral.ProtocolError

The server response does not follow the protocol.

GPU utilities

Uploaded Python can import these functions from kcoral.builtins.

kcoral.builtins.compile_tirx(fn, bindings=None)

Compile a @T.jit or @T.prim_func kernel handle. bindings supplies the T.constexpr values a @T.jit kernel is specialized on.

Compiled executables are cached by structural hash, with at most 32 entries. This function can call CUDA while compiling and should run with GPU access.

kcoral.builtins.benchmark(mod, *rest)

Per-iteration GPU activity timing from CUPTI. cfg: warmup_ms/repeat_ms budgets convert to iteration counts using a 5-call estimate, or warmup/repeat set explicit counts; flush_l2 zeroes a 2x-L2 buffer before every call, outside the timed span.

Pass the callable followed by its arguments and an optional configuration dict. Defaults: warmup_ms=25, repeat_ms=100, flush_l2=True. Explicit warmup and repeat counts override their respective budgets.

Returns latency_ms_median, latency_ms_mean, latency_ms_min and latency_ms_max, plus the selected counts, flush_l2 and activities_stable. A span runs from the first to last GPU activity launched by each call; host gaps between those activities are included. Requires PyTorch and cupti-python in the execution environment.

Server integration

class kcoral.ServerConfig(gpus=<factory>, log_dir=None, log_console=True, log_programs=True, cache_capacity_bytes=17179869184, disk_cache_dir=<factory>, disk_cache_capacity_mbytes=16384, default_timeout_seconds=300.0, max_timeout_seconds=900.0, worker_wait_timeout_seconds=1800.0, workers_per_gpu=8, max_requests_per_worker=1, sandbox='bubblewrap', sandbox_readonly_paths=<factory>, worker_termination_grace_seconds=5.0, max_request_bytes=268435456, max_response_bytes=268435456, output_limit_bytes=1048576, max_output_limit_bytes=16777216, device='gpu', num_workers=1, router_endpoint=None, node_id=None, node_token=None)

Immutable worker, cache, logging and request-limit configuration.

Construct with keyword arguments to override the defaults shown in the signature. GPU device identifiers are physical device numbers. CPU mode uses num_workers instead of workers_per_gpu and ignores gpus.

log_dir=None disables logging for applications created directly in Python. The command-line interface instead defaults its log directory to logs. The file-cache path defaults to an absolute XDG_CACHE_HOME followed by kcoral/files, or ~/.cache/kcoral/files otherwise. Set disk_cache_dir=None or disk_cache_capacity_mbytes=0 to disable it.

See the configuration guide for the meaning and unit of every field.

The Launch the server page explains each field, the corresponding command-line option and differences between command-line and Python defaults.

kcoral.create_app(config=None, *, runtime_factory=None)

Build a FastAPI application serving execution and health requests.

Parameters:

  • config (ServerConfig | None) – Worker and request settings; None uses ServerConfig().

  • runtime_factory (Callable | None) – Optional callable that builds a worker runtime, primarily for custom integration and testing. By default the selected CPU or GPU mode determines the runtime.

Returns:

An application to run with an HTTP server such as uvicorn.

Raises:

ValueError – If the device mode or disk cache capacity is invalid.

Return type:

FastAPI

FastAPI is the web application framework used by the server. Worker processes start during the application’s serving lifecycle, not when this function is imported. Install the server extra to use it.

The returned FastAPI application can be served with uvicorn or another compatible HTTP server.

kcoral.parse_program(body)

Validate decoded protocol data and build a server-side program.

Parameters:

body (Any) – A decoded JSON object with instructions and optional options.

Returns:

A kcoral.schemas.Program containing validated instructions and parsed options, not the client-side kcoral.Program builder.

Raises:

ValidationError – If fields, instruction order, references or paths violate the protocol. References must name earlier instructions.

Return type:

Program

This function does not execute instructions, resolve cached blobs or initialize a worker. It is intended for server integration.

class kcoral.schemas.Program

Parsed protocol data returned by kcoral.parse_program(). Its instructions contain validated protocol instruction objects, and its options contain the parsed request options. This is different from kcoral.Program, the client-side request builder.

This object is intended for server integration. Client applications should normally construct a kcoral.Program and call kcoral.Client.execute().

exception kcoral.errors.ValidationError

Raised when protocol data violates the schema, including malformed fields, duplicate identifiers, invalid paths or references to later instructions.

The HTTP front-end turns this exception into a request validation response.