Skip to content
CoCreateJSPublic

About

Local-first JavaScript document database with rich queries, automatic indexing, and distributed data support.

Topics

Resources

Contributing

Stars

0 stars

Watchers

0 watching

Forks

Repository files navigation

CoCreateDB

An embeddable JavaScript document database built around living leaves, automatic indexing and configurable roots.

CoCreateDB keeps a Hot working set in memory and uses Warm local disk for write-through storage, recovery and pressure relief. Objects retain their identity as they are queried and updated, while index entries reference the objects directly. The engine provides a common CRUD interface for data and structural leaves without requiring an external database provider or network connection.

Use CoCreateDB independently in Node.js or compose it with CoCreate Server, CRUD, Workflow and other CoCreateJS capabilities to build persistent and distributed applications.

Development status: this README describes the current Next implementation. The external persistence callback is an early leaf-level seam; complete external durability, distributed acquisition and production integration remain separate work. This is not yet a production release.

Features

  • Adaptive Hot-memory and Warm-local-disk residency with direct UUID lookup.
  • Automatic indexing of scalar values, including nested keys using dot notation.
  • Living leaves and index entries that reference the original object.
  • Configurable root hierarchies with dynamic request-driven root names.
  • Shared create, read, update and delete operations across configured roots.
  • Equality and range queries, filtering and pagination.
  • Update operators through @cocreate/utils.
  • Transferable structural configuration that preserves root identities.
  • Standalone operation and configurable primary/worker forwarding.
  • Optional Cool external Persistence and distributed communication through external capabilities, not dependencies of the engine.

Getting started

Package: @cocreate/db · Repository: CoCreateJS/db

For the current development implementation, install dependencies from the checked-out Next branch:

npm install
npm test

The examples below use the development source. Once this implementation is published, use the package import @cocreate/db instead.

import db from './src/index.js'

const database = db()
const context = {
    organization_id: 'example',
    storage: 'production',
    database: 'application',
    array: 'products'
}

const created = await database.send({
    ...context,
    method: 'object.create',
    object: { name: 'Keyboard', price: 80, quantity: 10 }
})

const uuid = created.object[0].uuid

const result = await database.send({
    method: 'object.read',
    object: { uuid }
})

send() returns the CRUD envelope with a success flag and an array under the requested root name. The input may be a single object or an array of objects. Check success before using the result; failures include an error array.

CRUD operations

Read

Read by UUID directly, by a named identity, or by a filter:

const result = await database.send({
    ...context,
    method: 'object.read',
    filter: {
        query: { price: { $gte: 50, $lte: 100 } },
        startingIndex: 0,
        limit: 20
    }
})

A UUID identifies a leaf directly. Supplied root constraints are still checked. Filters support dot-notation keys such as profile.name. A filter limit of 0 requests all local matches. Results describe the local database, not necessarily the complete contents of an external storage provider or cluster.

Update

const result = await database.send({
    method: 'object.update',
    object: { uuid, '$inc.quantity': 5, price: 90 }
})

The update response preserves the applied patch in object and provides complete resulting object snapshots in updated. For example, the patch includes $inc.quantity: 5, while the corresponding resulting snapshot contains quantity: 15. These are two representations of the same mutation, not two mutations. The applied operators are useful for communication; snapshots are useful for whole-object persistence. Snapshots are independent of later mutations. An upsert that creates a leaf returns the created object instead.

Delete

const result = await database.send({
    method: 'object.delete',
    object: { uuid }
})

The response contains the leaves actually deleted. Structural leaves with children cannot be deleted until those children are removed.

Leaves and roots

An leaf is a data or structural object with a UUID. A root defines its category within a configurable hierarchy. parent identifies the immediate parent leaf by UUID; name provides a named reference where needed. Roots are not separate storage engines.

The default configuration is:

{
    version: '1.0.0',
    roots: {
        $organization_id: {
            storage: {
                database: {
                    array: {
                        object: {},
                        index: {}
                    }
                }
            }
        }
    },
    leaves: {}
}

A root beginning with $ resolves its name from the corresponding request property. For example, $organization_id resolves organization_id. The value names the structural leaf; it does not become its UUID. Other dynamic root names work the same way.

const database = db({}, {
    config: {
        roots: { $tenant_id: { $project_id: { object: {} } } }
    }
})

Create and upsert can establish missing ancestors; reads do not create them. Structural identities are recorded in db.config.leaves, independently from ordinary object data. Configuration can be transferred to another engine to reuse structural UUIDs:

const config = JSON.parse(JSON.stringify(database.config))
const replica = db({}, { config })

This restores structural configuration, not the ordinary objects of the original database. Configuration access uses getValue(path) and setValue(path, value); the returned enumerable properties are defensive snapshots. Only root definitions can currently be edited via setValue, subject to structural validation. Occupied roots cannot be removed or converted between structural and leaf status. Structural renaming and reparenting are not supported.

Living leaves and automatic indexing

The engine maintains a UUID-keyed leaf map and grouped index trees internally. Index entries reference their original object rather than storing a second complete copy. Each tree has a root branch and index branches that organize scalar values and support ordered range traversal. Query planning selects an available candidate index and verifies matching objects against the query.

An leaf exposes getValue, setValue, getIndex and delete methods through its prototype. Use the mutation interface or CRUD operations rather than assigning directly to object properties, which can bypass index maintenance. UUID, root and parent are immutable through updates; structural names are also protected.

The index primitives are independently usable from the source modules. Their low-level interfaces are not yet a stable package-level contract. Current CRUD updates validate and reconcile a cloned object before modifying its living leaf; update snapshots add another allocation. Further reducing that cost is an optimization target.

Runtime integration

The public engine exposes send() and config. It accepts an optional living server object and configuration. The asynchronous initializer integrates with @cocreate/config:

import db from './src/index.js'

const server = { isPrimary: true }
server.db = await db.init(server, { prompt: false })

In standalone usage no server is required. In worker mode an injected forward(request) adapter is required; the engine does not implement IPC itself. The size option controls index branch sizing.

Local state defaults to mode: 'adaptive' and acknowledgement: 'memory'. Adaptive uses Hot memory and Warm local disk, evicting weighted leaves and branches under pressure. Its default requested budget is 25% of total system memory, capped by memory available at startup; this is a soft residency target, not a hard cap on process memory. At high pressure, adaptive temporarily uses disk-first request handling. mode: 'disk-first' remains available explicitly for mitigation/testing; mode: 'disk-cache' clears the DB-managed base on startup and starts fresh. The independent acknowledgement option also accepts 'disk', which waits for affected local writes. These options are instance-level; memory acknowledgement is not a transaction or disk-durability guarantee. Each instance uses a base directory containing config.json, branches/, and leaves/. Adaptive and disk-first resume the base Config; if no directory is supplied, a temporary base is created and is not restart-stable. External Persistence acknowledgement is not implemented by this engine yet.

The DB's memory policy manages its Hot/Warm working set. It cannot cap total server RSS or prevent unbounded ingress from consuming memory in sockets, HTTP handlers or queues. Process-level OOM protection belongs to server telemetry and traffic management: bound in-flight work, drain or throttle ingress under pressure, and have clients retry with backoff.

Persistence and distribution

CoCreateDB exposes a minimal external persistence transport and an in-memory pending journal grouped by root UUID. Register a provider and explicitly drain pending leaf changes:

database.persistence.register(async envelope => provider.send(envelope))
const persisted = await database.persist()

The drain sends complete UUID-bearing leaf snapshots for upserts and UUID-only requests for deletes. A successful provider acknowledgement clears the matching pending item; errors remain attached for retry. Reads can be sent through database.persistence.send() using persistence.read. This is an early integration seam: the separate Persistence Tree and root/index branch writes are not implemented, so this does not yet provide complete external durability. acknowledgement: 'persistence' is therefore unavailable.

CoCreate CRUD → CoCreateDB → CRUD workflow / AST
                              ├─ Persistence
                              ├─ Remote CRUD acquisition
                              └─ Communication

The default memory acknowledgement returns after the local Hot mutation is available; local Warm writes continue separately. External Cool Persistence runs only when explicitly drained or orchestrated by a workflow. Missing local data can be acquired from Persistence or other servers through standard CRUD requests. Sequential, parallel, recursive and latency-aware acquisition are workflow choices. Socket, IPC and Mesh own recipient awareness and mutation delivery. Cold archive integration is also external.

These integration designs are documented in persistence and distribution and awareness. External orchestration, complete acquired-leaf hydration, durable ordering and deduplication are not implemented by this engine checkpoint.

Tests and benchmarks

npm test
npm run benchmark -- 1000
BENCH_MODE=restored npm run benchmark -- 1000
npm run benchmark:ranges

Correctness tests and Node.js benchmarks live under test/. See benchmark instructions, baseline report and provider references.

Local benchmark baseline

November 19, 2026; Node 22.16.0, Linux x64, 1,000 objects, five independent runs, 40 measured query samples and 10 warmups. Fresh configuration; median of run metrics.

Operation Time
Create and index 1,000 75.378 ms
UUID read p50 0.0045 ms
Equality read p50 0.1102 ms
Nested read p50 0.1275 ms
Range read p50 0.2118 ms
Update p50 0.0318 ms
Delete 25 0.7866 ms

These historical measurements were in-process and memory-only, excluding network, workflow and durable storage costs. Historical MongoDB and Redis references are retained separately; they are not controlled equivalent-workload comparisons. No cross-provider performance claim is made.

Specifications

See the specifications index for detailed engine contracts, integration designs and outstanding requirements.

Source structure

src/
├── index.js       Engine factory and initialization
├── leaf.js    Living leaf and index-entry primitives
├── root.js       Configuration and root resolution
├── crud/
│   └── index.js   CRUD processing
└── tree/
    ├── index.js   Root indexes and index branches
    └── query.js   Query planning and selection

Current limitations

Sorting and search, broader data-type support, atomic batches, stale/concurrent mutation reconciliation, production authorization, complete root-index recovery and external workflow integration require further development. Root constraints are not a substitute for authorization. A generated temporary base supports disk offload but is not restart-stable; provide the same base path to resume the local Config and data. Warm-disk acknowledgement is local durability only; Cool Persistence and Cold archive require external integration. Detached persistence requires external error handling, ordering and shutdown policies.

CoCreateJS

CoCreateDB is part of CoCreateJS, the JavaScript project spanning CoCreate Client and CoCreate Server. Independently usable primitives and configurable workflows allow applications to compose the capabilities they need.

Developed by CoCreate LLC. Licensed under AGPL-3.0 (see repository license).

See the technical whitepaper for the broader architectural vision.

See Transactions vs. Synchronization for the rationale behind selective, workflow-based coordination.

About

Local-first JavaScript document database with rich queries, automatic indexing, and distributed data support.

Topics

Resources

Contributing

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Used by

Contributors

Languages