Skip to content

Merge Release 1.0.0 to master - #58

Open
parshimers wants to merge 16 commits into
masterfrom
release-1.0.0-rc0
Open

parshimers wants to merge 16 commits into
masterfrom
release-1.0.0-rc0

Conversation

@parshimers

Copy link
Copy Markdown
Member

Merge back all the changes from the release-1.0.0-rc0 and pre-rc branches to master

parshimers and others added 16 commits November 14, 2026 21:41
- user model changes: yes
- storage format changes: no
- interface changes: yes

Details:
Add basic auth to every API. Add a set of tools for the
various deployment methods that generate the passwd file
as needed. Alter every test to use it where appropriate.
Add a ping servlet that lets various scripts and such
determine the liveness of the system without auth.
Setting api.authentication.enabled=false turns auth off
for the APIs; the UDF endpoints, which nodes call with
system credentials, stay authenticated regardless.
With auth on, a missing credential file fails startup
rather than leaving a server that rejects every login;
a malformed one fails it either way.
stop-sample-cluster.sh reads the admin password from
conf/.admin_password when none is given.

Generated-by: Claude Opus 5.5 <noreply@anthropic.com>
Change-Id: I63be765a330bb15d53dfaa0ef1c86a9352392628
Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/20684
Reviewed-by: Ian Maxon <imaxon@apache.org>
Tested-by: Ian Maxon <imaxon@apache.org>
- user model changes: yes
- storage format changes: no
- interface changes: no

Details:
- Adds a modal UI dialog to let the dashboard work
  with basic auth on all servlets. The dialog uses the requests
  API to check if the password is right or not.
- Allows the OPTIONS method through the servlet wrapper
  so that the UI can discover the /query/serivce endpoint

Generated-by: Claude Opus 5.5
Change-Id: I492d341e84675cca57cef937fdaf482a37a981ce
Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/21246
Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>
Reviewed-by: Ian Maxon <imaxon@apache.org>
Reviewed-by: Calvin Thomas Dani <calvinthomas.dani@gmail.com>
Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>
- user model changes: no
- storage format changes: no
- interface changes: yes

Details:
The legacy web interface on 19001 runs any statement it is
posted, but was left outside basic auth, so it bypassed the
authentication on the query service. It is set up before the
other servers, so the credentials are now loaded on first use.
Its 401 carries a Basic challenge, as without one a browser
shows the error instead of asking for credentials. The other
APIs are not browsed to, so the wrapper only challenges when
configured to.

Generated-by: Claude Opus 5.5 <noreply@anthropic.com>
Change-Id: If4a58383883c44214fe81b46312bfd36e00f8549
Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/22197
Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>
Reviewed-by: Ian Maxon <imaxon@apache.org>
Details:
Helm chart deploying an AsterixDB cluster in cloud mode, with a bundled
RustFS on a PVC as the object store. Keeping all persistent state in the
object store leaves the CC and NC pods disposable: NCs cache on emptyDir
and repopulate from RustFS when rescheduled, and static storage
partitioning lets the number of NCs change.

The server does not start without a credential file, so the chart
generates one into a Secret or takes an existing Secret. The Hyracks
ports have no authentication, so a NetworkPolicy admits only the
release's own CC and NCs to them.

HelmChartIT takes the packaged chart through install, restart, scale and
uninstall on k3s. It is in asterix-podman as pom modules cannot have
tests, and follows dockerized-tests in CI as ci is at its limit of
concurrent jobs.

Generated-by: Claude Opus 5.5 <noreply@anthropic.com>
Change-Id: Ibdacf4e6b156a3b6ef15b4420a4102c122f8bf1c
Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18242
Reviewed-by: Calvin Thomas Dani <calvinthomas.dani@gmail.com>
Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>
Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>
Reviewed-by: Ian Maxon <imaxon@apache.org>
- user model changes: no
- storage format changes: no
- interface changes: yes

Details:
With authentication on by default, the image would not start
without a passwd file, and a bare run had no way to get one.
The entrypoint now generates a password on first start and
prints it once, as the sample cluster does, or takes one from
ASTERIX_PASSWORD or ASTERIX_PASSWORD_FILE. ASTERIX_AUTH=none
turns authentication off.
A password containing ':' is refused, as BasicAuthServlet could
never accept it. The variables are unset before the server
starts, so that UDF processes do not inherit the password.

Generated-by: Claude Opus 5.5 <noreply@anthropic.com>
Change-Id: Ie5f50bcb0f5840f1009f132aee150c81dc699c05
Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/22263
Reviewed-by: Calvin Thomas Dani <calvinthomas.dani@gmail.com>
Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>
Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>
Reviewed-by: Ian Maxon <imaxon@apache.org>
- user model changes: no
- storage format changes: no
- interface changes: yes

Details:
The CC console on 16001 was left outside basic auth, though it
starts NCs, dumps state and accepts application uploads, whose
deploy id and file name could also escape the deployment
directory. /admin/cluster exposes node configuration and was
open as well; ping and version stay open for health checks.
The console takes its credentials from the CC application, so
it now starts after the application, and NCs fetch deployed
binaries from it with a system credential sent in the deploy
request. BasicAuthServlet moves to hyracks-http so the console
can use it. With an extension loaded, the console stays open
unless the extension supplies credentials.
A deployment also waited forever when the nodes answered before
the wait began.

Generated-by: Claude Opus 5.5 <noreply@anthropic.com>
Change-Id: Ibfbb6c1125e3a7065379c6831026aa7d14effee1
Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/22350
Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>
Reviewed-by: Hongyu Shi <leoshy1005@gmail.com>
Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>
Reviewed-by: Ian Maxon <imaxon@apache.org>
- user model changes: no
- storage format changes: no
- interface changes: no

Details:
The Connector API and the administration APIs were absent from
api.md, leaving the HTTP surface of a running cluster largely
undocumented. Document them, grouping the administration endpoints
under a single section and folding the existing plan cache entry
into it:

- GET /connector
- GET /admin/cluster, /admin/cluster/summary
- GET /admin/cluster/cc[/config|/stats|/threaddump]
- GET /admin/cluster/node[/{node}[/config|/stats|/threaddump]]
- GET /admin/diagnostics
- GET|DELETE /admin/requests/running
- GET /admin/active[/{expiry}]
- GET /admin/version
- POST|DELETE /admin/rebalance
- POST /admin/shutdown
- GET /admin/storage[/partition/{id}|/stats], and its POST operations
- GET /admin/net
- POST|DELETE /admin/udf/{dataverse}/{library}

Parameters, response shapes and status codes were taken from the
servlet implementations and verified against a running cluster.

Also note which endpoints are served by the node controller rather
than the cluster controller, and correct the node controller API
port in udf.md, which pointed at the cluster controller port. Fix a
malformed anchor that broke the intra-page link to the async result
delivery example.

Generated-by: Claude Code w/ Opus 5
Change-Id: Ic5a1a622c9fd920f4fa60ff90cf805e1da0cb632
Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/21503
Reviewed-by: Ian Maxon <imaxon@apache.org>
Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>
Add a hands-on introduction to running AsterixDB in cloud mode: native
collections on S3-compatible storage, COPY TO/FROM, external JSON,
Parquet, Iceberg and Delta collections, Iceberg time travel with Nessie,
and recovery after replacing the AsterixDB container. A Compose file
with S3Mock and Nessie makes every step runnable locally.

Carry pathStyleAddressing into the Iceberg catalog's FileIO. Without it
an Iceberg collection on a path-style-only S3-compatible endpoint, such
as the primer's, cannot read its own metadata.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Change-Id: Iee9a055d4a01a2c239587555b7c4a7c0d35fe05a
Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/21666
Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>
Reviewed-by: Ian Maxon <imaxon@apache.org>
- user model changes: no
- storage format changes: no
- interface changes: no

Details: During k-means|| initialization, each oversampling round
computed the distance from every vector to every candidate in the
pool. The pool only grows between rounds, so most of these distances
had already been computed in the previous round.

Each round now retains the previous round's nearest-candidate column,
computes distances only to the candidates that the previous round
appended, and merges the two results. A new candidate replaces the
retained one only when it is strictly closer, so ties keep the earlier
candidate and the result is identical to that of a whole-pool scan.
The final weighing pass reuses the last round's column in the same way.

Generated-by: Claude Opus 5.5 <noreply@anthropic.com>
Change-Id: I88282ed9d8d36d4009f2f99b8bcd2827f83b184d
Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/22304
Tested-by: Ian Maxon <imaxon@apache.org>
Reviewed-by: Ian Maxon <imaxon@apache.org>
The version of bun in this branch doesn't work on any
CPU that doesn't have AVX2 instructions due to a bug.
Newer versions fix this issue.

Change-Id: Iee9a055d4a01a2c239587c55b7c4a7c0d3cf305b
Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/22384
Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>
Reviewed-by: Ian Maxon <imaxon@apache.org>
Tested-by: Ian Maxon <imaxon@apache.org>
This got moved into Hyracks, but Asterix uses it too,
so if you try to build Asterix from outside the TLP,
it can't build.

Change-Id: Iee9a055d4a01a2c239587c55b7c4a7c0d3cf305c
Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/22388
Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>
Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>
Reviewed-by: Ian Maxon <imaxon@apache.org>
The Containerfile was importing at a fixed version,
which doesn't work so well when the version is
bumped for release

Change-Id: Iee9a055d4a01a2c239587c55b7c4a7c0a3c3315c
Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/22392
Tested-by: Ian Maxon <imaxon@apache.org>
Reviewed-by: Ian Maxon <imaxon@apache.org>
The quantized top-K scan copied each stored code out of the page and
dequantized it into a fresh double[dim], about 9*dim bytes of garbage
per scanned candidate. Read the code in place and decode into one
array the cursor reuses. The identity quantizer the storage tests use
overrides the in-place decode as well, so those tests take a copy-free
path rather than the interface's copying default.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Change-Id: I4296399de8be7663b4b1aeb6d2e922a8d8404c57
Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/22383
Reviewed-by: Ian Maxon <imaxon@apache.org>
Tested-by: Ian Maxon <imaxon@apache.org>
Change-Id: I04c4076e9d3f66431c1aaeed92faab2a6ece0645
Change-Id: I04c4076e9d3f66431c1aaeed92faab2a6ede0645
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants