Repository navigation
Merge Release 1.0.0 to master - #58
Open
parshimers wants to merge 16 commits into
Open
parshimers wants to merge 16 commits into
parshimers wants to merge 16 commits into
Conversation
- user model changes: yes - storage format changes: no - interface changes: yes Details: Add basic auth to every API. Add a set of tools for the various deployment methods that generate the passwd file as needed. Alter every test to use it where appropriate. Add a ping servlet that lets various scripts and such determine the liveness of the system without auth. Setting api.authentication.enabled=false turns auth off for the APIs; the UDF endpoints, which nodes call with system credentials, stay authenticated regardless. With auth on, a missing credential file fails startup rather than leaving a server that rejects every login; a malformed one fails it either way. stop-sample-cluster.sh reads the admin password from conf/.admin_password when none is given. Generated-by: Claude Opus 5.5 <noreply@anthropic.com> Change-Id: I63be765a330bb15d53dfaa0ef1c86a9352392628 Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/20684 Reviewed-by: Ian Maxon <imaxon@apache.org> Tested-by: Ian Maxon <imaxon@apache.org>
- user model changes: yes - storage format changes: no - interface changes: no Details: - Adds a modal UI dialog to let the dashboard work with basic auth on all servlets. The dialog uses the requests API to check if the password is right or not. - Allows the OPTIONS method through the servlet wrapper so that the UI can discover the /query/serivce endpoint Generated-by: Claude Opus 5.5 Change-Id: I492d341e84675cca57cef937fdaf482a37a981ce Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/21246 Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu> Reviewed-by: Ian Maxon <imaxon@apache.org> Reviewed-by: Calvin Thomas Dani <calvinthomas.dani@gmail.com> Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>
- user model changes: no - storage format changes: no - interface changes: yes Details: The legacy web interface on 19001 runs any statement it is posted, but was left outside basic auth, so it bypassed the authentication on the query service. It is set up before the other servers, so the credentials are now loaded on first use. Its 401 carries a Basic challenge, as without one a browser shows the error instead of asking for credentials. The other APIs are not browsed to, so the wrapper only challenges when configured to. Generated-by: Claude Opus 5.5 <noreply@anthropic.com> Change-Id: If4a58383883c44214fe81b46312bfd36e00f8549 Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/22197 Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu> Reviewed-by: Ian Maxon <imaxon@apache.org>
Details: Helm chart deploying an AsterixDB cluster in cloud mode, with a bundled RustFS on a PVC as the object store. Keeping all persistent state in the object store leaves the CC and NC pods disposable: NCs cache on emptyDir and repopulate from RustFS when rescheduled, and static storage partitioning lets the number of NCs change. The server does not start without a credential file, so the chart generates one into a Secret or takes an existing Secret. The Hyracks ports have no authentication, so a NetworkPolicy admits only the release's own CC and NCs to them. HelmChartIT takes the packaged chart through install, restart, scale and uninstall on k3s. It is in asterix-podman as pom modules cannot have tests, and follows dockerized-tests in CI as ci is at its limit of concurrent jobs. Generated-by: Claude Opus 5.5 <noreply@anthropic.com> Change-Id: Ibdacf4e6b156a3b6ef15b4420a4102c122f8bf1c Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18242 Reviewed-by: Calvin Thomas Dani <calvinthomas.dani@gmail.com> Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu> Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu> Reviewed-by: Ian Maxon <imaxon@apache.org>
- user model changes: no - storage format changes: no - interface changes: yes Details: With authentication on by default, the image would not start without a passwd file, and a bare run had no way to get one. The entrypoint now generates a password on first start and prints it once, as the sample cluster does, or takes one from ASTERIX_PASSWORD or ASTERIX_PASSWORD_FILE. ASTERIX_AUTH=none turns authentication off. A password containing ':' is refused, as BasicAuthServlet could never accept it. The variables are unset before the server starts, so that UDF processes do not inherit the password. Generated-by: Claude Opus 5.5 <noreply@anthropic.com> Change-Id: Ie5f50bcb0f5840f1009f132aee150c81dc699c05 Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/22263 Reviewed-by: Calvin Thomas Dani <calvinthomas.dani@gmail.com> Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu> Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu> Reviewed-by: Ian Maxon <imaxon@apache.org>
- user model changes: no - storage format changes: no - interface changes: yes Details: The CC console on 16001 was left outside basic auth, though it starts NCs, dumps state and accepts application uploads, whose deploy id and file name could also escape the deployment directory. /admin/cluster exposes node configuration and was open as well; ping and version stay open for health checks. The console takes its credentials from the CC application, so it now starts after the application, and NCs fetch deployed binaries from it with a system credential sent in the deploy request. BasicAuthServlet moves to hyracks-http so the console can use it. With an extension loaded, the console stays open unless the extension supplies credentials. A deployment also waited forever when the nodes answered before the wait began. Generated-by: Claude Opus 5.5 <noreply@anthropic.com> Change-Id: Ibfbb6c1125e3a7065379c6831026aa7d14effee1 Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/22350 Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu> Reviewed-by: Hongyu Shi <leoshy1005@gmail.com> Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu> Reviewed-by: Ian Maxon <imaxon@apache.org>
- user model changes: no
- storage format changes: no
- interface changes: no
Details:
The Connector API and the administration APIs were absent from
api.md, leaving the HTTP surface of a running cluster largely
undocumented. Document them, grouping the administration endpoints
under a single section and folding the existing plan cache entry
into it:
- GET /connector
- GET /admin/cluster, /admin/cluster/summary
- GET /admin/cluster/cc[/config|/stats|/threaddump]
- GET /admin/cluster/node[/{node}[/config|/stats|/threaddump]]
- GET /admin/diagnostics
- GET|DELETE /admin/requests/running
- GET /admin/active[/{expiry}]
- GET /admin/version
- POST|DELETE /admin/rebalance
- POST /admin/shutdown
- GET /admin/storage[/partition/{id}|/stats], and its POST operations
- GET /admin/net
- POST|DELETE /admin/udf/{dataverse}/{library}
Parameters, response shapes and status codes were taken from the
servlet implementations and verified against a running cluster.
Also note which endpoints are served by the node controller rather
than the cluster controller, and correct the node controller API
port in udf.md, which pointed at the cluster controller port. Fix a
malformed anchor that broke the intra-page link to the async result
delivery example.
Generated-by: Claude Code w/ Opus 5
Change-Id: Ic5a1a622c9fd920f4fa60ff90cf805e1da0cb632
Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/21503
Reviewed-by: Ian Maxon <imaxon@apache.org>
Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>
Add a hands-on introduction to running AsterixDB in cloud mode: native collections on S3-compatible storage, COPY TO/FROM, external JSON, Parquet, Iceberg and Delta collections, Iceberg time travel with Nessie, and recovery after replacing the AsterixDB container. A Compose file with S3Mock and Nessie makes every step runnable locally. Carry pathStyleAddressing into the Iceberg catalog's FileIO. Without it an Iceberg collection on a path-style-only S3-compatible endpoint, such as the primer's, cannot read its own metadata. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Change-Id: Iee9a055d4a01a2c239587555b7c4a7c0d35fe05a Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/21666 Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu> Reviewed-by: Ian Maxon <imaxon@apache.org>
- user model changes: no - storage format changes: no - interface changes: no Details: During k-means|| initialization, each oversampling round computed the distance from every vector to every candidate in the pool. The pool only grows between rounds, so most of these distances had already been computed in the previous round. Each round now retains the previous round's nearest-candidate column, computes distances only to the candidates that the previous round appended, and merges the two results. A new candidate replaces the retained one only when it is strictly closer, so ties keep the earlier candidate and the result is identical to that of a whole-pool scan. The final weighing pass reuses the last round's column in the same way. Generated-by: Claude Opus 5.5 <noreply@anthropic.com> Change-Id: I88282ed9d8d36d4009f2f99b8bcd2827f83b184d Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/22304 Tested-by: Ian Maxon <imaxon@apache.org> Reviewed-by: Ian Maxon <imaxon@apache.org>
The version of bun in this branch doesn't work on any CPU that doesn't have AVX2 instructions due to a bug. Newer versions fix this issue. Change-Id: Iee9a055d4a01a2c239587c55b7c4a7c0d3cf305b Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/22384 Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu> Reviewed-by: Ian Maxon <imaxon@apache.org> Tested-by: Ian Maxon <imaxon@apache.org>
This got moved into Hyracks, but Asterix uses it too, so if you try to build Asterix from outside the TLP, it can't build. Change-Id: Iee9a055d4a01a2c239587c55b7c4a7c0d3cf305c Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/22388 Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu> Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu> Reviewed-by: Ian Maxon <imaxon@apache.org>
The Containerfile was importing at a fixed version, which doesn't work so well when the version is bumped for release Change-Id: Iee9a055d4a01a2c239587c55b7c4a7c0a3c3315c Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/22392 Tested-by: Ian Maxon <imaxon@apache.org> Reviewed-by: Ian Maxon <imaxon@apache.org>
The quantized top-K scan copied each stored code out of the page and dequantized it into a fresh double[dim], about 9*dim bytes of garbage per scanned candidate. Read the code in place and decode into one array the cursor reuses. The identity quantizer the storage tests use overrides the in-place decode as well, so those tests take a copy-free path rather than the interface's copying default. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Change-Id: I4296399de8be7663b4b1aeb6d2e922a8d8404c57 Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/22383 Reviewed-by: Ian Maxon <imaxon@apache.org> Tested-by: Ian Maxon <imaxon@apache.org>
Change-Id: I04c4076e9d3f66431c1aaeed92faab2a6ece0645
Change-Id: I04c4076e9d3f66431c1aaeed92faab2a6ede0645
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Merge back all the changes from the release-1.0.0-rc0 and pre-rc branches to master