Skip to content

fix(source, connection): source auth flags on WEBHOOK sources, retry status code ranges, and related upsert fixes - #496

Merged
leggetter merged 20 commits into
mainfrom
claude/clever-bohr-87hc9u
Nov 13, 2026
Merged

leggetter merged 20 commits into
mainfrom
claude/clever-bohr-87hc9u

Conversation

@leggetter

@leggetter leggetter commented Nov 13, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Fixes #341, fixes #412, fixes #406, fixes #466, fixes #472.

Fixes two bugs found while live-testing a runbook with CLI v3.1.0 against API version 2026-09-01, plus the related problems that turned up while fixing and reviewing them.

Reported bugs

  • Source auth flags on WEBHOOK sources returned 422. --source-api-key / --api-key failed with config.auth is not allowed, and so did the basic auth, HMAC and --webhook-secret flags. The API checks config.auth against the method named in config.auth_type for any source type that offers a choice of verification method, and the CLI only sent auth_type for HTTP. The CLI now infers auth_type on every source-config path: flags, --config/--config-file, --source-config, create/upsert/update, inline connection sources and partial updates.
    • It reads each source type's methods from the cached OpenAPI spec (pkg/cmd/sources/auth_methods.go) and picks the method whose fields best fit the auth keys. WEBHOOK and HTTP have a built-in table for when the spec can't be fetched.
    • This also covers ORDINAL, AZURE_EVENT_GRID, SENDGRID_INBOUND, PROMETHEUS_ALERTMANAGER, BETTER_STACK, GRAFANA and GITLAB.
    • If no single method fits the auth keys, auth_type is left unset and the API reports the error.
  • --rule-retry-response-status-codes "429,500-599" was rejected client-side. Each entry is now validated against the API's own response_status_codes pattern, so codes, ranges, comparisons (>400) and exclusions (!401) are accepted, up to the API's limit of 10 entries. The flag help is updated.

Related fixes

  • source upsert without --type silently converted an existing non-WEBHOOK source to WEBHOOK and dropped its verification. PUT /sources defaults a missing type to WEBHOOK. The CLI now looks the source up by name and sends its stored type.
  • source update with config flags but no --type failed with config is not allowed. The CLI now sends the stored type and builds the config for it.
  • connection upsert with only --source-* config flags on an existing connection failed. The stored auth_type came back without its credentials. Changing auth methods now works, and non-auth changes now succeed.
  • connection upsert creating a new connection with --source-name but no --source-type sent source.type: "", which the API rejects. It now sends WEBHOOK, matching the config it builds.
  • connection upsert with an inline source and no destination flags on an existing connection failed with must contain at least one of [destination_id, destination]. It now keeps the stored destination, and the stored source in the reverse case.
  • Spec-driven source auth validation was a no-op. FetchSourceTypes read a SourceCreateRequest schema the spec no longer has, and cached the empty result. It now parses SourceTypeConfig<TYPE> and never caches an empty parse.
    • The 27 types it treats as auth-required match the API's source type definitions in hookdeck/core. ONFLEET is the exception: it requires only a handshake, so its auth is treated as optional.

auth_type inference is documented in the help for the source and connection commands. That includes the case where a bare webhook_secret_key is treated as Standard Webhooks.

Follow-up fixes (from review)

Source auth built per source type. The auth flags (--webhook-secret, --hmac-secret, --api-key, the basic auth flags) and the legacy --config keys (webhook_secret, api_key, basic_auth, hmac) now build config.auth and auth_type from the source type's own methods in the spec, not one generic shape.

  • --webhook-secret no longer overrides a provider's signature header. The API applies a source's stored header_key, algorithm and encoding over the provider's defaults. The flag sent x-webhook-signature/sha256/hex for every type except STRIPE, so VAPI, FORMSTACK, BETTER_STACK, GRAFANA and SUPABASE sources rejected their provider's real requests. It now sends the secret alone, and adds the HMAC settings only where the method requires them: WEBHOOK and HTTP, which are unchanged.
  • auth_type is set for every type that offers a choice of method. GITLAB, GRAFANA, PROMETHEUS_ALERTMANAGER, BETTER_STACK and ORDINAL returned 422 config.auth is not allowed with the auth flags.
  • Flags a type can't use are refused before the request is sent. --webhook-secret on PROMETHEUS_ALERTMANAGER, BETTER_STACK and ORDINAL used to get a 422. --api-key on ORDINAL, GRAFANA, BETTER_STACK and ACUMATICA, which need their own header name, used to store x-api-key. Each error shows the --config form to use.
  • Spec download: it times out after 10s, and both caches share one download. When the spec can't be fetched, WEBHOOK and HTTP use the built-in methods, and other types get the bare secret or key.

Connections and destinations

Testing for the follow-up: go test ./... passes. Acceptance on the combined branch: connection and connection_update 91 passed; source, connection_upsert and destination 143 passed; none skipped or failed. The live VAPI test checks that a request signed in x-signature is verified and a wrong signature is not. Each fix was reverted in turn to confirm a test fails without it. REFERENCE.md is regenerated.

Testing

  • Unit: spec parser and inference tests use a fixture cut from the real 2026-09-01 spec. Command-level tests run the real commands against a stub API and assert on the request body. Each fix was removed in turn to confirm a test fails without it. pkg/cmd tests are kept offline by a TestMain that serves the fixture.
  • Acceptance: new tests under the existing connection, source and connection_upsert tags. The full source (36), connection_upsert (38) and connection (69) suites pass against the live API on the combined branch. The new gap tests fail against the pre-follow-up commit.
  • go vet ./..., gofmt and go run ./tools/generate-reference --check are clean, and REFERENCE.md is regenerated.
  • Not run: golangci-lint. The installed binary was built with Go 1.25, and the repo targets 1.26.

Notes for reviewers

  • The source auth validation is back in force, so commands the API would reject are now stopped by the CLI first. Examples: no auth flag for an auth-required type such as AIRCALL, or --webhook-secret on PAYPAL, whose auth requires webhook_id. The error points to the flag or --config shape to use.
  • The cause of the upsert type conversion is the .default('WEBHOOK') on type in the API's source create schema. The CLI works around it, but it may be worth changing on the API side too.

Generated by Claude Code

Posted by an AI agent with human oversight.

claude and others added 20 commits November 10, 2026 21:35
The API validates config.auth on WEBHOOK and HTTP sources against the
method named in config.auth_type, and rejects auth without it with
"config.auth is not allowed". The CLI only added auth_type for HTTP, so
--api-key / --source-api-key (and --basic-auth-*, --hmac-*,
--webhook-secret) failed with 422 on WEBHOOK sources.

- Generalize ensureSourceConfigAuthTypeForHTTP to ensureSourceConfigAuthType
  for WEBHOOK and HTTP, and call it inside the shared config builders so
  flag and --config/--config-file paths both get it.
- Infer STANDARD_WEBHOOKS for a bare webhook_secret_key (HMAC requires
  algorithm/encoding/header_key, so that shape can only be Standard
  Webhooks); keep HMAC when those fields are present.
- source update without --type: the API rejects config sent without type
  ("config is not allowed"), so look up the stored source and send its
  type, which also selects the auth_type to infer.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0172mwz15DSjQ64NaYbtyCxw
…atus-codes

The flag only accepted integers, so "429,500-599" failed client-side even
though the API (and --rules JSON) accept range, comparison and exclusion
expressions. Validate each entry against the API's RetryRule
response_status_codes pattern instead, and document the forms in the
flag help.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0172mwz15DSjQ64NaYbtyCxw
… path

Follow-up from review of the auth_type fix, which left paths that still
failed with "config.auth is not allowed":

- source upsert without --type: the API upserts as WEBHOOK, so config is
  now built for WEBHOOK.
- connection upsert of a new connection with --source-name but no
  --source-type: sent source.type "" (rejected by the API); now WEBHOOK.
- connection upsert partial source update (--source-* flags only): built
  config with an empty type and merged it over the stored config, whose
  auth_type comes back without credentials. Now built for the stored
  type, and the stored auth_type is dropped unless its auth is sent too
  (sending it bare failed with "source.config.auth is required").
- source update without --type: rebuild config for the stored type, so
  the auth shape matches it, not only auth_type.

auth_type inference now reads each source type's methods from the cached
OpenAPI spec (pkg/cmd/sources/auth_methods.go) instead of a WEBHOOK/HTTP
list, covering ORDINAL, AZURE_EVENT_GRID, SENDGRID_INBOUND,
PROMETHEUS_ALERTMANAGER, BETTER_STACK, GRAFANA and GITLAB. The method is
the best fit for the auth keys; ties are left to the API. WEBHOOK/HTTP
keep a built-in table when the spec cannot be fetched.

Also: document auth_type inference (including bare webhook_secret_key ->
STANDARD_WEBHOOKS) in command help, reject more than the API's 10
response_status_codes client-side, and add command-level tests against a
stub API plus acceptance tests for the WEBHOOK auth and retry range paths.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0172mwz15DSjQ64NaYbtyCxw
PUT /sources validates with the API's source create schema, where type
defaults to WEBHOOK. `gateway source upsert <name>` sent no type when
--type was omitted, so upserting an existing STRIPE source with only
--description converted it into a WEBHOOK source: same id, default
WEBHOOK allowed methods, verification gone.

When --type is omitted, look the source up by name (including disabled
sources) and send its stored type, building config (and inferring
auth_type) for that type. When no source has the name, nothing changes:
no type is sent, config is built for WEBHOOK, and the API creates a
WEBHOOK source. If the lookup fails, the command fails rather than
falling back to the typeless request that converts the source. This
mirrors connection upsert, which fills a missing inline source type from
the existing connection.

The upsert Long help now says how the type is chosen instead of warning
about the conversion, and REFERENCE.md is regenerated. The stub-API
harness gains stubStatus so a test can fail the lookup.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0172mwz1hz2ZKGd77wwmvG5h
The spec-driven auth check behind `source create/upsert` and
`connection create` had been off since the API moved to 2026-09-01.
parseOpenAPISpec read components.schemas.SourceCreateRequest, which the
spec no longer has, so FetchSourceTypes returned an empty map. That map
was cached for 24 hours, every type lookup missed, and
validateSourceAuthFromSpec returned nil. The tests stayed green because
their mock spec still used the old shape.

Parse the SourceTypeConfig<TYPE> schemas instead, reusing the
auth_methods.go schema types. For each source type: its name as the API
spells it (x-docs-type, so PROPERTY-FINDER), the fields its config.auth
takes and requires, and whether auth is required.

The per-type schema is shared by request and response, so its auth is
always nullable. The request rule shows up only as x-required on the
type's SourceTypeConfig.oneOf entry. The server sets that when any
feature is required, a handshake included. So auth counts as required
when x-required is set, the type has one fixed method (the API always
accepts auth_type null for a choice of method), and that method has
fields (handshake-only types like STRAVA have none). Against the live
spec this gives exactly the server's 26 verification-required types,
plus ONFLEET (handshake required, verification optional), which
optionalAuthSourceTypes now lists in place of STRIPE. STRIPE's auth is
now read as optional from the spec.

validateSourceAuthFromSpec drops the AuthScheme switch. The spec has no
such field, and the old parser invented it. It now builds the
config.auth the flags would send (sourceAuthFromFlags, extracted from
the builder) and checks it against the type's fields:
- no auth flag for a type requiring auth: name the flag that sets the
  fields, or --config when none does (ALIPAY's public_key)
- an auth flag missing a required field (--api-key for STRIPE): the API
  strips the keys a type does not take, then rejects the request

An empty or failed parse is now an error and is never cached. An empty
cache file is never served. The cache file name gains "_v2", so the
`{}` the old parser cached under the current name is not read.

types_test.go now runs against the fixture cut from the real spec. The
fixture gains the SourceTypeConfig umbrella with its x-required markers,
plus types covering each case: required (CLOUDSIGNAL, ALIPAY,
TIKTOK_SHOP, VERCEL_LOG_DRAINS), optional (GITHUB, POSTMARK, ACUMATICA)
and handshake (ONFLEET, STRAVA). The pkg/cmd tests serve the same
fixture, and command-level tests check that a rejected create or
upsert sends no request.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0172mwz23fhwEFwmLFdWoEbX
…rce is given inline

`connection upsert <name> --source-name X --source-type WEBHOOK` on an
existing connection, with no destination flags, failed with 422 "value
must contain at least one of [destination_id, destination]".

needsExistingConnection() skipped the lookup for that flag combination,
so buildUpsertRequest got existing == nil and its fallback that carries
the stored destination over by ID never ran. The same was true the
other way round: an inline destination with no source flags sent no
source.

The lookup now runs whenever either side is not named by the flags (no
--*-id, --*-name or --*-type). That replaces the no-flags and
config-only conditions, which it covers.

Tests:
- command-level tests against a stub API assert the PUT /connections
  body carries destination_id (inline source) and source_id (inline
  destination)
- needsExistingConnection cases for both sides and for --source-id
- acceptance test under the connection_upsert tag

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0172mwz1iqs4Qa9xmLagyE3v
…ection up

connection upsert took the first model returned by the connections list
`name` filter without comparing names. The API filter is exact and
case-sensitive today (checked against the live API), but its spec only
says "Filter by connection name", so a partial or case-different match
could silently select the wrong connection. Add Client.FindConnectionByName
which compares the name again.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
--webhook-secret, --hmac-secret and --api-key sent one generic shape to every type. The API
stores the keys a type's auth schema lists and lays them over the provider's defaults when it
verifies a request, so a generic header_key replaced VAPI's x-signature and FORMSTACK's
x-fs-signature. Types with a choice of method (GITLAB, GRAFANA, PROMETHEUS_ALERTMANAGER,
BETTER_STACK, ORDINAL) got an auth block with no auth_type and a 422.

The flags now read the type's methods from the spec: the secret alone unless the method
requires the HMAC settings (WEBHOOK and HTTP keep sha256/hex/x-webhook-signature), auth_type
set to the method used, and --api-key refused where the type needs its own header name.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
downloadOpenAPISpec used http.Get with no timeout, and source types and auth methods kept
separate caches, so a cold cache downloaded the 1.3 MB spec twice. The download now gives up
after 10 seconds, and each fetch also fills the other cache from the same download.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The help said a bare webhook_secret_key resolves to STANDARD_WEBHOOKS, but --webhook-secret
sends the HMAC shape on WEBHOOK; only --config reaches STANDARD_WEBHOOKS. It also named
WEBHOOK and HTTP as the types that infer auth_type, which overstated it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…estination change is ignored

The API never changes an existing connection's source or destination, but
connection upsert and update sent the change and printed success, and
upsert also left an inline destination behind unattached.

- upsert looks the connection up whenever a source or destination is
  named by ID or name, and fails before any write (including the inline
  destination upsert) if it differs from the stored one. Dry-run reports
  the same error.
- update checks --source-id and --destination-id against the stored
  connection and errors when they differ; matching values are accepted
  and not sent. The flags are kept rather than removed, because removing
  them would be a breaking change.
- the gateway_connections_write MCP tool applies the same check to its
  upsert and update actions.

Fixes #466

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e live API

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
connection upsert only set the request's rules when the built list was
non-empty, so --rules '[]' or an empty --rules-file left the stored
rules in place while reporting success. Key it off whether the flag was
given, as connection update does, and say in the dry run that the rules
will be cleared.

Fixes #472 (point 2; points 1 and 3 were fixed separately)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…t without --type

The code fix for dropped --url/--cli-path when --type is omitted is
already on this branch (the stored type is resolved once, for update and
upsert alike). What the issue asked for and was missing is the
acceptance test that reads the destination back: update --url, update
--cli-path, upsert --url, a negative control that an update without --url
leaves the URL alone, and a mismatched --url on a CLI destination that
must be refused rather than dropped.

Refs #406

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…uilder

webhook_secret, api_key, basic_auth and hmac in --config still produced the generic shape, so
webhook_secret on VAPI stored header_key x-webhook-signature and broke verification. They now go
through the same builder as the flags, with the same refusals and offline fallback. The webhook
secret flag descriptions no longer say e.g. Stripe.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tion

Checked against the live API after the destination guard landed: upsert
keys on source and name together, so naming a different source creates a
second connection with the same name rather than re-pointing the stored
one, and that connection may have any destination. Refusing it broke an
existing acceptance test and a working feature.

- SelectConnectionForUpsert picks the stored connection by the source the
  caller names, and reports when the source differs; the destination
  guard is skipped in that case (CLI and MCP).
- Several connections with the same name and no source named is now an
  error rather than silently picking the first.
- Reword the update and MCP text: update ignores a new source as well as
  a new destination, upsert only ignores a new destination.

Refs #466

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…erate REFERENCE.md

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ts own algorithm

Refusing the flag broke commands that worked before, such as
connection create --source-type TWILIO --source-hmac-algo sha1: the API
dropped the algorithm and the provider's own one applied.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@leggetter
leggetter merged commit eb89078 into main Nov 13, 2026
11 of 12 checks passed
@leggetter
leggetter deleted the claude/clever-bohr-87hc9u branch November 13, 2026 17:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment