Skip to content

perf(scripts): skip the per-byte loop in bash json_escape when there is nothing to escape - #5430

Open
b84330808 wants to merge 1 commit into
github:mainfrom
b84330808:fix/bash-json-escape-fast-path
Open

b84330808 wants to merge 1 commit into
github:mainfrom
b84330808:fix/bash-json-escape-fast-path

Conversation

@b84330808

Copy link
Copy Markdown
Contributor

Description

When jq is not installed, the bash helpers build their --json output with json_escape from scripts/bash/common.sh. After the \\, ", \n, \t, \r, \b and \f replacements, it walks the whole value one byte at a time (substring, printf -v, compare, printf) to catch the remaining U+0001–U+001F characters. That is cheap for paths, but setup-tasks.sh, resolve-template.sh and check-prerequisites.sh --template pass the whole template content through it, so for the stock tasks template the loop runs about 9,000 times on every /speckit.tasks.

This change adds a fast path: if the value has no control byte left ([[ "$s" != *[[:cntrl:]]* ]], checked under the existing LC_ALL=C), it prints the value as is and returns. Otherwise the existing loop runs unchanged. [[:cntrl:]] also matches DEL (0x7f), which takes the loop as before. The same change goes into the copy in extensions/github/scripts/bash/resolve-tasks.sh, which is kept in step with core.

The output does not change. Base and patched json_escape give byte-identical output on every file under templates/ and templates/commands/, on strings with each control byte 0x01–0x1F and 0x7F mixed with backslashes, quotes and multi-byte UTF-8, and on an empty string. The --json output of setup-tasks.sh, resolve-template.sh (spec and constitution templates), check-prerequisites.sh (--template checklist-template and --paths-only) and setup-plan.sh is also byte-identical between base and patched.

Numbers

Windows 11, Git for Windows bash 5.2.37 (specify init --script sh), no jq on PATH. Two copies of the same one-feature project, one with the original common.sh and one with the patched one. Base and patched runs alternate, 7 runs each after a warm-up run; wall time of the whole script:

Script (command) base, median ms (min–max) patched, median ms (min–max)
setup-tasks.sh --json (/speckit.tasks) 2126 (1987–2339) 698 (587–1039)
resolve-template.sh spec-template --json (/speckit.specify) 774 (700–867) 290 (227–298)
resolve-template.sh constitution-template --json (/speckit.constitution) 479 (436–536) 291 (275–360)
check-prerequisites.sh --json --template checklist-template (/speckit.checklist) 768 (715–827) 626 (557–677)

json_escape alone on the stock tasks template (one bash process per run, so about 90 ms of bash startup is included): 1336 ms base vs 102 ms patched (medians of 7).

On GitHub Actions (my fork), alternating base/patched runs, medians of 7, no jq:

Base Patched
Linux (bash 5.3), setup-tasks.sh --json 336 ms 55 ms
Linux, resolve-template.sh spec-template --json 126 ms 14 ms
Linux, resolve-template.sh constitution-template --json 63 ms 14 ms
Linux, check-prerequisites.sh --json --template checklist-template 75 ms 33 ms
macOS /bin/bash 3.2, json_escape on the tasks template 2196 ms 1162 ms
macOS /bin/bash 3.2, json_escape on the spec template 504 ms 202 ms

The base and patched outputs were byte-identical on both runners for all 49 inputs, and the repo's json_escape tests pass there.

The cost grows with template size, so presets that append to a template gain more.

Testing

  • tests/extensions/github/test_github_extension.py: 200 passed, 5 skipped (Windows 11, Python 3.14, Git Bash). I added three cases to the existing json_escape round-trip and core-parity test: ESC and DEL (loop path) and non-ASCII text (fast path). They pass with and without the change.

  • Bash helper tests (test_setup_tasks.py, the *_python_parity.py tests for resolve-template, check-prerequisites, setup-plan and create-new-feature, test_timestamp_branches.py, test_setup_plan_feature_json.py, test_check_prerequisites_paths_only.py, tests/extensions/git, non-PowerShell cases): the same 430 pass before and after. The same 126 fail before and after on this Windows machine (Git Bash vs Windows path spellings in the parity checks), so they are unrelated to this change.

  • Byte-for-byte comparison of base vs patched json_escape, plus a JSON round trip of the patched output, on the inputs described above.

  • shellcheck --severity=error on both scripts: clean. ruff check on the test file: clean.

  • Tested locally with .venv/bin/specify --help (Windows PowerShell: .\.venv\Scripts\specify.exe --help)

  • Ran relevant existing tests with .venv/bin/python -m pytest tests -q (Windows PowerShell: .\.venv\Scripts\python.exe -m pytest tests -q); use affected test paths for focused checks and LC_ALL=en_US.UTF-8 on macOS

  • Tested with a sample project (if applicable)

AI Disclosure

  • I did not use AI assistance for this contribution
  • I did use AI assistance (fill in the disclosure below)

AI disclosure: Opened on behalf of @b84330808. Found by profiling the bash helpers, then implemented, benchmarked and written up by Claude Code (model: claude-opus-5-5) running autonomously: code, tests, benchmarks and this description.

…g is left to escape

Without jq, setup-tasks.sh, resolve-template.sh and check-prerequisites.sh
--template pass whole template content through json_escape, whose
byte-by-byte loop for U+0001-U+001F takes most of their run time (about
2.1 s for setup-tasks.sh under Git Bash on Windows). When the value has no
control byte left, print it directly; values that still have one take the
existing loop, so the output is unchanged.

Apply the same change to the copy in the github extension's
resolve-tasks.sh, and add ESC, DEL and non-ASCII cases to its round-trip
and core-parity test so both paths stay covered.

Assisted-by: Claude Code (model: claude-opus-5-5, autonomous)
@b84330808
b84330808 requested a review from mnriem as a code owner November 14, 2026 23:31
Copilot AI dynamic review requested due to automatic review settings November 14, 2026 23:31

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues are recorded for the Bash JSON escaping optimization.

No open issues 🎉

What changed in this PR

This PR speeds up Bash JSON output when jq is unavailable by skipping byte-by-byte escaping when no control characters remain.

Changes:

  • Add the same fast path to the core and GitHub extension helpers, preserving the existing control-byte fallback.
  • Extend JSON round-trip and core-parity tests with ESC, DEL, and non-ASCII inputs.
File Description
tests/​extensions/​github/​test_github_extension.py Add control-byte and UTF-8 test cases.
scripts/​bash/​common.sh Skip the escaping loop when no control bytes remain.
extensions/​github/​scripts/​bash/​resolve-tasks.sh Mirror the core JSON escaping optimization.

Review effort: Balanced (dynamic)

@mnriem mnriem added the triage-nice-to-have Verdict: evidence-backed fix or greenlit feature — land after review label Nov 15, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

triage-nice-to-have Verdict: evidence-backed fix or greenlit feature — land after review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants