Skip to content

perf(Cache): atomic HMSET and EXPIREAT using Lua script in Redis and Predis handlers - #10707

Open
gr8man wants to merge 3 commits into
codeigniter4:developfrom
gr8man:perf/redis-pipeline-save
Open

gr8man wants to merge 3 commits into
codeigniter4:developfrom
gr8man:perf/redis-pipeline-save

Conversation

@gr8man

@gr8man gr8man commented Nov 1, 2026 •

Copy link
Copy Markdown
Contributor

Description
In both RedisHandler::save() and PredisHandler::save(), saving a cache item with a TTL previously executed two sequential network round trips:

  1. hMset (hmset)
  2. expireAt (expireat)

This PR executes both commands atomically in a single network round trip using a small, self-contained Lua script (EVAL):

  • Atomicity & Type Safety: Uses redis.pcall('HMSET', ...) inside Lua. If the existing key is not a hash (WRONGTYPE), the script immediately returns 0 without executing EXPIREAT, leaving the existing key's value and TTL intact.
  • Fail-safe Fallback: If EVAL is blocked or unsupported in a specific environment, a try/catch block seamlessly falls back to sequential execution (hMset followed by expireAt).
  • Regression Tests: Includes regression coverage for non-hash key preservation and for the EVAL-denied fallback path in both handlers.

Benchmark (5,000 save() operations, local Redis instance)

  • Sequential (2 round trips: hMset + expireAt): 0.549 s (~0.110 ms per save)
  • Atomic Lua script (1 round trip: EVAL): 0.424 s (~0.085 ms per save)
  • Local speedup: 1.30x (22.8% latency reduction). Over distributed/cloud networks with non-zero ping, eliminating the second round trip yields an estimated ~50% network latency reduction per save.

Checklist:

  • Securely signed commits
  • Component(s) with PHPDoc blocks, only if necessary or adds value (without duplication)
  • Unit testing, with >80% coverage
  • User guide updated
  • Conforms to style guide

@carson-codeigniter4 carson-codeigniter4 Bot added the enhancement PRs that improve existing functionalities label Nov 1, 2026
@paulbalandan
paulbalandan requested a balanced review from Copilot November 5, 2026 13:26

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Redis and Predis saves can apply expiration even when HMSET fails, changing an existing key’s TTL or potentially deleting it.

1 open issue
What changed in this PR

This PR reduces network round trips when saving expiring cache entries through CodeIgniter’s Redis and Predis handlers.

Changes:

  • Pipelines HMSET and EXPIREAT for nonzero TTLs.
  • Retains the existing hash format and single-command path for zero TTLs.

Compatibility was assessed under the stricter develop policy because the base branch was unspecified. Runtime validation was left to CI.

File Description
system/​Cache/​Handlers/​RedisHandler.php Uses a phpredis pipeline for writes with expiration and checks the write result.
system/​Cache/​Handlers/​PredisHandler.php Uses a Predis pipeline for writes with expiration and checks the returned status.

Review effort: Balanced

Comment thread system/Cache/Handlers/RedisHandler.php Resolved
@gr8man
gr8man force-pushed the perf/redis-pipeline-save branch from 7de6a6d to 92844d1 Compare November 5, 2026 19:55

@paulbalandan paulbalandan left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please update the title and description to reflect the Lua implementation. The current description still describes pipelining and says Lua was deliberately avoided. Please also rerun the benchmark against the final implementation; the pipeline measurements do not establish the performance of this version.

Please add regression coverage for the EVAL-denied fallback in both handlers.

Lastly, please also consider whether using cached EVALs makes sense here (i.e. EVALSHA).

Send HMSET and EXPIREAT together in a single pipeline round trip rather
than issuing two sequential network requests on every cache save.

For normal cache usage, behavior and data structures remain identical.
In error scenarios where HMSET fails (e.g. key contains a non-hash type),
the pipelined EXPIREAT will still execute on the existing key, while
save() returns false as expected.
@gr8man
gr8man force-pushed the perf/redis-pipeline-save branch from 92844d1 to adaee6c Compare November 13, 2026 22:17
@gr8man gr8man changed the title perf: pipeline HMSET and EXPIREAT in Redis and Predis cache handlers perf(Cache): atomic HMSET and EXPIREAT using Lua script in Redis and Predis handlers Nov 13, 2026
@gr8man

gr8man commented Nov 13, 2026

Copy link
Copy Markdown
Contributor Author

Thanks @paulbalandan for the review and the detailed pointers!

1. Title & description updated

The PR title and description now reflect the actual implementation (atomic Lua script with EXPIREAT), replacing the outdated pipelining narrative.

2. Benchmark rerun against the final implementation

Measured on 5,000 save() operations against a local Redis instance:

Variant Total Per operation
Sequential (hMset + expireAt, 2 RTT) 0.549 s ~0.110 ms
Atomic Lua (EVAL, 1 RTT) 0.424 s ~0.085 ms

Local speedup: 1.30x (22.8% latency reduction). On distributed deployments where Redis sits on a separate host, the saved round trip translates to roughly a 50% reduction of network latency per save.

3. Regression coverage for the EVAL-denied fallback

Added tests in both handlers that replace the client with one whose eval() throws, then assert that save() still returns true, the value is stored, and the handler remains usable:

4. Re: EVALSHA

I considered it, but I would keep plain EVAL here:

  • The script is only ~160 bytes, so it fits in a single TCP frame; the payload saving is negligible compared to an extra round trip.
  • EVALSHA requires handling the NOSCRIPT error and retrying, and the exception shape differs between phpredis and predis. That adds a second network round trip on every cache flush/restart, plus cache-management complexity on the client side.
  • The main benefit of the change (one round trip instead of two) is already achieved by EVAL; EVALSHA would only shave off a constant-sized payload, not a network round trip.

Happy to revisit EVALSHA if you'd prefer it for very large deployments, but I'd avoid the added failure modes in the framework itself.

…is handlers

Execute HMSET and EXPIREAT atomically via a Lua script so that EXPIREAT is
never executed when HMSET fails (e.g. WRONGTYPE error on non-hash keys).
Falls back to sequential execution if EVAL is unsupported or blocked.
Adds regression tests verifying non-hash keys preserve value and TTL.
@gr8man
gr8man force-pushed the perf/redis-pipeline-save branch from adaee6c to 061a1f3 Compare November 13, 2026 22:37
Comment thread system/Cache/Handlers/RedisHandler.php Outdated
Comment on lines +157 to +161
return (bool) $this->redis->eval(
self::SAVE_SCRIPT,
[$key, $expiresAt, $dataType, (string) $value],
1,
);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When EVAL is disabled via rename-command, phpredis returns false rather than throwing, so the fallback is skipped. Could you handle this case and add a regression test where the mocked eval() returns false? Please distinguish it from integer 0, which indicates a failed HMSET inside the script.

Comment thread system/Cache/Handlers/PredisHandler.php Outdated
Comment on lines +35 to +37
local res = redis.pcall('HMSET', KEYS[1], '__ci_type', ARGV[2], '__ci_value', ARGV[3])
if type(res) == 'table' and res.err then
return 0

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This changes Predis error handling: a WRONGTYPE error previously raised a ServerException, but now returns false for non-zero TTLs. TTL 0 still throws. Please preserve the existing exception behavior.

…avior

RedisHandler: treat `false` from phpredis EVAL (e.g. EVAL disabled via
`rename-command`) as unavailable and fall back to sequential hMset and
expireAt. Integer 0 still means HMSET failed inside the script and must
not trigger the fallback.

PredisHandler: call HMSET and EXPIREAT with redis.call so a WRONGTYPE
error aborts the script before EXPIREAT and is propagated to the client
as ServerException again.

Add regression tests for both behaviors.
@gr8man
gr8man requested a review from michalsn November 14, 2026 23:17

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement PRs that improve existing functionalities

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants