Skip to content

Query: Select random redirect posts by looking up random post IDs. - #16050

Closed
aaronjorbin wants to merge 1 commit into
WordPress:trunkfrom
aaronjorbin:trac/66654
Closed

aaronjorbin wants to merge 1 commit into
WordPress:trunkfrom
aaronjorbin:trac/66654

Conversation

@aaronjorbin

Copy link
Copy Markdown
Member

Random content requests currently select the post with ORDER BY RAND() LIMIT 1. The database reads and sorts every eligible post for every request, so the cost grows with the size of the site.

The post is now selected by rejection sampling:

  • 50 random post IDs between 1 and the highest post ID are looked up by the primary key, limited to the posts eligible for the requested archive, and one of the eligible posts found is selected at random.
  • If none of the IDs are eligible posts, other random IDs are looked up, up to three times in total.
  • If no post is found, eligible posts are too sparse among the post IDs, for example a small category on a large site. A pool of up to 1,000 randomly ordered eligible posts is selected instead and stored in the object cache. It is used for 10 minutes.

Why this is better than the current approach:

  • The cost of a request no longer depends on the number of posts. In local testing, a request read 53 rows at both 100,000 and 10,000,000 posts, where ORDER BY RAND() read every eligible post. The median time to select a post at 10,000,000 posts fell from around 9 seconds to around 5 milliseconds.
  • It does not rely on a persistent object cache. Only the fallback for sparse archives is cached.
  • Every eligible post remains equally likely to be selected, whether or not there are gaps between post IDs. Every ID is equally likely to be looked up, so gaps only reduce how often a lookup finds a post. This was confirmed with 100,000 selections from posts with large gaps between their IDs.
  • The database is only read from. The object cache is used directly, without transients, so sites using a read-only database user for front end requests are unaffected.

The fallback for sparse archives is the only part that still reads every eligible post of an archive, so its cost is limited:

  • The pool is selected without excluding password protected posts, which are skipped when the post is selected from it. The post password is not indexed, so this allows the pool to be selected from an index rather than reading each post in full.
  • Only one request selects the pool at a time. Once the pool is 10 minutes old, the first request to need it selects the next pool, and other requests select from the previous pool until it has. If there is no previous pool, concurrent requests each select one.

Trac ticket: https://core.trac.wordpress.org/ticket/66654

AI assistance: Yes
Tool(s): Claude Code
Model(s): Fable 5.2
Used for: This change, its tests and the benchmarks behind the figures above were produced with AI assistance, directed and reviewed by me.


This Pull Request is for code review only. Please keep all other discussion in the Trac ticket. Do not merge this Pull Request. See GitHub Pull Requests for Code Review in the Core Handbook for more details.

Random content requests, such as `/random/`, currently select the post with `ORDER BY RAND() LIMIT 1`. The database reads and sorts every eligible post for every request, so the cost grows with the size of the site. As the redirect is not cached, extremely large sites can not safely enable the feature.

The post is now selected by rejection sampling:

* 50 random post IDs between 1 and the highest post ID are looked up by the primary key, limited to the posts eligible for the requested archive, and one of the eligible posts found is selected at random.
* If none of the IDs are eligible posts, other random IDs are looked up, up to three times in total.
* If no post is found, eligible posts are too sparse among the post IDs, for example a small category on a large site. A pool of up to 1,000 randomly ordered eligible posts is selected instead and stored in the object cache. It is used for 10 minutes.

Why this is better than the current approach:

* The cost of a request no longer depends on the number of posts. In local testing, a request read 53 rows at both 100,000 and 10,000,000 posts, where `ORDER BY RAND()` read every eligible post. The median time to select a post at 10,000,000 posts fell from around 9 seconds to around 5 milliseconds.
* It does not rely on a persistent object cache. Only the fallback for sparse archives is cached.
* Every eligible post remains equally likely to be selected, whether or not there are gaps between post IDs. Every ID is equally likely to be looked up, so gaps only reduce how often a lookup finds a post. This was confirmed with 100,000 selections from posts with large gaps between their IDs.
* The database is only read from. The object cache is used directly, without transients, so sites using a read-only database user for front end requests are unaffected.

The fallback for sparse archives is the only part that still reads every eligible post of an archive, so its cost is limited:

* The pool is selected without excluding password protected posts, which are skipped when the post is selected from it. The post password is not indexed, so this allows the pool to be selected from an index rather than reading each post in full.
* Only one request selects the pool at a time. Once the pool is 10 minutes old, the first request to need it selects the next pool, and other requests select from the previous pool until it has. If there is no previous pool, concurrent requests each select one.

On sites with very few posts, the request makes one more query than before.

Trac ticket: https://core.trac.wordpress.org/ticket/66654

AI disclosure: This change, its tests and the benchmarks behind the figures above were produced with AI assistance (Claude Code, Claude Fable 5.2), directed and reviewed by Aaron Jorbin.

See #66654.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs.

Core Committers: Use this line as a base for the props when committing in SVN:

Props jorbin.

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.


Props Bot commands

You can trigger Props Bot actions by commenting on this PR:

  • @propsbot add <username> to give props to a GitHub or WordPress.org user. Requires write access.
  • @propsbot remove <username> to leave a user out of the props. Requires write access.
  • @propsbot refresh to refresh this list. Adding the props-bot label works too.

@github-actions

Copy link
Copy Markdown

Test using WordPress Playground

The changes in this pull request can previewed and tested using a WordPress Playground instance.

WordPress Playground is an experimental project that creates a full WordPress instance entirely within the browser.

Some things to be aware of

  • All changes will be lost when closing a tab with a Playground instance.
  • All changes will be lost when refreshing the page.
  • A fresh instance is created each time the link below is clicked.
  • Every time this pull request is updated, a new ZIP file containing all changes is created. If changes are not reflected in the Playground instance,
    it's possible that the most recent build failed, or has not completed. Check the list of workflow runs to be sure.

For more details about these limitations and more, check out the Limitations page in the WordPress Playground documentation.

Test this pull request with WordPress Playground.

wporg-sync pushed a commit that referenced this pull request Nov 17, 2026
Change Random content requests from using `ORDER BY RAND() LIMIT 1` to using a sampling with rejection method. The post is now selected by looking up random post IDs by primary key, which keeps the cost of a request independent of the number of posts while leaving every eligible post equally likely to be selected.

Archives whose posts are too sparse among the post IDs to be found this way fall back to a pool of randomly ordered posts stored in the object cache.

Developed in: #16050

Follow-up to r64402, r65042.

Props jorbin, peterwilsoncc, sukhendu2002, rmccue, gddrt, renehermi, djpaul.
Fixes #66654. See #64498.


git-svn-id: https://develop.svn.wordpress.org/trunk@65542 602fd350-edb4-49c9-b593-d223f7449a82
@github-actions

Copy link
Copy Markdown

A commit was made that fixes the Trac ticket referenced in the description of this pull request.

SVN changeset: 65542
GitHub commit: 3943b1d

This PR will be closed, but please confirm the accuracy of this and reopen if there is more work to be done.

@github-actions github-actions Bot closed this Nov 17, 2026
wporg-sync pushed a commit to WordPress/WordPress that referenced this pull request Nov 17, 2026
Change Random content requests from using `ORDER BY RAND() LIMIT 1` to using a sampling with rejection method. The post is now selected by looking up random post IDs by primary key, which keeps the cost of a request independent of the number of posts while leaving every eligible post equally likely to be selected.

Archives whose posts are too sparse among the post IDs to be found this way fall back to a pool of randomly ordered posts stored in the object cache.

Developed in: WordPress/wordpress-develop#16050

Follow-up to r64402, r65042.

Props jorbin, peterwilsoncc, sukhendu2002, rmccue, gddrt, renehermi, djpaul.
Fixes #66654. See #64498.

Built from https://develop.svn.wordpress.org/trunk@65542


git-svn-id: http://core.svn.wordpress.org/trunk@64615 1a063a9b-81f0-0310-95a4-ce76da25c4cd
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant