Skip to content

docs: add guide on including tests in package distributions - #821

Open
InessaPawson wants to merge 4 commits into
pyOpenSci:mainfrom
InessaPawson:343-treetests
Open

InessaPawson wants to merge 4 commits into
pyOpenSci:mainfrom
InessaPawson:343-treetests

Conversation

@InessaPawson

@InessaPawson InessaPawson commented Oct 11, 2026 •

Copy link
Copy Markdown
Contributor

Adds a guide on distributing tests with a package, covering:

  • including tests in the sdist only
  • shipping tests inside the package (in-tree tests), with how to lay them out and run them

Closes #343.

@netlify

netlify Bot commented Oct 11, 2026 •

Copy link
Copy Markdown

👷 Deploy request for python-package-guide pending review.

Visit the deploys page to approve it

Name Link
🔨 Latest commit b76c438

@tkoyama010 tkoyama010 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@lwasser

lwasser commented Oct 21, 2026

Copy link
Copy Markdown
Member

This is great. i'd love @ucodery to have a look. We talked a lot about non-pure Python packages and cases where you want to ship tests. i just want another set of eyes on this one if possible as I am not the expert here but recall many long debates about this. !!

@lwasser
lwasser requested a review from ucodery October 21, 2026 20:48

@ucodery ucodery left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This guide looks great. My only two followups, not blocking

Even when a package does choose to include tests, it is often done as part of the sdist building, yet still left out of any wheels. If the tests aren't included in wheels, this also allows the tests to remain out-of-tree and then explicitly added for the sdist step (I believe all backends could be configured to do this, setuptools explicitly outlines how to do this).

Smaller nit, pytest is only required if the package is using pytest imports. If the package is using unittest, It would be better to not include an extras requirement for end users, even if the development workflow uses pytest as a test runner. (or if some other framework, that would need to be the requirement)

@InessaPawson
InessaPawson force-pushed the 343-treetests branch 3 times, most recently from 86811f8 to 7b2baf7 Compare November 11, 2026 01:37

@ucodery ucodery left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this this explainer is great! Often being one of those downstream packagers, this is all the best advice we can give, with a lot of supporting arguments and examples. It is a bit in the weeds, but I would hope that would enlighten those who seek an answer to the question of shipping tests.

One final quibble, the page is still called in-tree tests, but it is more about including tests in distributions, in which in-tree is just one way.

```text
graft tests
```
This does not itself control wheel contents: ensure `tests/` is not selected

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we need to ask a packaging learner to ensure that a sibling tests isn't installed by a wheel build? That's pretty technical, and if they are following all our guidance, even these optional bits, it won't get included.

Comment thread tests/in-tree-tests.md Resolved
@InessaPawson InessaPawson changed the title docs: add guide on in-tree tests for packages with compiled code docs: add guide on including tests in package distributions Nov 17, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

in-tree tests

4 participants