Migrate your package from Level 4 to Level 5

Overview

In this guide, you will learn to migrate your package from Level 4 to Level 5. Once you have your package up to the Level 5 standard, you can share your code with the world.

What’s the difference between Level 4 and Level 5?

Besides the final goal of releasing your package, you will also have the following features:

  • Develop documentation with Sphinx with live loading.

  • Host documentation on a public URL with GitHub Pages.

  • Use GitHub tags to release your package to GitHub and PyPI.

  • Maintain changelogs and release notes automatically for each version.

Prerequisites

We assume you have already completed and created your scientific code in Level 4, where you have a lightweight Python package that can be installed locally and have your project hosted on GitHub. .. include:: ../snippets/scikit-installation.rst

Create a new Level 5 project and migrate files

The first step is to create a new project with scikit-package using the Level 5 public template. Then we will migrate the files and folders from the existing Level 4 project to the new Level 5 project. Let’s begin!

Create a new project with scikit-package

  1. Visit your project directory and sync with the latest version of the main branch:

    $ cd <project-name>
$ git checkout main
$ git pull origin main

  2. Create a new project with scikit-package using the Level 5 public template:

    $ package create public

    Important

    We will not run git add or git commit on the main branch at this stage. You will create a new branch called skpkg-public in a later section, as described in (Level 5) Share your code as a publicly installable package. Don’t worry! You will be guided through the process.

  3. Answer the following questions:

    Prompt

    Description and example

    maintainer_name

    The name of the project maintainer. This person will make the public releases. e.g., Simon Billinge

    maintainer_email

    The maintainer’s email address. e.g., sbillinge@columbia.edu

    maintainer_github_username

    The maintainer’s GitHub username. e.g., sbillinge

    contributors

    Individuals or groups contributing to the project. e.g., Sangjoon Lee, Simon Billinge, Billinge Group members

    license_holders

    The license holders listed in LICENSE.rst. e.g., The Trustees of Columbia University in the City of New York

    project_name

    The name displayed in the README.rst and documentation. Use name-with-hyphens e.g., my-package. To support namespace imports, see FAQ

    github_username_or_orgname

    The GitHub username or organization name. e.g., sbillinge or billingegroup

    github_repo_name

    The GitHub repository name. Use name-with-hyphens e.g., my-package

    conda_pypi_package_dist_name

    The name used for publishing to PyPI and conda-forge. Use name-with-hyphens e.g., my-package

    package_dir_name

    The name of the package directory under src. This name will be used in when importing modules from this package. Use name_with_underscores e.g., my_package

    project_short_description

    A brief description of the project, shown in pyproject.toml. e.g., A Python package standard for scientific code

    project_keywords

    A list of keywords included in pyproject.toml. e.g., PDF, diffraction, neutron, x-ray

    min_python_version

    The minimum supported Python version. e.g. 3.11. Current practice (SPEC 0 specification) is to drop support for Python versions that were released more than 72 months ago.

    max_python_version

    The maximum supported Python version e.g. 3.13. Ideally, the current latest version.

    needs_c_code_compiled

    Specifies whether C code compilation is required. For pure Python packages, the default value is No.

    has_gui_tests

    Specifies whether GUI tests are included. For most packages, the default value is No but will be yes if your app has a gui and there are tests that run the GUI.

  4. Enter into the Level 5 project directory:

    $ cd <package-name>

  5. Check that you have the following nested folder structure. Here is the structure. We will go through each file and folder:

    <package-name>     # (Level 4)
└── <package-name> # (Level 5)
    ├── AUTHORS.rst
    ├── CHANGELOG.rst
    ├── CODE_OF_CONDUCT.rst
    ├── LICENSE.rst
    ├── MANIFEST.in
    ├── README.rst
    ├── doc
    ├── news
    ├── pyproject.toml
    ├── requirements
    ├── src
    └── tests
├── LICENSE.rst
├── README.md
├── pyproject.toml
├── requirements
├── src
└── tests

Migration files from Level 4 to Level 5

  1. Enter into the nested Level 5 project directory:

    $ cd <package-name>

  2. Move the local git repository from the Level 4 (..) to the Level 5 folder (.):

    $ mv ../.git .

  3. Move the src and tests folders from Level 4 to Level 5:

    $ cp -n -r ../src .
$ cp -n -r ../tests .

  4. Copy the requirements files from Level 4 to Level 5:

    $ cp ../requirements/conda.txt ./requirements/conda.txt
$ cp ../requirements/pip.txt ./requirements/pip.txt
$ cp ../requirements/test.txt ./requirements/test.txt

  5. At this point, you should be able to install the package locally and test it using your existing conda environment:

    $ conda activate  <package-name>-env
$ pytest tests

  6. Once the tests pass, let’s manually migrate hand-written files like README.md from Level 4 to Level 5.

    Note

    In Level 5, we provide a rich template for README.rst instead of using README.md. If you already had a rich README.md in Level 4, you can use a tool to convert .md to .rst. For example, you may use this free CloudConvert tool.

Build documentation locally

/doc is the the Sphinx documentation folder. The documentation will be built locally first and then automatically built and hosted on GitHub Pages when a new release is created.

  1. Install documentation related dependencies:

    $ conda install --file requirements/docs.txt

  2. Enter into the doc project directory and render documentation:

    $ cd doc
$ make html

  3. Open the rendered documentation via web browser:

    $ open _build/html/index.html

  4. Here is a shortcut if you want to use it from the root directory of the project:

    $ cd doc && make html && open _build/html/index.html && cd ..

    See also

    You can use a alias shortcut. Open ~/.bashrc in your text editor and add the following line:

    alias doc='cd doc && make html && open _build/html/index.html && cd ..'

    Apply the changes to your current terminal session:

    $ source ~/.bashrc

    Now, you can simply enter the doc command in your terminal to build and open the documentation:

    $ doc

(Optional for macOS/Linux only) Do you want to re-render documentation without running doc command every time? You can use sphinx-reload.

  1. Install the dependencies including sphinx-reload sourced from PyPI:

    $ conda install --file requirements/docs.txt
$ pip install sphinx-reload

  2. Run the following command to start live-reloading:

    $ sphinx-reload doc

  3. Now, each time you make changes to the documentation, it will be automatically reloaded in your web browser.

What’s next?

Let’s setup advanced GitHub Actions to automate code linting and testing and upload to the GitHub repository using a pull request from the skpkg-public branch to main. Please continue the rest of the Level 5 tutorial starting from Step 2. Automate code linting and testing with GitHub Actions.