Read the Docs Blog - Posts tagged builders

Build errors with docutils 0.18

2021-11-02T00:00:00Z

Starting about a week ago, some users started reporting new errors with their project builds. In most cases, these errors appeared out of nowhere and are usually rather cryptic errors referencing Sphinx and docutils.

So, what is happening?

These errors are due to an incompatibility bewteen old versions of Sphinx and one of its dependencies, docutils. You probably have not directly interacted with docutils, but this is the library that provides reStructuredText parsing to both Sphinx and to the Python standard library documentation tooling.

There are a number of errors that users have reported due to this incompatibility, but the most common error is:

TypeError: 'generator' object is not subscriptable

If your project’s builds suddenly start failing with the above error message, or other cryptic errors that don’t seem related to your project, your project is most likely encountering this bug.

Why is this happening?

The underlying cause for this error is using an outdated version of Sphinx (version < 3.0) with the latest release of docutils (version 0.18, released October 26). This latest release of docutils is no longer comptible with Sphinx versions 1.x and 2.x, however these versions do not specify a upper limit to the version of docutils installed, and so will install the latest release by default.

The reason so many projects use an old version of Sphinx is that, before October 2020, Read the Docs had a more strict pinning policy and Sphinx 1.8 was the default version. New projects are not affected.

Fixing the error

We have proposed to the Sphinx and docutils maintainers a solution that doesn’t involve manual intervention. However, for now we suggest you manually resolve this issue for any of your projects that have encountered the bug.

There are two ways to resolve this issue:

Upgrade Sphinx to version 3 or later

We recommend this option for most users. If your project doesn’t specifically still require Sphinx 1.x or 2.x, your project can most likely be upgraded, without issues, to a modern version of Sphinx. As a bonus, you’ll be able to use the latest Sphinx releases and extensions that only support modern Sphinx releases.

It is worth noting that one reason why you might still be using an old version of Sphinx is if you generate API documentation for Python 2 compatible code. In this case, you’ll have more work ahead of you in order to upgrade, as Sphinx version 3 and 4 only support Python 3.

Downgrade docutils

This is the next best option if you can’t immediately upgrade Sphinx. If you are familiar with Python packaging, the best specifier to use is docutils<0.18.

Fixing the error on Read the Docs

If you aren’t already, you should add a configuration file for your project, so that you can specify additional dependencies to install at build time. There are a few supported methods for installation, depending on how you normally install dependencies for your project. If you are not currently specifying any dependencies on Read the Docs, you most likely will want to use the python.install.requirements configuration option.

This is how the reference .readthedocs.yaml and docs/requirements.txt files would look like:

.readthedocs.yaml

version: 2

python:
  install:
  - requirements: docs/requirements.txt

docs/requirements.txt

docutils<0.18

If you still experience problems, feel free to open an issue.

Ubuntu 20.04, Python 3.10, and support for Node, Rust, and Go

2021-10-14T00:00:00Z

We are excited to announce that now Read the Docs users can use a newer build specification in their projects that will change the base image to one based on Ubuntu 20.04, ship the recently released Python 3.10, and allow users to easily specify the version of Node.js, Rust, and Go. This feature has been a long time in the making, and we think it will simplify the configuration of many projects.

Previous situation

The Docker images used by our builders were based on Ubuntu 18.04. Recently, we added a new feature to install custom system packages, which allowed many projects to have better control of their build process without having to use conda to manage non-Python dependencies.

However, this Ubuntu version was released three years ago and, even though it was good enough for most projects, we have been receiving requests to upgrade certain system packages over the years. Since these are managed through the operating system package manager, and we do not provide a way to add custom Personal Package Archives, such upgrades were not possible.

Many Python projects use a secondary programming language for a variety of reasons. Established compiled languages like C, C++, and FORTRAN have readily available compilers that will work for the majority of use cases. However, other younger technologies move at a faster pace, and developers often want specific versions of the toolchains that cannot be installed with the system package manager. This affected projects that combine backend and frontend code that need Node.js to compile their JavaScript sources, such as Jupyter extensions, and libraries with performance-critical code written in Rust or Go.

We have often suggested specifying custom versions of non-Python dependencies using conda, to specify custom versions of non-Python dependencies, but this requires learning another tool that might be unfamiliar for some projects.

New `build` YAML configuration

To overcome all these problems, we have added a new configuration value, build.tools, that receive a dictionary of toolchain versions. This new configuration requires specifying the base image name in build.os, currently ubuntu-20.04. Our configuration documentation contains a simple example:

version: 2

# Set the version of Python and other tools you might need
build:
  os: ubuntu-20.04
  tools:
    python: "3.9"
    # You can also specify other tool versions:
    # nodejs: "16"
    # rust: "1.55"
    # golang: "1.17"

And you can draw inspiration from some community projects that are using this feature already:

cryptography, the reference Python library for cryptographic algorithms, can now specify the Rust version required to build the package.
parsel, a Python library to manipulate HTML and XML using XPath and CSS selectors, leverages the new specification to build their documentation on the just released Python 3.10.
Thebe, a Jupyter-powered tool to make HTML pages interactive, needs Node.js to build the latest version of the code.

Do you think this feature will be useful for you as well? Give it a try and let us know.

Remember you can always see the latest changes to our platforms in our Read the Docs Changelog.

—

Considering using Read the Docs for your next Sphinx or MkDocs project? Check out our documentation to get started!

Install custom operating system packages (apt)

2021-05-19T00:00:00Z

We are thrilled to announce that now Read the Docs users can declare custom operating system packages in their project configuration that will get installed in our Ubuntu-based builders using apt. This has been a long awaited feature, and we think it will simplify the configuration of many projects, especially scientific ones.

Previous solutions

The Ubuntu images used by our builders contain lots of preinstalled system packages that we ship to all the projects to make the most common use cases possible. This includes compilers, development headers of common libraries, and others.

However, on one hand this makes our images way bigger than necessary, and it is still not enough to solve 100 % of the use cases. Fortunately, Read the Docs has supported the conda package manager for a long time already, and this has allowed users with special needs to add any non-Python libraries they needed. Still, interoperability between conda and pip is not perfect, and for users that are not used to conda this could feel like a hack.

New `apt_packages` configuration

To overcome all these problems, we have added a new configuration value, build.apt_packages, that receive a list of APT packages that will be installed in our Ubuntu-based images. Our configuration documentation contains a simple example:

version: 2

build:
  image: latest
  apt_packages:
    - libclang
    - cmake

(Notice that at the moment our images are based Ubuntu 18.04, this will change in the near future)

And you can draw inspiration from some community projects that are using this feature already:

geoserver-rest, a Python library to interact with GeoServer, uses apt_packages to install some GDAL and pycurl dependencies.
UCX-Py, a Python interface for the low-level networking library UCX, installs extra dependencies to temporarily work around upstream issues.

We are happy that this feature is already being used minutes after being deployed and are looking forward to seeing more projects make use of it.

Remember you can always see the latest changes to our platforms in our Read the Docs Changelog and Ethical Ad Server Changelog.

Considering using Read the Docs for your next Sphinx or MkDocs project? Check out our documentation to get started!

Better support for scientific project documentation

2020-04-28T00:00:00Z

In the past year, we’ve been having issues when building projects’ documentation using conda. Our build servers were running out of memory and failing projects’ builds. Read the Docs has this memory constraint to avoid misuse of the platform, causing a poor experience for other users.

Our first solution was a dedicated server for these kind of projects. We would manually assign them to this server on user requests. This workaround worked okay, but it involves a bad experience for the user and also us doing a manual step each time. Over time, we hit again the same issue of OOM, even giving all the memory available to one project to build its documentation. After some research, we found that this is a known issue in the conda community and there are some different attempts to fix it (like mamba). Unfortunately, none of them became the standard yet and the problem is still there.

Meanwhile, in another completely different set of work, we were migrating all of our infrastructure to a different architecture:

Azure Virtual machine scale sets for autoscaling our servers,
Azure storage accounts to store all the user’s documentation
Proxito, an internal service to remove a lot of the state from our servers (more about this migration coming in a future post)

This helped us to reduce costs and allowed us to spin up bigger instances for the builders. We have also made some other important operational changes:

Our builders are now single-process, giving all the memory available to only one project without worrying about affecting others.
We added custom task router that routes small builds to small servers (3GB RAM), and big builds to larger servers (7GB RAM). This removes the need for users to ask us to upgrade their isntances.
Assigned all conda projects to be built by big servers by default.

If you ever had a memory issue on Read the Docs, we’d appreciate if you give it a try again. Please let us know about your experience when building scientific documentation. If you know that any of your friends were hitting this issue in the past, we encourage you to talk to them and tell them to give it a try.

We are excited to have more scientific documentation on our platform and we are doing our small part to make this happen and have better science in the world.