Read the Docs Blog - Posts tagged grant

Announcing Embed API v3 and sphinx-hoverxref 1.0

2021-12-07T00:00:00Z

We are thrilled to announce the availability of Read the Docs Embed API v3, along with its official client, sphinx-hoverxref 1.0. This work has been possible in part thanks to the the CZI grant we received.

The value of content embedding

As we wrote in our first blog post about sphinx-hoverxref, one of the most powerful features of Sphinx is the possibility of creating cross references to other documentation projects. However, a reader finding several links in a technical documentation might need to open several browser tabs to fully understand the context, resulting in a lot of friction in the form of context switching.

One way of mitigating the problem is to show a small preview in the form of tooltip or modal with the contents of the link. This is where the Embed API and sphinx-hoverxref come in.

sphinx-hoverxref is a Sphinx extension that addresses this problem by adding tooltips on the cross references of the HTML documentation with the content of the linked section. To do so, it inserts a small JavaScript file that calls the Read the Docs Embed API when the user hovers any link. Back in June we announced intersphinx support in sphinx-hoverxref 0.6b1, which allowed users to include modals and tooltips also for cross references to other projects.

sphinx-hoverxref displaying a tooltip including an equation

Some big projects already using it include Tweepy, a Python client for Twitter, and Scrapy, a framework for crawling websites.

Embed API v3 and sphinx-hoverxref 1.0

After some months of work, we are excited to publish v3 of our Embed API, and with it, version 1.0 of sphinx-hoverxref.

Among other things, the new versions allow you to embed content from pages hosted outside Read the Docs. For security reasons, we have an allowlist of such external domains that currently includes Python, NumPy, SciPy, SymPy, and we would like to invite the community to suggest more domains to add.

Other additions and fixes include:

Support for JS and CSS assets in Sphinx 4.1+.
Support for glossary / term and sphinxcontrib-bibtex.
Avoid showing the browser built-in tooltip for links that have the extension enabled.

Finally, version 3 of the Embed API paves the way for supporting non-Sphinx projects in the future.

Embedding content with sphinx-hoverxref

To use sphinx-hoverxref in your Read the Docs project, declare it as part of your dependencies:

requirements.txt

sphinx==4.3.1
sphinx_rtd_theme==1.0.0
sphinx-hoverxref==1.0.0

And add it to your list of Sphinx extensions:

conf.py

extensions = [
    # ...other extensions
    "hoverxref.extension",
]

To enable the extension on all :ref: of your documentation, set the hoverxref_auto_ref to True:

conf.py

hoverxref_auto_ref = True

And you will start seeing tooltips on your internal references. Apart from :ref: roles, you can also enable tooltips on:

Note

sphinx-hoverxref includes the JavaScript embed client in the HTML assets, but it is not yet available as a standalone library that can be reused with standard frontend packaging tools. If you would like to see this happening, let us know.

Calling the Embed API directly

As explained in our embedding guide, you can call the API directly from any HTTP client:

$ curl -s "https://readthedocs.org/api/v3/embed/\
> ?url=https://docs.readthedocs.io/en/latest/features.html\
> %23read-the-docs-features" | python -m json.tool
{
  "url": "https://docs.readthedocs.io/en/latest/features.html#read-the-docs-features",
  "fragment": "read-the-docs-features",
  "content": "<div class=\"section\" id=\"read-the-docs-features\">\n<h1>Read the Docs ...",
  "external": false
}

Or visually explore the query in the web interface instead.

The reference documentation includes more detail about the parameters and return values of the API.

Try it out!

We would like to invite the community to try out these features and send us feedback. With the help of our users, we will keep moving towards a more cohesive documentation ecosystem of interlinked Python projects.

Considering using Read the Docs for your next Sphinx or MkDocs project? Check out our documentation to get started!

New release of sphinx-hoverxref with support for intersphinx

2021-06-10T00:00:00Z

We have released version 0.6b1 of sphinx-hoverxref, a Sphinx extension that shows a content preview of a cross-reference.

This extension is an essential part of the work we are doing to improve interoperability of documentation in general, and scientific documentation in particular, thanks to the CZI grant we received last year.

sphinx-hoverxref displaying a tooltip including an equation

Embedding content previews

Technical documentation is full of cross-references (or xrefs): hyperlinks to related parts of the documentation (or even to an external project) that help contextualize the information. Sphinx provides powerful tools for creating xrefs, which can be used to link other parts of the documentation, and in the case of software projects, also to specific objects of the API.

However, jumping through several links while reading some documentation can be distracting, since opening them might need either a context switch or keeping a new browser tab for a later moment.

Here is where sphinx-hoverxref comes to the rescue: it scans the cross-references and attaches content previews to them (either tooltips or modal dialogues) that embed the contents of the page the xrefs point to. This way, when the user is interested in a particular topic, they can hover the mouse over the link and display the content preview instead of opening the link.

Moreover, the content previews created by sphinx-hoverxref also work if the embedded content requires a particular extension to be rendered. This makes it possible to embed content that includes mathematical equations, for example.

Support for `intersphinx`

Version 0.6b1 of sphinx-hoverxref introduces opt-in support for Intersphinx linking. sphinx.ext.intersphinx is a Sphinx extension that generates automatic links to the documentation of objects in other projects, and now it’s possible to show floating windows on them too.

A project (left) embedding content from another project linked through intersphinx (right) using sphinx-hoverxref

To enable it, the user needs to specify which projects from the intersphinx_mapping will be scanned by sphinx-hoverxref, using a new configuration value:

# Values of the intersphinx_mapping keys that you want to enable hoverxref on
hoverxref_intersphinx = [
   "astropy",
   "numpy",
   "scipy",
   "matplotlib",
]

Moreover, the look & feel of the current project is preserved even if the target documentation has a different theme or colors, which makes the result more visually consistent.

Note

At the moment, the target project needs to be hosted on Read the Docs, which is a limitation that we hope to remove in the future.

You can draw inspiration from some community projects that are using this feature already:

The Grill, an extension for digital content creation with Maya, Houdini, and other 3D modeling applications, and references another project called naming.
poliastro, a Python library for interactive Astrodynamics that makes cross-references to Astropy and other projects.

We are excited to see what our users can do with sphinx-hoverxref, and will continue working on it in the future.

Considering using Read the Docs for your next Sphinx or MkDocs project? Check out our documentation to get started!

Announcing Chan Zuckerberg Initiative Grant to Expand the Interoperability of Scientific Documentation

2020-11-19T00:00:00Z

We’re excited to announce that Read the Docs has received a $200,000 grant from the Chan Zuckerberg Initiative’s Essential Open Source for Science (EOSS) program. Read the Docs is the largest open source documentation hosting platform in the world. We provide hosting for many scientific software packages, including some that have received EOSS funding in the past. You can read more about this round of grants in the official announcement.

Our grant has two parts:

Part 1 allows us to develop new software to improve the interoperability of scientific documentation.
Part 2 allows us do advocacy work around the importance and value of documentation in the scientific community.

We’re excited about the chance to work with the scientific community to improve the overall experience of writing and reading documentation. We plan to do a lot of outreach to various community members, so that we can ensure we’re building tools that provide the most value. We also plan to write content that will hopefully make the process of documenting scientific code easier.

Part 1: improve the interoperability of scientific documentation

This work is something that we’ve been doing as a side project for some time now. We have an extension to Sphinx that we call sphinx-hoverxref. It provides hovering tooltips for Sphinx projects when hover over a link. You might be familiar with this functionality on GitHub or Wikipedia – this project adds that functionality to any project hosted on Read the Docs. You can see a live demo on a fork of the Python documentation.

The goal of the grant will be to expand this functionality by doing a few things:

Improving and documenting the backend Embed API on Read the Docs, allowing documentation content to be embedded anywhere on the web.
Building a JavaScript client for the Embed API, which includes tooltips showing the content on any website.
Integrating this JS client and backend API together in a new version of sphinx-hoverxref.
Extending sphinx-hoverxref to work across multiple Read the Docs sites, allowing content hovers across the entire scientific documentation ecosystem.
If time allows, also extending the backend Embed API to support Sphinx documentation hosted outside of Read the Docs, allowing even more interoperability between documentation.

We are planning to contract a frontend developer to help with this work. The Python work will likely be carried out by our existing team, funded via the grant. This is because of the complexities involved with the Read the Docs and Sphinx integration, it will be more cost effective to have people already familiar with the codebases on these tasks.

Part 2: Advocacy work around documentation in the scientific community

We know that documentation is a pain point for scientific projects. There is a lack of resources in the community, and we feel that we can be a central clearing house for better documentation tutorials and best practices. This part of the funding will be about growing appreciation and knowledge of documentation in the scientific community in general, and advocating for time to be spent on crafting good documentation with the best tooling.

Part of this work would be letting people know about the work in Part 1, promoting the tooling to make better documentation in the Scientific Python ecosystem. Beyond that, we will also promote tutorials and best practices around documentation for Scientific users, leading to more resources being available and better documentation in the community.

The high-level roadmap for this part of the grant is:

Improve documentation and promote the work in Part 1.
Adapt the Guides section of the Read the Docs documentation into a more full-featured Education section, along with standardizing and editing all existing guides.
Expand our existing Sphinx Tutorial, hopefully adding it upstream to Sphinx as the official tutorial
Write additional documentation guides focused on the scientific community, based on feedback from people in the community. This will likely include tutorials on integrating Jupyter with Sphinx, Jupyterbook, and other pain points that users have.
If time allows, we would also love to work on curating some of the documentation resources at Write the Docs, specifically topics that are relevant to scientific projects.

As part of this work, we plan to contract someone in a technical writer/developer evangelist role. Someone with knowledge of the scientific community would be a great benefit, given that we plan to seek feedback from many scientific projects on the priority of tasks we work on.

Looking forward

We are starting to look for contractors who would be a good fit for the work. People who care about documentation and are already connected with the scientific community are a big plus. We hope to post proper job descriptions for these roles in the near future, but wanted to note that we’re looking in our initial announcement.

This is the largest grant that we’ve ever received, and we are working hard to make sure it’s successful. We appreciate the trust that the Chan Zuckerberg Initiative has placed in us, and we will work hard to make sure the results of this grant will be adopted across the scientific ecosystem.

If you want to keep up to date with this work, you can follow along on GitHub or subscribe to our blog for updates.

Read the Docs Blog - Posts tagged grant

Announcing Embed API v3 and sphinx-hoverxref 1.0

The value of content embedding

Embed API v3 and sphinx-hoverxref 1.0

Embedding content with sphinx-hoverxref

Calling the Embed API directly

Try it out!

New release of sphinx-hoverxref with support for intersphinx

Embedding content previews

Support for intersphinx

Announcing Chan Zuckerberg Initiative Grant to Expand the Interoperability of Scientific Documentation

Part 1: improve the interoperability of scientific documentation

Part 2: Advocacy work around documentation in the scientific community

Looking forward

Support for `intersphinx`