Automatically building, previewing, and pushing your book with CircleCI

    In order to accomplish each of these, we'll use a CircleCI configuration file. Thisis a YAML file that is used to tell Circle what to do with your repository.

    In each case, the expectation is that your master branch holds your book content.

    We'll step through each piece of a sample CircleCI configuration to show you howto accomplish this.

    First of all, you should set up your CircleCI account to start running CI jobs foryour book repository. Follow these steps:

    Find the "build forked pull requests" section, and switch it to ON.

    Now, CircleCI will start watching your repository. If it finds a Circle configurationfile (more information on this below), it'll run a CI job according to the configurationit finds.

    1. # Tell CircleCI which version of its API you wish to use
    2. version: 2.1
    3. jobs:
    4.     # Jobs define the different parts of your CircleCI workflow
    5.     # They can depend on one another, use pieces from one another, etc.
    6.     # We'll fill them in later
    7. workflows:
    8.     # Workflows tell CircleCI the order in which to run your jobs
    9. commands:
    10.     # Commands are re-usable chunks of steps that can be shared across jobs

    Let's start filling out this template with a few jobs. In each case, we'lluse a Python Docker image to both build each page's HTML, and build the book'sHTML.

    Step 1: Build each page's HTML

    First you'll build each page's HTML. This is the initial conversion from, md, etc files. We'll use a Python container for this in order touse the Jupyter Book command-line interface with jupyter-book build.

    You can build your book's HTML files and preview them using CircleCI artifacts.To do this, you'll need to use two CircleCI jobs:

    We'll need to persist the results of this step so that they are available insubsequent steps. Here's the CircleCI configuration that will accomplish this, which you canadd to the skeleton configuration you've created above:

    Note that, at the end of this job, we've persisted the contents of the folder.This allows us to re-use these contents in subsequent jobs.

    Now that our page HTML files have been built, we can use Jekyll to buildthe HTML for our entire book. This is useful for two purposes:

    • previewing your Jupyter Book using CircleCI artifacts.
    • publishing the HTML of your book to someplace online In both cases we need to install Jekyll and build the HTML for the book,so let's first define a CircleCI command to do this.
    1. commands:
    2.   prepare_jekyll_installation:
    3.     steps:
    4.       - checkout
    5.       - attach_workspace:
    6.           # Must be absolute path or relative path from working_directory
    7.           at: /tmp/workspace
    8.  
    9.       # Grab the the built intermediate files from the last step
    10.       - run:
    11.           name: Copy over built site files
    12.           command: |
    13.             rm -rf _build
    14.             cp -r /tmp/workspace/_build .
    15.  
    16.       # Install miniconda to test install
    17.       - run:
    18.           name: install miniconda
    19.           command: |
    20.             export MINICONDA=$HOME/miniconda
    21.             echo "export PATH=$MINICONDA/bin:$PATH" >> $BASH_ENV
    22.             source $BASH_ENV
    23.             hash -r
    24.             wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh -O miniconda.sh
    25.             bash miniconda.sh -b -f -p $MINICONDA
    26.             conda config --set always_yes yes
    27.             conda update conda
    28.             conda info -a
    29.             conda create -n testenv python=3.7.0
    30.             source activate testenv
    31.             rm miniconda.sh
    32.  
    33.       # Install Ruby/Jekyll dependencies
    34.       - run:
    35.           name: Installing Ruby/Jekyll from conda-forge
    36.           command: conda install -c conda-forge rb-github-pages
    37.  
    38.       # Build the book's HTML w/ the base_url for CircleCI artifacts
    39.       - run:
    40.           name: Install book Ruby dependencies
    41.           command: bundle install

    Step 3: Build and preview your book's HTML with Circle artifacts

    We'll re-use the command from above in order to preview our built siteusing Circle artifacts. Add the job below to your CircleCI configuration file.

    After the Jekyll installation command, it then runs the build command from Jekyll,which outputs all the HTML for your site. We add the —baseurl /0/html because thisis the prefix for the Jekyll artifact URL. Finally, we use the command to tell Jekyll to keep these artifacts for later. Once this job completes,you'll be able to click the "Artifacts" tab to preview your book HTML.

    You can also choose to automatically push built HTML from your master branchto a live textbook. This lets you automatically deploy changes to your bookso that they go live online. Again, we'll use the page HTML command defined above.The job will be very similar to the HTML artifacts preview job, with an extrastep to actually push the book's HTML online.

    This step assumes that you are hosting your live book on a Git repositoryusing GitHub Pages. We'll need the include a security key that allows CircleCI push accessto your GitHub repository. You should first create an SSH key with write access to therepository that will be hosting your live site. Then, use the below configurationto tell Circle to automatically push to this repository.

    1. jobs:
    2.   deploy:
    3.     docker:
    4.       - image: circleci/python:3.7-stretch
    5.     steps:
    6.       # Add deployment key fingerprint for CircleCI to use for a push
    7.       - add_ssh_keys:
    8.             - "{{ YOUR SSH KEY FINGERPRINT }}"
    9.  
    10.       - prepare_jekyll_installation
    11.       - run:
    12.           name: Build the website for deploy
    13.           command: bundle exec jekyll build
    14.  
    15.       # Deploy the built site with ghp-import
    16.       - run:
    17.           name: Deploying site using ghp-import
    18.           command: |
    19.             pip install ghp-import
    20.             ghp-import -p -f -n ./_site/

    In this case we've used the excellent .The command pushes the contents of ./_site (your book's HTML) to the gh-pagesbranch of the repository. The flag adds a .nojekyll file to the built HTML,which ensures that Jekyll will treat it as raw HTML.

    Step 4: Tying these workflows together

    Now that we've defined several jobs above, we need to tell CircleCI how touse them sequentially (or in parallel). In particular, we want the jobthat builds each page's HTML to run first sothat the each page's HTML can be stitched together into a book.Here's the configuration for this:

    1. # NOTE: This is an example CircleCI configuration that
    2. # will build your book and preview its HTML content.
    3. # You will probably have to modify it in order to get it working
    4. # just the way you want. See https://jupyterbook.org/advanced/circleci.html
    5. # for more information
    6. version: 2.1
    7. jobs:
    8.   build_page_html:
    9.     docker:
    10.       - image: circleci/python:3.7-stretch
    11.     steps:
    12.       - checkout
    13.       - run: pip install --user -r requirements.txt
    14.       - run:
    15.           name: Build site intermediate files
    16.           command: jupyter-book build .
    17.  
    18.       # Persist the built files for the deploy step
    19.       - persist_to_workspace:
    20.           root: .
    21.           paths:
    22.             - _build/
    23.  
    24.   doc:
    25.     docker:
    26.       - image: circleci/python:3.7-stretch
    27.     steps:
    28.       - prepare_jekyll_installation
    29.       - run:
    30.           name: Build the website
    31.           command: bundle exec jekyll build --baseurl /0/html/
    32.  
    33.       # Tell Circle to store the documentation output in a folder that we can access later
    34.       - store_artifacts:
    35.           path: _site/
    36.           destination: html
    37.  
    38.   # Deploy the built site to jupyter-book.github.io
    39.   deploy:
    40.     docker:
    41.       - image: circleci/python:3.7-stretch
    42.     steps:
    43.       # Add deployment key fingerprint for CircleCI to use for a push
    44.       - add_ssh_keys:
    45.           fingerprints:
    46.             - "{{ YOUR SSH FINGERPRINT }}"
    47.  
    48.       - prepare_jekyll_installation
    49.       - run:
    50.           name: Build the website for deploy
    51.           command: bundle exec jekyll build
    52.  
    53.       # Deploy the built site with ghp-import
    54.       - run:
    55.           name: Deploying site using ghp-import
    56.           command: |
    57.             ghp-import -p -f -n ./_site/
    58.  
    59.  
    60. # Tell CircleCI to use this workflow when it builds the site
    61. workflows:
    62.   version: 2
    63.   default:
    64.     jobs:
    65.       - build_page_html:
    66.           filters:
    67.             branches:
    68.               ignore:
    69.                 - gh-pages
    70.       - doc:
    71.           requires:
    72.             - build_page_html
    73.           filters:
    74.             branches:
    75.               ignore:
    76.                 - gh-pages
    77.       - deploy:
    78.           requires:
    79.             - build_page_html
    80.           filters:
    81.               branches:
    82.                 only:
    83.                   - master
    84.                 ignore:
    85.                   - gh-pages
    86.  
    87. commands:
    88.   prepare_jekyll_installation:
    89.     steps:
    90.       - checkout
    91.       - attach_workspace:
    92.           # Must be absolute path or relative path from working_directory
    93.           at: /tmp/workspace
    94.  
    95.       # Grab the the built intermediate files from the last step
    96.       - run:
    97.           name: Copy over built site files
    98.           command: |
    99.             rm -rf ./_build
    100.             cp -r /tmp/workspace/_build .
    101.  
    102.       # Install miniconda to test install
    103.       - run:
    104.           name: install miniconda
    105.           command: |
    106.             export MINICONDA=$HOME/miniconda
    107.             echo "export PATH=$MINICONDA/bin:$PATH" >> $BASH_ENV
    108.             source $BASH_ENV
    109.             hash -r
    110.             wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh -O miniconda.sh
    111.             bash miniconda.sh -b -f -p $MINICONDA
    112.             conda config --set always_yes yes
    113.             conda update conda
    114.             conda info -a
    115.             conda create -n testenv python=3.7.0
    116.             source activate testenv
    117.             rm miniconda.sh
    118.  
    119.       # Install Ruby/Jekyll dependencies
    120.       - run:
    121.           name: Installing Ruby/Jekyll from conda-forge
    122.           command: conda install -c conda-forge rb-github-pages
    123.  
    124.       # Build the book's HTML w/ the base_url for CircleCI artifacts
    125.       - run:
    126.           command: bundle install

    This page was created by