Jupyter Notebooks: Boost your productivity with Extensions and Magic Commands

In this blogpost I will share some tips for working with Jupyter Notebooks. Those tips greatly improved my productivity when working with Jupyter Notebooks and I wish someone would have told me earlier. The two main topics of this post are extensions and magic commands.

Jupyter Extensions

Have you ever missed a feature in your Jupyter Notebook that IDEs have? E.g. you were hoping for autocompletion or automatically formatting code? Then there might be a Jupyter Notebook extension for you. Jupyter extensions simply add features to your Jupyter Notebook, similar to what browser plug-ins do to browsers. Just pip install jupyter-contrib-nbextensions. On your localhost:8888 you will now find a tab called Nbextensions. There, you can select the extensions you would like to use by checking the boxes. Usually, you have to restart your kernel for the extension to work. In the following, I’ll show you my five favorite extensions.

Notify

Have you ever waited for some time consuming code to execute, switched tabs and then forgot about it? Or you kept staring at your notebook to wait for the execution of the cell to finally have finished? No need for that with the Notify extension: It displays a web notification, when a cell has run. After you added this extension, you will see a new dropdown menu in your notebook’s toolbar. You can choose, if and when you want to be notified. Your choices are: Disabled, 0, 5, 10 and 30. If you select for example 5, you will get a notification, if the kernel is busy longer than 5 seconds. I am using the extension with Safari and it works great. It should also work with Firefox and Chrome browsers.

Snippets

Do you sometimes find yourself typing the same code snippets over and over again? Maybe you copy paste your database connection strings, some of your most used functions or you always load the same packages? Then the Snippets extension may be useful for you.

You have to add your snippet as a new JSON block to the list of existing snippets in snippets.json. You can find this file in your jupyter directory with jupyter --data-dir and then going to /nbextensions/snippets/snippets.json. There are already 2 example snippets. You can add your snippets there by just appending to the .json file in the same format. For example, to add a new snippet that imports your favorite data science packages, just add it to the json file.

        {
            "name" : "ds imports",
            "code" : [
                "import pandas as pd",
                "import numpy as np",
                "import matplotlib.pyplot as plt",
                "import seaborn as sns"
            ]
        }

After you added a snippet, it shows up in the tool bar in the drop down menu snippets. I use this a lot for database configurations and loading the standard packages.

Autopep8

If you neglected coding style guidelines Autopep8 could help you. It automatically reformates your code to meet the PEP 8 standards, Python’s most popular style guide. If you do not already have it, you need to %%pip install autopep8. Then you can autoformat your code by pressing Alt-A for a cell or Alt-Shift-A for the entire notebook. Or you could also select a cell and click on the little hammer icon on the right of your toolbar and your code will be prettified!

hinterland

Autocompletion in Jupyter Notebooks exists, but it is cumbersome: You need to press tab everytime you need it. So I really missed the quick autocompletion with every keypress until I found out about hinterland. With hinterland you will have the autocompletion with every key press. You just need to add the extension.

ExecuteTime

Sometimes it is interesting to know how long the execution of entire cells took. Of course you could use %%timit in every cell. Or you save some time by using the ExecuteTime extension. For each cell, you will automatically see when the execution started and how long it took.

The following table summarizes my top 5 extensions. Check out the official documentation for all nbextensions for more.

Magic Commands

Magic commands are shortcuts that significantly extend a notebook’s capabilities. The Python kernel uses the % syntax element for magic commands. You can use them line and cellwise. In line-wise, all given commands must start with the % character. For an entire cell, all commands start with%%.

Let’s look at my personal top 5 magic commands:

%history

So you did something super smart but then you deleted your cell and you forgot to hit the undo button and you can’t remember how you did it? %history shows you the history of all commands of your current notebook since the kernel started.

%matplotlib inline

During your data analysis you may create a lot of plots. If you insert %matplotlib inline in a cell at the beginning of your notebook, your plots will be directly shown below the code cell that produced it.

%%system

You briefly want to check your directory and directory contents without using your terminal separately? You can use shell commands from your Jupyter Notebook with %system for a single line of shell script or %%system at the beginning of your cell for using the shell in the entire cell. %%system let’s you use your Jupytper notebook cell like your terminal.

%%latex

This one comes in handy if you want to include mathematical formulas in your notebook. Formulas in Markdown Syntax are limited and cumbersome. Just insert a latex cell and type your formulas with latex syntax.

%%capture

Let’s say you have some time consuming code running in a cell. Closing the Jupyter window won’t prevent the cell from running. However, you would lose your printed output, which may be of interest for you. If you start your cell with %%capture <someName>, the cell output will be saved and you can retrieve it later, even if you closed your notebook.

%%capture output
# Time-consuming code here
# show all interim print results
output.show()	

Let’s summarize my favorite magic commands. If you want to find out more about magic commands check out the IPython Documentation