Magic Commands That Every Data Wizard Should Know
Jupyter Notebook is a web-based, interactive computational environment that is used extensively by the machine learning and data science community. It is a very convenient tool to create and share documents, from codes to full-blown reports, making it one of the most popular tools among data scientists.
One feature that I love in Jupyter Notebook is called magic commands. To put it simply, they are shortcuts that can extend the capabilities of a notebook significantly. In this article, I’ll go through some of the magic commands that you should know in order to become a better data wizard.
If you want to follow along, you can refer to this Jupyter Notebook for the complete code:
A quick note.
Before we start with the first command, it is important to know that there are 2 types of magic commands: the % prefix and the %% prefix.
The % prefix indicates that the command runs over a single line of code whereas the %% prefix allows the command to runs over an entire cell. Don’t worry, you’ll see the difference between these two in the following examples.
1. %lsmagic
Now, let’s start by looking at all the magic commands that we can use in Jupyter Notebook. To do this, we can simply execute the %lsmagic command in a single cell.
Well, it seems like there’s a lot to choose from, so let’s just go over some of my favorite commands.
2. %%writefile
Sometimes, you might feel that you need to add your cell’s content into Python script or a text file from your Jupyter Notebook. Instead of copying everything and creating a new file, you can directly export your cell’s content by adding the %%writefile command before the code.
Notice that here, we’re using the double % before the command to indicate that we want the whole cell’s content to be exported.
Since I didn’t have a file named ‘hello.py’ in my current directory, it displays ‘Writing hello.py’ specifying that it will create a new Python file with the content as shown above. However, if I had created the file, it will overwrite the original content of the file with the exported cell’s content.
3. %pycat
Usually, you might want to copy a few lines of code from an external file into your Jupyter Notebook. The good news is, you don’t need to fetch the file and opening it up to copy the contents anymore. The %pycat command allows you to display the content of any external files.
Remember the ‘hello.py’ file we wrote earlier? We can use the %pycat command to display its content. This command can also be considered as the reverse of %%writefile command in terms of its application.
4. %timeit and %%timeit
Ever wondered how much time does your code takes to run? I personally use the %timeit command a lot as it is a fast way to benchmark your code and indicate to other people how much time they would need to reproduce your results.
The above code simulates 1 million dice rolls using NumPy’s randint method. The %timeit command helps in getting the execution time required to run the simulation. Let’s see another example:
As we can see from the examples, the %timeit command can be used for both a single-line code or a whole cell. However, if you want to measure the execution time for a whole cell, you need to use the %% prefix instead.
5. %who
The next magic command is %who, which will list all the variables defined in your notebook. This command is handy as it keeps you organized when you need to find something.
In the following example, I have 3 variables — a, b, and c. If we run %who, it will list down all the 3 variables that we have defined.
The above command displays all the variables irrespective of their data types. But it also allows you to search for specific data types by adding it to the end of the command. For instance,%who str will list all the str variables.
6. %store
This last magic command might be the simplest way to share data of any data type between different notebooks. This can be very useful particularly if you’re working with a bunch of people in the same environment, sharing them the data from your notebook can make the process smoother for everyone.
The %store command allows you to share any variable between different notebooks just by passing the variable that you want to share after the magic command.
Next, to retrieve the variable, you only need to open a different notebook and pass the same command with an additional ‘-r’ parameter.
And there you have it! The top 6 magic commands that I think every data wizard should know. Of course, there are way more magic commands than I covered in this article. You can refer to the IPython Documentation for a complete list of the commands.
What magic commands do you use and why? Let me know in the comments or on Twitter!
Comments