This HOWTO shows you how to run a Jupyter Notebook on one of the School’s compute servers, and then interact with it from a web browser running on your own computer.

This presents one of the simplest ways of accessing Jupyter Notebooks remotely like this. It relies on the magic of SSH tunnels, and you can reuse the basic concept presented here to access Notebooks running on other computing facilities.

Step 0: Prerequisites

This HOWTO assumes you’re already familiar accessing School compute servers using SSH, either from your own computer or from another School Linux host.

In this guide, we’ll assume you want to run a Jupyter notebook on studentrun01 – our Student Compute Server. You can however run Jupyter on any other compute server simply by substituting studentrun01 below with the name of the compute server you want to use.

Step 1: SSH into a compute server as usual

Here’s an example of how you can do this from your own computer. As mentioned above, this HOWTO assumes you’re already familiar with this, so we’ve only included the key details here.

Firstly, open a Terminal window (if you’re using macOS or Linux) or a command prompt (on Windows).

Then type the following command:

ssh -J [UUN]@student.ph.ed.ac.uk [UUN]@studentrun01

NOTE: Substitute [UUN] in the above with your University UserName. Don’t type in the square brackets  – they’re being used here to indicate something that needs replaced with your own details.

Enter your password when prompted (typically twice).

You should hopefully then be logged into studentrun01 and ready to interact with it.

Step 2: Start a Jupyter Notebook server

First of all, think about whether you’re using Anaconda – for example if you’ve created your own conda environment containing specific Python packages.

If you are using Anaconda. then:

• Activate your conda environment now, as you’ll need to use the Jupyter Notebooks package that has been installed into your conda environment rather than the “default” Jupyter we provide on our Ubuntu hosts.
• Note that you’ll need to install the necessary Jupyter Notebook packages into your conda environment if you haven’t already installed them. Do this by typing: conda install notebook
• Type which jupyter to check which Jupyter you’re getting in the command below. Hopefully you should recognise the location of your conda environment in the output of this. If it reports /usr/bin/jupyter then you’re getting the Jupyter provided by our default set of Python packages, which may not have everything you need.

You can then launch a Jupyter notebook server with the following command:

jupyter notebook --no-browser

(Note that we’ve added the –no-browser option to stop Jupyter from trying to open a web browser, as that would try to run a browser on the compute host itself, which isn’t going to be useful here!)

Now copy the URL (web address) advertised by your Jupyter notebook once it starts running – you’ll use it in Step 4 below.

Step 3: Create an SSH tunnel for accessing Jupyter from your own computer

Open a 2nd terminal or command prompt window. (Make sure you keep your existing window from Steps 1 & 2 running!)

Then type the following command:

ssh -L 8888:localhost:8888 -J [UUN]@student.ph.ed.ac.uk [UUN]@studentrun01

NOTE: Change both of the 8888s to match the number (=port number) in the web address as advertised by Jupyter in Step 2, and replace the [UUN]s by your own University UserName in the same way as Step 1.

Again, enter your password when prompted.

If this is successful, you should now see a prompt on studentrun01 in the same way as in Step 1. You don’t need to type anything else here, so you can safely minimise or hide this window – just make sure you don’t close it down.

The above command has created something called an SSH tunnel. We’ll use this below to connect a web browser running on your own computer to your Jupyter server running on studentrun01.

Step 4: Access your Jupyter server with your web browser

Open a web browser on your own computer, and paste in the URL you got when you launched your Jupyter server in Step 2.

You should now see the familiar Jupyter interface on your browser. All of the interactions you make with Jupyter here will pass through the SSH tunnel you created in Step 3 and get run on the Jupyter server on studentrun01. This always feels a bit magical the first time you try it!

Make sure you leave the SSH tunnel from Step 3 running – it’s a necessary lifeline for making all of this work.

Troubleshooting

Here are some of the most common problems that can occur here, and how you can hopefully avoid or fix them.

Your browser says something like “Unable to connect to the server at localhost:8888”

This probably indicates that the SSH tunnel set up in Step 3 either hasn’t been started, or has stopped running because of a network blip. Check Step 3 and restart your SSH tunnel if necessary.

You get lots of messages like “Channel 3: open failed: connect failed: Connection refused” in your SSH tunnel window

This probably indicates that your Jupyter notebook server isn’t running on studentrun01, or is running on a different port to the one you used in Step 3.

Check that the Jupyter server is running in your window from Step 1, and that the port number (typically 8888) reported when it started matches with the numbers you’ve used for your SSH tunnel in Step 3.

Error saying “bind [127.0.0.1]:8888: Address already in use” after Step 3

This probably means you’ve already got an SSH tunnel running that’s using port 8888 on your own computer – and you’ve possibly forgotten about it.

Check other open terminal windows on your computer and close the other SSH tunnel.

Note that it is possible to run multiple SSH tunnels at once, but you need to carefully pair up the port numbers. In particular, the -L [N1]:localhost:[N2] tunnel parameters in Step 3 need to be picked as follows:

• The right number [N2] must match the port selected by the Jupyter server running on studentrun01.
• The left number [N1] determines the port you’ll use on your local computer. You’ll connect to Jupyter from your own computer via the web address http://localhost:[N1]

Possible simplifications

You can often simplify this workflow down a wee bit by simply running Jupyter from within your SSH tunnel session, i.e. doing Step 3 first, followed by Step 2 and Step 4.

However this only works if nobody else is already running Jupyter on studentrun01, otherwise your Jupyter will run on a different port to 8888, which won’t match with the ports you’ve already picked for your SSH tunnel. This results in a bit of a chicken & egg situation, so having 2 separate sessions is generally safer though a bit more work.

Alternatively, you can pre-select an alternative port of your own to use instead of 8888 – you can tell Jupyter to use this port via the –port option. There’s a good chance this will work, though it’s also not foolproof.

Note that it’s also possible to simplify the ssh commands a wee bit by creating a personal SSH configuration. We won’t cover this here, but please do ask if you’d like to learn more about this.

HOWTO: Running a Jupyter Notebook on a School compute server / Linux & Scientific Computing - Physics & Astronomy by blogadmin is licensed under a Creative Commons Attribution CC BY 3.0