In order to properly use the Jupyter notebook/lab on the HPC you will have to make some modifications in your workflow.
To have a singular resource for each user we had to make a global install of conda so every user would have conda installed regardless of their own personal install.
This global install of conda has been configured to allow users to change environments on the fly.
To access this installation you need to run the following command:
/opt/conda/bin/conda init bash
Note: If for some reason you want to switch back to your personal conda setup and loose the use of environment switching using Jupyter Notebooks/Lab on the HPC simply run:
/home/<username>/*conda/bin/conda init bash
You will need to re-run /opt/conda/bin/conda init bash to enable environment switching again.
This modifies how you will be able to call your own existing envs from command line.
Namely all names are removed (i.e. if you had an env made with '-n my_env' typing `conda activate my_env` would no longer activate the environment) as the conda binary is looking in /opt/conda and /home/$user/.conda/envs for environments.
To activate the environments you have created you’ll need to use the full path (i.e. `conda activate /home/$user/*conda/envs/my_env`).
This will also change where you create environments. When you use `conda create -n my_env` it will place it in a new folder located at /home/<username>/.conda/envs/ and you can activate it using the typical `conda activate my_env`.
If you want to install in the old location instead of using `conda create -n my_env` you will need to use the prefix option to point to your env folder: `conda create -p /home/<username>/*conda/envs/my_env`. All environments created with the prefix option will have to be activate with the full path instead of the name.
In order to leverage the environment switching inside Jupyter Notebook you will need to install the appropriate kernels for the languages you have inside the environment in order to be able to switch kernels.
You can do this with the following commands within the environment you want to use:
To install the python kernel run: conda install -c anaconda ipykernel
To install the R kernel run: conda install -c r r-irkernel
You can change the active environment once inside the notebook (or when creating one) via kernels: Kernel > Change kernel > whatever_you_name_your_env