Miscellaneous Tips

This is a page for miscellaneous topics that do not belong in any of the other sections

Setting up SSH for Passwordless Authentication

When using VSCode to remote log in to OSC, it kept prompting for my password and never saved it between sessions. To get around this, we need to set up an SSH public/private key pair.

  1. Navigate to ~/.ssh on your personal computer and type in

    ssh-keygen -t rsa -b 4096

    to generate a public/private key pair. The public key is stored in the file with file extension .pub which is what needs to get transferred to OSC and the private key (does not have a file extension) stays on your personal computer. You can decline to set a passphrase for this key and use the default name it suggests.

  2. The next steps depend on if you have MAC/LINUX or WINDOWS on your personal computer.

  • If MAC/LINUX: type in

    ssh-copy-id username@domain

    For example, my username/domain on OSC would be ronakndesai@pitzer.osc.edu.

  • If WINDOWS: the copy-id command may not be available, so we will need to manually transfer the public key. Open up the public key file (something like id_rsa.pub) and there should be a lot of text on just one line starting with ssh-rsa. Copy this entire line and paste it into the file called authorized_keys in the ~/.ssh folder on the remote server. If this has never been done before, you may need to manually create a file called authorized_keys here.
  • An easy way to transfer the key without manually copy/pasting would be to first copy the public key file id_rsa.pub to the ~/.ssh directory on the remote server. Then, (in the ~/.ssh directory) type cat id_rsa.pub >> authorized_keys. After doing this, make sure there is no blank line in the authorized_keys file between two adjacent keys. IMPORTANT: make sure there are TWO > characters so that we are appending the key to the end of the file instead of overwriting all the contents.

Conda Environments

To create an environment type

conda create --name env-name

To remove an environment type

conda remove -n env-name --all

Installing Packages on an offline computer

First, on a computer with internet access, create a directory to store the packages, cd into it, and type in the download pip command

mkdir offline-packages
cd offline-packages
pip download <packagename>

Then, you need to transfer the files to the offline computer, enter that directory, and type in the following command

pip install --no-index --find-links . <package-name>

Using sphinx to create python documentation.

Sphinx is a tool to automatically generate documentation from python doc strings. First, you should install sphinx using conda or pip (e.g. conda install sphinx). Some other useful packages to download would be

  • sphinx_rtd_theme to support Read the Docs style documentation

  • You should create a directory called docs and cd to it. Then, run sphinx-quickstart and answer the prompts to set everything up. I chose to keep the build and source in the same directory.
  • Make the following changes to conf.py
    • import os, import sys, and sys.path.insert(0, os.path.abspath('../')) should be three lines added to the top of the file where the specified path is the root directory of the package. This may not be necessary depending on how the project is set up.
    • Place sphinx.ext.autodoc in the extensions list (a built in sphinx tool to automatically generate documentation).
    • Change the theme by specifying html_theme = 'sphinx_rtd_theme'. There are many other themes one can choose, but the read the docs theme seems to be popular with many people.
  • Then, in the docs directory, create .rst files for each of the python modules that you want to document. Here’s an example using data_manager.rst
data_manager module
==================

.. automodule:: data_manager
    :members:
  • Make sure the string of equal signs is equal or longer than the text above.
  • For each .rst file you create, you need to specify its name (without the .rst extension) under the :caption: Contents: section of index.rst.
...
:caption: Contents:

data_manager
module_2
module_3
  • Create documentation in the specified classes.
    • In VSCode, I used the autoDocstring extension (go into the extension settings and change the format to sphinx) to help generate documentation. The way this extension works is that you type in three double quotes immediately after a class or function declaration and clicking enter will generate documentation in the sphinx format.
    • Generally, the documentation consists of a summary, param/type pairs, and return types.
    • Here is an example of what this documentation looks like for one of my defined functions remove_outliers:
 def remove_outliers(df, column, q1_mult = 1.0, q3_mult = 1.0):
     """Static Method to remove outliers from a dataframe based on a specified column

     :param df: dataframe to remove outliers from
     :type df: `pd.DataFrame`
     :param column: column to remove outliers from
     :type column: string
     :param q1_mult: minimum value included below q1 in units of IQR, defaults to 1.0
     :type q1_mult: float, optional
     :param q3_mult: maximum value included above q3 in units of IQR, defaults to 1.0
     :type q3_mult: float, optional
     :return: dataframe with outliers removed
     :rtype: `pd.DataFrame`
     """
  • Finally, type in make html (if you want to generate html documentation) in the docs directory to build the documentation.
    • Sphinx supports building documentation in pdfs, epubs, etc. so you can experiment with different options if desired.
    • You can double click index.html to open the documentation in a web browser and this file is located in docs/_build/html.

Referencing functions

Within a docstring, use :func:`function_name` to reference another function in the same module.

  • if the function is in a different module use :func:`module_name.function_name` or :func:`module_name.ClassName.function_name`
Written on September 29, 2023