Skip to content

Setting Up and Using Pyhelix-Tools

Tools for Workhelix, by Workhelix.

Tool Useful if...
cloud_utils You want to query tables from Athena (severless) and analyze the data
plotting_utils You want to plot data interactively using graphs we've already proven out.
Timer You want to measure the execution time of your functions.
SlackAlert You want to send alerts to Slack channels when specific events occur.
configure_logger You want to set up a logger to be used throughout your project.
ProjectInit You want to manage Weights and Biases projects (create, delete, etc.).

Setup and Installation

Using pyenv and poetry for managing dependencies and virtual environments provides several advantages:

  1. Isolation: By creating a virtual environment, you can isolate the dependencies of your project from the global Python environment, avoiding conflicts between packages and different versions.
  2. Reproducibility: poetry manages dependencies using the pyproject.toml file, which makes it easy to share and reproduce the exact same environment across different systems and among team members.
  3. Versioning: It allows you to easily update or downgrade package versions without affecting the global Python environment.

By hosting the tools in a remote repository like packagecloud.io, you can:

  1. Centralize package management: Maintain a centralized location for all packages and versions, making it easier to control and distribute them.
  2. Control access: Restrict access to your packages, ensuring only authorized users can access and install them.
  3. Streamline updates: Easily distribute updates and bug fixes to all users by updating the package in the remote repository. Users only need to update the package in their projects to get the latest version.

To install and use the tools:

  1. Make sure you have pyenv and poetry installed.
  2. In the terminal, navigate to the project directory and run make activate to open a pyenv.
  3. Type poetry install to install the dependencies based on the pyproject.toml file. Alternatively, you can add the package to an existing project by typing poetry add pyhelix-tools.

To update the package to a more recent version, type poetry update pyhelix-tools.

Usage

To use a tool, at the head of a script simply type from pyhelix_tools import tool_name as alias. Replace tool_name with the name of the tool you want to use and alias with a short name you want to use for the tool in your script.

For example:

from pyhelix_tools import cloud_utils as aws
from pyhelix_tools import plotting_utils as whplt
from pyhelix_tools.decorators import Timer
from pyhelix_tools.decorators import SlackAlert
from pyhelix_tools.logging import configure_logger
from pyhelix_tools.wandb import ProjectInit

cloud_utils

For example, uses, see pyhelix-tools/notebooks/cloud_utils_example.ipynb

Note: Be sure to have your .awscredentials setup before using the package.

plotting_utils

WIP

Timer

The Timer decorator allows you to measure the execution time of your functions. You can customize the time measurement units and optionally use a logger to log the output. For example uses, see the Timer demonstration in projects/wikihelix/examples/tools.ipynb.

SlackAlert

The SlackAlert decorator sends alerts to Slack channels when specific events occur. You can customize the alert type and the Slack channel that receives the alert. For example uses, see the SlackAlert demonstration in projects/wikihelix/examples/tools.ipynb.

configure_logger

The configure_logger function is used to set up a logger that can be used throughout your project. For example uses, see the configure_logger demonstration in projects/wikihelix/examples/configure_logger_example.ipynb.

ProjectInit

The ProjectInit class helps you manage Weights and Biases projects. You can create, delete, and select projects using this class. For example uses, see the ProjectInit demonstration in projects/wikihelix/examples/project_init_example.ipynb.

Common Errors and Troubleshooting

While using pyhelix-tools, you might encounter some common errors. Here are a few examples and their solutions:

Import "pyhelix_tools.decorators" could not be resolved

This error might be due to either of the following reasons:

  1. Outdated package: Make sure you have the latest version of pyhelix-tools installed. To update the package to a more recent version, run poetry update pyhelix-tools in your project's virtual environment.
  2. Incorrect working directory: Ensure you are working in the correct project directory, which contains the pyproject.toml file. If you are in the root directory instead of the project directory, you might encounter this error.

Other import errors

If you encounter import errors related to other packages, make sure to check your project's pyproject.toml file and verify that all required dependencies are listed and installed correctly. You can update your dependencies using poetry update.

ModuleNotFoundError

If you encounter a ModuleNotFoundError such as ModuleNotFoundError: No module named 'wandb', the solution is to run poetry add wandb in the terminal of your project.

Resetting the Virtual Environment

If you want to reset the virtual environment, you can do so by deleting the .venv and poetry.lock files and then running make activate again. This will create a new virtual environment and reinstall the dependencies based on your pyproject.toml file.