Welcome to the second part of our series on Automating Periodic Website Pinging. In this post, we’ll learn about setting up cron job with an active conda virtual environment.

Prerequisites

Before we dive in, make sure you have:

• Basic command line familiarity
• Access to a Unix or Unix-like operating system (such as Linux or macOS)
• Miniconda or Anaconda installed

Setting Up Conda Environment

Let's start by creating a Conda virtual environment. We'll call this environment env_ping_website and install latest python version available in conda repository.

conda create -n env_ping_website python

After executing this command, you'll see output similar to this (package versions and paths may differ):

  environment location: /home/username/miniconda3/envs/env_ping_website

  added / updated specs:
    - python


The following NEW packages will be INSTALLED:

  _libgcc_mutex      pkgs/main/linux-64::_libgcc_mutex-0.1-main 
  _openmp_mutex      pkgs/main/linux-64::_openmp_mutex-5.1-1_gnu 
  bzip2              pkgs/main/linux-64::bzip2-1.0.8-h5eee18b_6 
  ca-certificates    pkgs/main/linux-64::ca-certificates-2024.7.2-h06a4308_0 
  expat              pkgs/main/linux-64::expat-2.6.2-h6a678d5_0 
  ld_impl_linux-64   pkgs/main/linux-64::ld_impl_linux-64-2.38-h1181459_1 
  libffi             pkgs/main/linux-64::libffi-3.4.4-h6a678d5_1 
  libgcc-ng          pkgs/main/linux-64::libgcc-ng-11.2.0-h1234567_1 
  libgomp            pkgs/main/linux-64::libgomp-11.2.0-h1234567_1 
  libstdcxx-ng       pkgs/main/linux-64::libstdcxx-ng-11.2.0-h1234567_1 
  libuuid            pkgs/main/linux-64::libuuid-1.41.5-h5eee18b_0 
  ncurses            pkgs/main/linux-64::ncurses-6.4-h6a678d5_0 
  openssl            pkgs/main/linux-64::openssl-3.0.14-h5eee18b_0 
  pip                pkgs/main/linux-64::pip-24.0-py312h06a4308_0 
  python             pkgs/main/linux-64::python-3.12.4-h5148396_1 
  readline           pkgs/main/linux-64::readline-8.2-h5eee18b_0 
  setuptools         pkgs/main/linux-64::setuptools-69.5.1-py312h06a4308_0 
  sqlite             pkgs/main/linux-64::sqlite-3.45.3-h5eee18b_0 
  tk                 pkgs/main/linux-64::tk-8.6.14-h39e8969_0 
  tzdata             pkgs/main/noarch::tzdata-2024a-h04d1e81_0 
  wheel              pkgs/main/linux-64::wheel-0.43.0-py312h06a4308_0 
  xz                 pkgs/main/linux-64::xz-5.4.6-h5eee18b_1 
  zlib               pkgs/main/linux-64::zlib-1.2.13-h5eee18b_1 


Proceed ([y]/n)? 

After creating the environment, let's activate it and verify the Python path:

conda activate env_ping_website
which python

You should see a path similar to

/home/username/miniconda3/envs/env_ping_website/bin/python

creating a Bash Script

Now, Let us create a bash script called ping_website.sh inside the folder ping_website in home directory:

mkdir ~/ping_website # make dir ping_website
cd ~/ping_website
nano ping_website.sh # use nano or vi/vim

Add the following content to the script:

   #!/bin/bash
   
   source ~/miniconda3/etc/profile.d/conda.sh 
   conda activate env_ping_website  
   
   which python

Let's break down each line:

#!/bin/bash

• This is called a 'hashbang' line. It tells the system that this script should be executed using the Bash shell.

source ~/miniconda3/etc/profile.d/conda.sh

• This line sources the Conda initialization script. It's necessary to make the conda command available in the script.
• The path may vary depending on your Miniconda/Anaconda installation. Adjust it if necessary.

conda activate env_ping_website

• This activates the Conda environment we created earlier named env_ping_website.
• After this line, any Python commands will use the Python interpreter from this environment.

which python

• This command prints the full path to the Python interpreter that's currently being used.
• It's included here as a verification step to ensure we're using the correct Python from our Conda environment.

Now, let us make this script executable with:

chmod +x ping_website.sh

setup cronjob

Let us set a crobjob to run this script. Open the crontab editor:

crontab -e

and add the following line to run the script:

*/5 * * * * /home/username/ping_website/ping_website.sh  >> /home/username/ping_website/ping_website.log 2>&1

This cron job runs our ping_website.sh script every 5 minutes. The */5 * * * * part sets the schedule. The script's output (both standard output and error messages) is appended to ping_website.log in the same directory. This allows us to monitor the script's execution and troubleshoot any issues by checking the log file.

Remember to replace "username" with your actual username when setting up the cron job.

Checking Cron Logs

After the cron job runs, we should see a line similar to this in the log file:

/home/username/miniconda3/envs/env_ping_website/bin/python

This confirms that the cron job is using the correct Python interpreter from our Conda environment.

conclusion

In this post, we've:

• Created a Conda environment with Python installed
• Created a bash script to activate the Conda environment and print the Python path
• Set up a cron job to execute the bash script every five minutes

In the next blog post, we'll learn how to write a Python script to ping websites and set up a cron job to run it periodically.