Measuring Performance

The first step in accelerating any program is to measure how fast it runs. There are many ways of measuring the speed of programs. At their simplest, you can use a stopwatch (or an automated stopwatch, e.g. the time command that is available on MacOS or Linux).

At their most in-depth, you can use full profiling tools to measure how long every function call takes, and see which functions call which functions.

In this workshop, we will use something that sits in the middle. Something that is simple enough for day-to-day use, while complex enough that you can get useful information to track your progress when accelerating your code.

timeit

The timeit function is built into Python via the timeit module. This is integrated into Jupyter so that you can interactively time any function. For example, start a Jupyter Notebook and define this simple function;

def slow_function():
   import time
   time.sleep(1)

This function will sleep for one second. So, it should take about one second to run.

You can time this function by calling it within a timeit function, e.g.

timeit( slow_function() )

I get the output;

1 s ± 1.66 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

(you may see something different)

The timeit function has run the function several times (in my case, 7 times). It has measured how long each run took, and then calculates an average and a standard deviation. In this case, the 7 runs took an average of 1 second each, with a 1.66 ms standard deviation.

You can use timeit to time any function call in Python. It is a very convenient and quick way to measure how long something takes.

Note that the function that is called should be safe to call several times in a row, e.g. it doesn’t have any side effects. In general, it is not good practice to write functions that have side effects.

Note also that the timeit call must be the only call in the Jupyter notebook cell.

Exercise

We will be using the exercises throughout this workshop to examine a single piece of code.

To start, you need to download the code to your computer. To do this, copy and paste the following into a Jupyter notebook cell;

import urllib.request
url = "https://raw.githubusercontent.com/chryswoods/siremol.org/main/chryswoods.com/accelerating_python/code"
filename = "slow.py"
urllib.request.urlretrieve(f"{url}/{filename}", filename)

This should download the exercise code from the course website, and will write it to the file in your current directory, called slow.py.

Exercise 1

Try to run the code. How long does it take to execute?

To do this on Linux or MacOS you can use the time command, e.g.

time python slow.py

On Windows powershell you can (probably!) time programs using

Measure-Command { python slow.py | Out-Host }

although if that doesn’t work, you can use your watch or phone’s stopwatch.

Exercise 2

There are three functions in this program, which are called in sequence by the script:

  1. load_and_parse_data - this loads a percentage of the data for analysis, placing the data into three variables.

  2. calculate_scores - this calculates all of the scores from all of the loaded data.

  3. get_index_of_best_score - this finds the index of the pattern with the best score, from the calculated scores.

These functions are called at the bottom of the script, i.e.

if __name__ == "__main__":
    # Load the data to be processed
    (ids, varieties, data) = load_and_parse_data(5)

    # Calculate all of the scores
    scores = calculate_scores(data)

    # Find the best pattern
    best_pattern = get_index_of_best_score(scores)

    # Print the result
    print(f"The best score {scores[best_pattern]} comes from pattern {ids[best_pattern]}")

The code has been written so that it can be loaded as a module, so that each function can be called individually. This means that you can use timeit to time each individual function, e.g. in a Jupyter notebook you can type

import slow

timeit( slow.load_and_parse_data(5) )

to find out how long it takes to load 5% of the data.

Next, load all of the data using

(ids, varieties, data) = slow.load_and_parse_data(5)

Now use timeit to find out how long the calculate_scores function takes.

Next, get all of the scores using

scores = slow.calculate_scores(data)

Now use timeit to find out how long the get_index_of_best_score function takes.

  • Does the runtime of each of these three functions sum up to be about the same runtime that you measure in Exercise 1?

  • Which function is the slowest?

  • How much quicker would the script run if you could double the speed of the load_and_parse_data function?

  • How much quicker would the script run if you could double the speed of the calculate_scores function?

  • How much quicker would the script run if you could double the speed of the get_index_of_best_score function?

  • Which function should you concentrate on if you want to accelerate the script?

Answers to the above exercises