Stats
We generate timing statistics using pytest-benchmark using \(R\) rounds for circuits A, B, C 1.
In this section, we benchmark between PyQTorch and Horqrux:
- the
runmethod, - the
expectationmethod using a single observableZ,
The current execution times (with \(R=10\)) are for circuits defined over \(2, 5, 10, 15\) qubits and \(2, 5\) layers for the run and expectation methods.
import json
import pandas as pd
import re
import matplotlib.pyplot as plt
import os.path
fname = "stats.json"
if not os.path.isfile(fname):
fname = "docs/stats.json"
with open(fname, 'r') as f:
data= json.load(f)['benchmarks']
data_stats = [{'name': x['name']} | x['params'] | x['stats'] for x in data]
frame = pd.DataFrame(data_stats)
frame['name'] = frame['name'].apply(lambda x: re.findall('test_(.*)\\[', x)[0])
frame['fn_circuit'] = frame['benchmark_circuit'].apply(str)
frame['fn_circuit'] = frame['fn_circuit'].apply(lambda x: re.findall('function (.*) at', x)[0])
Run method
Here are the median execution times for the run method over a random state.
run_frame = frame[frame['name'].str.startswith('run')]
run_frame['name'] = run_frame['name'].str.replace('run_', '')
axes = run_frame.boxplot('median', by=['fn_circuit', 'name'])
axes.set_title("Timing distributions by test and circuit \n for `run` method")
axes.set_xlabel('')
axes.set_ylabel('Time (s)')
axes.set_yscale('log')
plt.xticks(rotation=75)
plt.suptitle('')
plt.tight_layout()
Expectation method: Z(0) observable
Here are the median execution times for the expectation method over a random state and the \(Z(0)\) observable.
expectation_frame = frame[frame['name'].str.startswith('expectation')]
expectation_frame['name'] = expectation_frame['name'].str.replace('expectation_', '')
axes = expectation_frame.boxplot('median', by=['fn_circuit', 'name'])
axes.set_title("Timing distributions by test and circuit \n for `expectation` method")
axes.set_xlabel('')
axes.set_ylabel('Time (s)')
axes.set_yscale('log')
plt.xticks(rotation=75)
plt.suptitle('')
plt.tight_layout()