Optimizing a Variational Quantum Classifier through the Behavior Analysis of its Components

Introduction

Quantum machine learning (QML) is an area of study where quantum computing and machine learning (ML) are intermingled to define and solve various real-life problems [1]. In quantum computing, computation is performed by utilizing the phenomenon of quantum mechanics [2].

Machine learning deals with the feature space of the empirical data to find generalized patterns and insights with the help of algorithms and statistical models without explicit instructions. In supervised machine learning settings, binary classifiers classify unknown data into two classes, y ∈ [−1, 1] based on what they learn from training instances {(x1, y1), (x2, y2),........,(xn, yn)}, where y ∈ [−1, 1] [3]. At first, the dataset of the problem is encoded in machines and is sent to the functions. The similarity is measured between the actual labels and the classified labels. Finally, the models are optimized by changing the hyper-parameters of the function that approximates the classes or labels. The linearly separable data is very straightforward and easy to classify. However, the real-life dataset, such as IRIS data sets, breast cancer data sets, Pandas datasets, etc., are not linearly separable. So, the kernel trick is applied to take the data to a higher dimensional feature space. But when feature space is getting larger, kernel function estimation becomes very expensive. Quantum computer offers a solution to this problem. With controllable quantum phenomena, there is a promise of achieving computational speed-ups through quantum algorithms [4]. Another phenomena is the similarity of the mathematical structure of the classical and quantum classifier. Moreover, the quantum feature map takes a feature vector to the Hilbert space automatically. So, kernel trick is unnecessary in quantum approaches. However, this is the Noisy Intermediate-Scale Quantum (NISQ) Technology era, where there are only less than 100 qubits in a quantum processor [5]. So, some scalable solutions are needed to be found for the current QML problems. Hence, hybrid quantum-classical architecture approaches are often taken to come up with solutions. One of the approaches contains a variational quantum circuit with a measurement operator and cost function. The variational quantum circuit are used to encode the data into the quantum circuit and tune the hyper-parameters with rotation gates, Rx, Ry and Rz [6, 7]. The cost function measures the similarity between the resultant label of the circuit and the actual label [8]. It also updates the parameters of the variational circuit. This process of updating the circuit hyper-parameters to train the circuit to recognize the correct label is often known as optimization. The optimization process of classifiers often gets stuck in local minima. The main goal is to avoid the local minima and to reach the global minima [8]. Only then the optimum hyper-parameters can be obtained for the quantum circuit. These optimal hyper-parameters ensure better solutions to the classification problems. With these ideas in mind, the experiments were conducted on the IRIS datasets. Some quantum classifier models are already proposed to address this classification problem. However, the training and testing accuracy were very poor and unstable. So, the circuit and the optimizer are not having the right hyper-parameters or the existing approaches get stuck in local minima. In this study, some experiments are performed to optimize VQC, and are tested on IRIS data set and breast cancer data set. The aim is to create an optimized quantum classifier that performs better or similar to the existing machine learning algorithm.

Related Works

The fundamental approach to classification problems with quantum computers is to encode the data with unitary gates, use some parameterized circuits, perform measurement operators and apply the cost function to update hyper-parameters of the machine learning algorithm [8]. In this approach, the training data sets were encoded with many different techniques. The parameterized circuit has CNOT and Rx, Ry, and Rz gates and measurement is performed on the first qubit [9]. A gradient descent optimizer is used to optimize the hyper-parameter of the parameterized circuit [9]. Another approach is using QuClassi architecture. In this method, a two-dimensional dataset can be encoded in the qubits. In the first qubit or controlled qubit, Hadamard operations are applied. On the rest of the four qubits, $R_y$ and $R_z$ gates are implemented. In this approach, the parameterized circuits are applied on different qubits with a SWAP circuit for optimization process. The measurement is performed on the controlled qubit. This model is quite successful [10]. In this study, Dr. Ying Mao, Dr. Qiang Guan, and others have got almost 95% accuracy while they were working with IRIS dataset. The third approach is to build Hybrid QNNs. Here, a combined classical and quantum neural networks is implemented for classification [11]. This model needs a lesser number of qubits. However, from the experiment with the MNIST data set, this model performances is quiet unstable. Quantum classifier architecture of Tensor flow works better than some classical machine learning algorithms. This architecture was applied to CIFAR10 dataset and found it to be effective [12]. However, the performance is not stable. In another experiment, 67.5% of accuracy was obtained. Aside from those, the ensemble method can be implemented to solve multiclass classification problem. In this method, multiple physical devices are combined and are tried to solve a multiclass classification problem [13]. However, this needs the collaboration of multiple physical quantum systems or technologies to solve a multiclass problem. Another problem is that not all physical devices perform the same. So, the performance of the ensemble method is very unstable and inefficient. Transfer learning with quantum computer is another quantum approach to resolve classification issues. But in one of the experiments, it is found that there is an over fitting problem [14]. Moreover, in some of the cases, Deutsch-Josza circuit is implementing along with classification circuit. Moreover, Dr. Caiwen Ding has proposed a QF-MixNN solution [15].

Mathematical Modeling of Classification Problem in Classical Setting

A dataset is given $\left\{ \begin{array}{l} \vec{x}_1, y_1, \vec{x}_2, y_2 \\ \dots \dots, \vec{x}_n, y_n \end{array} \right\}$ where $\vec{x}$ is a d-dimensional vector representing the numbers of features for algorithm and $y_i \in \{+1, -1\}$ for binary classification problem. $\vec{x}$ is another vector perpendicular to the decision line and $\vec{x}$ is a vector that shows a new data point. Now, to find out whether $\vec{x}$ is in the +ve or -ve side, $\vec{x}$

have to be projected onto $w$ to determine its location. If $\vec{x}+$ data points are for label, $y_i = +1$ and $\vec{x}-$ data points are for label, $y_i = -1$ and $b$ is a parameter, the equations for correctly classified data points are [16]:

$$\vec{w}.\vec{x}_+ + b \geq 1$$

$$\vec{w}.\vec{x}_- + b \leq 1$$

Now, for $y_i = (+1, -1)$, (1) and (2) equations can be rewritten as following [16]:

$$y_i(\vec{w}.\vec{x}_+ + b) - 1 = 0$$

where, the maximum width of the decision boundary is $\frac{2}{\|\vec{w}\|}$$

[16]. The support vectors lie at the edge of the boundary [17]. So, there are two constraints. One is equation (3) and another is the width of the boundary. If Lagrange multiplier is used, then the minimization of the result will be as below while honoring the both constraints. The primal form of the equation can be written as [18]:

$$L = \frac{1}{2} \| \vec{W} \|^2 - \sum_i \alpha_i [y_i(\vec{w}.\vec{x}_i + b) - 1]$$

Mathematical Modeling of Classification Problem in Quantum Setting

It is important to understand whether a VQC is mathematically or theoretically capable to solve any classification problem. In the classical machine learning setting, it has been seen that SVM is a linear classifier. Support vectors helps to determine that boundary between two classes. According to the equation (4), classification issues are just convex optimization problems which can be solved effectively with classical machines. But in this study, quantum algorithm is to be implemented to check whether the algorithm performs better than classical ones or any quantum advantage can be achieved. In this section, the theoretical analysis of the classification problem is discussed and it is shown that VQC is also a linear classifier like SVM. In the procedure, at first the dataset is needed to take data points to the higher dimensional vector space, $\emptyset : x \rightarrow \phi(z)$ where, $\emptyset$ is non-linear [19]. Then, the dataset is linearly classified. To ensure this, the following equation is to be found out [19]:

$$f_\theta(z) = \langle \phi(z) \mid w_\theta^+ ZW_\theta \mid \phi(z) \rangle \in [-1, +1]$$

At first, consider that [19], $H_\theta = W_\theta^+ ZW_\theta$ (6)

Where, $H_\theta$ is optimized over the Hamiltonian or the observable against the feature vectors. Now,

$$f_\theta(z) = \langle \phi(z) \mid H_\theta \mid \phi(z) \rangle$$ (7)

$< \phi(z) \mid H_\theta \mid \phi(z)$ is a 1-dimensional scalar number whose trace is equal to the number, itself.

$$f_\theta(z) = \text{Tr} \left[ \langle \phi(z) \mid H_\theta \mid \phi(z) \rangle \right]$$ (8)

So, $f_\theta(z) = \text{Tr} \left[ \langle \phi(z) \mid H_\theta \mid \phi(z) \rangle \right]$ (9)

It is also known that, $\|\phi(z)\| > \|\phi(z)\|$ is a density matrix in $2^n \times 2^n$ complex vector space which is another representation of $\phi(z)$ or mathematically,

$$\phi(z) \approx \|\phi(z)\| < \phi(z)$$ (10)

$$f_\theta(z) = \text{Tr} \left[ H_\theta \phi(z) \right]$$ (11)

If $H_\theta$ and $\phi(z)$ can be decomposed as following [19]:

$$H_\theta = \frac{1}{(2)^n} \sum_{\alpha \in 4^n} < H_\theta P_\alpha > P_\alpha$$ (12)

$$H_\theta = \frac{1}{(2)^n} \sum_{\alpha \in 4^n} \text{Tr} \left[ H_\theta P_\alpha \right] P_\alpha$$ (13)

Let, $h_\alpha(\theta) = \text{Tr} \left[ H_\theta P_\alpha \right]$ (14)

So, it can be written, $H_\theta = \frac{1}{(2)^n} \sum_{\alpha \in 4^n} h_\alpha(\theta) P_\alpha$ (15)

Similarly, $\phi(z) = \frac{1}{(2)^n} \sum_{\alpha \in 4^n} < \phi(z), P_\alpha > P_\alpha$ (16)

which can be written as [19]:

$$\phi(z) = \frac{1}{(2)^n} \sum_{\alpha \in 4^n} \phi(z) P_\alpha$$ (17)

So, equation (11) can be rewritten as following [19]:

$$f_\theta(Z) = \text{Tr} \left[ \left( \frac{1}{(2)^n} \sum_{\alpha \in 4^n} h_\alpha(\theta) P_\alpha \right) \left( \frac{1}{(2)^n} \sum_{\alpha \in 4^n} \phi(Z) P_\alpha \right) \right]$$ (18)

This can be combined and written as [19]:

$$f_\theta(Z) = \frac{1}{(4)^n} \text{Tr} \left[ \left( \sum_{\alpha \in 4^n} h_\alpha(\theta) \phi(Z) P_\alpha \right) \right]$$ (19)

$$ f _ {\theta} (Z) = \frac {1}{(4) ^ {n}} \sum_ {\alpha , \beta \in_ {4} ^ {n}} h _ {\alpha} (\theta) \phi (Z) \operatorname {T r} \left[ P _ {\alpha} P _ {\beta} \right] $$ Or, (20) It is known that [19], $$ \operatorname {T r} \left[ \mathrm {P} _ {\alpha} \mathrm {P} _ {\beta} \right] = 2 ^ {\mathrm {n}} \mathrm {i f} \alpha = \beta \mathrm {a n d} 0 \mathrm {i f} \alpha \neq \beta $$ (21) $$ \mathrm {f} _ {\theta} (Z) = \frac {1}{(2) ^ {n}} \sum_ {\alpha \in 4 ^ {n}} \mathrm {h} _ {\alpha} (\theta) \phi (Z) $$ So, (22) $$ f _ {\theta} (Z) \in [ - 1, + 1 ] $$ This (23) If threshold, [ ] b 1,1 ∈− , [ ] b 1, 1 ∈−+ +then, it can be written [19], ( ) 4 1 ( ) ( ) (2) $$ \operatorname {l a b e l} (Z) = \operatorname {s i g n} \left(\frac {1}{(2) ^ {n}} \sum_ {\alpha \in 4 ^ {n}} h _ {\alpha} (\theta) \phi (Z) + b\right) \tag {24} $$ which is equal to the solution, $$ \operatorname {L a b e l} (Z) = \operatorname {s i g n} \left(f _ {\theta} (Z) + b\right) \tag {25} $$ $$ x \approx f _ {\theta} (z), $$

, as x data point is mapped to

( ) f z θ , so, † ( ) ( ) Label Z

sign w x

$$ 0 = \operatorname {s i g n} \left(w ^ {\dagger} x + b\right) \tag {26} $$ Now, the primal form of the equation can easily be worked out just like classical SVM algorithm as from equation (3) and (4) by solving for the optimal separating hyper plane given as following:

( )

(27)

Classification with Variational Quantum Classifier Model

At first, the unitary gates ( ) U θ are applied to the initial states of the quantum circuit, | 0 n > in a quantum variation classifier (VQC). The parameters of ( ) U θ depend on the dataset of the problem. A variational circuit block ( ) W θ is added. This block is tuned, tested and optimized. After that, the circuit is measured on Z-basis. The result is a string of bit, { } 0,1 n z ∈ . Then, the string of bits is mapped to a cost function, C. The actual labels are compared with the labels from the circuit. After that, optimizer is used to optimize the circuit. The optimizer feed ( ) W θ with a new set of parameters. The circuit is rerun and from that the expectation value, ( ) ( ) ( ) ( ) † z z p x W M W x ϕ θ θ ϕ = ρ ρ ρ ρ is estimated. These values are again mapped to { } 1, 1 C ∈+ − [20]. The circuit is rerun again and again until the optimum hyper parameters are reached. This algorithm is applied on the test data to check the performance of the algorithm. This time the parameters of ( ) W θ are fixed.

In classical ML algorithm, the dataset, X, is imported into a ML model. The model takes the data and provides a prediction or answer. The model depends on the parameter, θ . Then there is a cost function which compares the performance of the model with actual answer. The optimizer optimizes the algorithm performance by updating the parameter of the ML model. The model is then applied to test dataset to check the performance of the algorithm. In QML, the classical ML model portion is replaced by a quantum circuit. This circuit consists of encoding and parameterized circuit. The measurement operator in the architecture is to obtain the output of the circuit. Before training a quantum circuit, the classical data are needed to be encoded into the quantum states. To do that, the unitary transformation gates are used. So, the raw data, d x ∈ ρ is mapped to a new quantum state, ( ) ( ) ( ) 2 n x x φ φ ⊗ >< ρ ρ Σ ò [21]. In this state, dimension of all linearly inseparable data is increased and mapped to higher dimensional Hilbert space. A variational quantum circuit is added to the new quantum state, ( ) ( ) ( ) 2 n x x φ φ ⊗ >< ρ ρ Σ ò [21]. This variational circuit depends on the parameter, θ . After that, the circuit measurement is performed on Z-basis and $$ \left(z\right) \epsilon \left\{- 1, + 1 \right\} $$ is calculated. [21] This is the result of

labeling bit string

{ } 0,1 n z ∈

based on the Boolean function

$$ f: \{0, 1 \} ^ {n} \rightarrow \{+ 1, - 1 \}, \mathrm {S o}, \mathrm {i f} $$ $$ f (z) \epsilon \{- 1, + 1 \} $$ is the label of the dataset, then probability of measuring one or another label is[4]:

$$ p _ {y} = \frac {1}{2} (1 + f (z) (< \varphi (\vec {x}) \left| W ^ {\dagger} (\vec {\theta}) f W (\vec {\theta}) \right| \varphi (\vec {x}) > $$ where, $$ f = \sum_ {z \in \{0, 1 \} ^ {n}} f (z) | z > < z | \tag {28} $$ The circuit diagram of this VQC model is shown below Figure 1:

Click to enlarge

The equation (28) is needed to be solved and it is

required to answer what is

$$ p _ {z} \mathrm {a n d i f} p _ {z} \mathrm {i s} \mathrm {p} _ {+ 1} \mathrm {o r}, p _ {- 1}. \mathrm {T h e} $$

result is compared with the actual label to determine the

performance of the quantum model.

A threshold has to be defined after that. Various optimization techniques are used to update the parameters of VQC model. These updated parameters are fed to the ( ) W θ block and are tuned and tested as long as optimum θ values for the classifier are not found. However, the values of θ are also depended on the unitary gates in the circuit. As Hilbert space is a complex vector space, some parameters of ( ) W θ is not accessible by quantum gate operation [22]. So, only the subset of those parameters can only be accessible to optimize the equation. It is also import to remember that the more parameter the circuit can access, the better the expressivity of the quantum circuit [23, 24]. But better expressivity does not mean that the circuit is going to be performing better in classification or any machine learning problem [8]. Moreover, it can be seen that the computation is basically done by quantum circuit. Cost analysis and optimization are done by classical machines. So, the quantum classifier is like a hybrid loop. This loop optimizes the circuit performance.

Components of Variational Quantum Classifier

Variational quantum classifiers have several essential components. In this experiment, only three of them are analyzed. They are:

1. Encoder,
2. Ansatz circuit and
3. Optimizer

Experimental Analysis and Results

As discussed earlier, the quantum classifier is a linear classifier. However, VQC classifier cannot classify more than two labels at a time. So, only two classes were extracted out from the given dataset at first. There are other approaches in QML which may have the chance to classify more than 2 classes at a time. In Figure 6, sample images of IRIS dataset are shown:

Click to enlarge

At first the structure of the problem is analyzed by visualizing the given data. In the Figures 7 and 8, IRIS dataset is plotted which contains three classes based on the patel length and patel width. In this setting, setosa and versicolor can be easily separated with linear boundary but it is hard to classify versicolor and virginica linearly.

Click to enlarge

Now, the pattern of data has to be checked based on the other two properties of IRIS dataset. They are- sepal length and sepal width. Here, it can be seen that virginica and setosa can be easily classified but it is hard to classify versicolor and virginica. Figure 8 shows the distribution of the three classes based on these two properties of IRIS dataset.

Click to enlarge

It can be understood from Figures 7 and 8 that there are four properties or features of each data. If VQC algorithm should be run on this dataset, it should be ensured that the qubit states alter with the values of the features of each data point [8]. As the data is four dimensional, four quantum bits are required [8]. The test dataset sample of IRIS dataset is shown in Figure 9 below:

Click to enlarge

Conclusion

From this study, it can be concluded that an optimized VQC can be developed to solve classification problem by analyzing the behavior of its components. Z feature map should be chosen as the encoder with circuit depth 2.

The ansatz of this study can be used as the tuning circuit which must also have depth 2. Additionally, TNC optimizer will help to obtain the best possible results from the VQC. This optimized VQC performs similar or better for lower dimensional dataset provided that the physical quantum computer is ideal or qubits are totally immunized from noise.