Authentication Scheme for Preventing Data Duplicate in Cloud Storage

Abbreviations

IT: Information Technology; IIM: Inverted Index Method; T: Tag; L: label.

Introduction

Cloud computing is one of the new emerging technologies which has been helping several organizations to save their money and time, it also provides convenience to the cloud end users. Cloud provides many benefits in terms of cost and on demand services and real time communication [1, 2]. Cloud computing provides online information technology (IT) resources which ensure the results desired by the user at any time. It allows users to borrow cloud computing resources such as server, software and storage [3]. Cloud storage is one of the cloud computing resources which provides virtual storage to cloud users based on demand and it is also used for different purposes depend on the users. Some use cloud storage as backup purposes while some users use data storage to substantiate their applications. There are some users that move their archive storage to the cloud where more storage capacity can be achieved at lower cost, instead of wasting fund on acquiring additional physical storage. The cloud storage has becoming organizations’ utility where end users can virtually store their data without bothering the entire mechanism. The increasing usage of internet and data availability and accessibility on social media continue to exert influences on very large-scale data in everyday activities. Many cloud vendors such as Netflix, Amazon and Flipkart often perform data collection, data mining, and data analysis from different sources and allow sharing of large volumes of analyzed data over the internet, this data can be accessible through cloud storage [4]. Today, the amount of data in the cloud storage continues increasing to become large volume of data and all cloud users are expected to reach the cloud services at any time irrespective of their location. Due to increasing amount of stored data in the cloud storage, the data disks are unable to recognize duplicate data that appear on the disk. Data duplicates affect storage space of the disk and more of data duplicates affect the performance and usage of the disk’s speed and space [5]. When cloud storage devices are occupied with large amount of duplicate data, the deduplication techniques which are capable of detecting and removing any data of the same type or the data that are of the same content are however, needed [4].

Deduplication is the process that enhances cloud storage efficiency which aims to manage storage space for only one replicate of the data in the storage (SNIA). However, aiming to address the data duplication challenges, many studies have proposed different deduplication methods and the literature revealed that some of the studies focus on hash functions such as Inverted Index Method and cryptographic algorithms such as AES, DES and Blowfish etc. to secure files over cloud. But notwithstanding, deduplication with user authentication has not been given much attention as numbers of studies have proposed different techniques to reduce or eliminate data duplication in cloud storage. This study used an Apriori algorithm augmented with optimization techniques to authenticate and validate every data that were stored in the cloud storage. The technique used in this study guaranteed that every data in the cloud storage (datacenter) has no duplicates. That is, the authentication method used eliminates set of duplicates data that might come from either adversaries or even come by error with the intension to occupy the storage space with redundancy data. The method used also keeps only unique and essential data, thus it is significantly clearing the storage space for incoming data. The optimization and breadth first-search techniques used enhanced the system throughput when unnecessary processing that might come through Apriori algorithm was avoided.

Large amount of cloud storage infrastructures is filled with duplicate data records; however, aiming to address the data duplication issues, literature revealed that many studies have proposed different techniques. Many studies have focused on techniques for deduplication using biometric user authentication while many studies focused on cryptographic and hash function techniques to authentication user so as to remove duplicate data in the cloud storage. Literature revealed that some of the studies that used cryptography techniques for deduplication were not practically implemented for authentication scheme and while some of the techniques are weak due to information leaks. Public-key cryptography is exposed to attacks such as phishing, spoofing, shoulder surfing and other attacks. In the situation of exposure to attacks, a malicious third party intercepts a public key when it is sent to one of the parties involved. Moreover, other problems of using public-key encryption system for deduplication is the processing speed, the encryption and decryption processes which involved in cryptography creates non-scalability of the method. Achieving the system throughput which is one of the metrics of the deduplication might be difficult. Among the studies that were reviewed include the works of Assam A, et. al. [6]; Wong and Kim [7]; Yujuan T, et al. [8]; Maharasi M, et al. [9]; Aghili H [10]; Akhila K, et al. [11]; Maragatharajan and Prequiet [12]; Yang Z, et al. [13]; Ryan NSW, et al. [14]; Qinlu and Bilin [15]; AbdulSalam and Assmaa [16].

Assam A, et al. [6] examined different authentication techniques in cloud environments. But it was observed that some of the deduplication using simple password-based authentication lacks positive recognition as far as cloud data deduplication is concerned. However, there is need for a strong deduplication with authentication scheme to prevent data duplication in the cloud storage. This study used associative data mining (Apriori) algorithm which capable of comparing input and stored data ( X Y ∪) whether they appear together in the data storage to detect duplicate using S (for support) and C (for confidence). The study also used an optimization technique to avoid unnecessary processing which may cause delay when computing support.

Wong and Kim [7] proposed a concept of biometric-based authentication scheme in cloud computing system, the study discussed the challenges and other limitations of traditional methods alongside with different attacks scenarios, such as tracking of an individual to leak his/her confidential data, misuse of biometric data etc. The study further explained that the privacy of cloud-based biometric authentication might not capable to resolve some technical challenges of authentication scheme cloud system. But, the Apriori algorithm proposed in this study used associative rules to discover patterns in the datasets which systematically detect duplication between input data and the stored data when they were compared. Results include finding the relationship among the data items with the aid of breadth first-search algorithm which improved the performance of Apriori to quickly remove any duplicate data in the cloud storage.

Yujuan T, et al. [8] proposed a system which its architecture consists of three-tier systems: file agent, master server agent and storage server agent. Whenever the clients sign up for substitute services, then file agents will be installed on the clients’ machines, while service providers produce master server agent and storage server agent in cloud data center to service the substitute services request from the clients. But, cost and time to transfer file (transfer overhead) from one server to another is one of the challenges of their architecture which composed three subsystems. An optimization technique was used to augment the Apriori algorithm to eliminate unnecessary processing which can cause delay when searching for item sets and computing the support.

Maharasi M, et al. [9] proposed attribute-based encryption system for removal of data duplicates in the cloud- based storage. When a file storage request is sent, each data provider firstly creates a tag (T) and a label (L) connected with the data, and then the data is encrypted underneath of an access structure over a set of attributes. But, large number of tags and labels constituted system difficult when retrieving data. However, the ABE systems do not support secure deduplication which leads to the application of the system to be more expensive for some business storage services. Moreover, the technique is not scalable for large amount of data. The Apriori augmented with optimization technique used in this study is a simple association technique with feasible implementation which removes duplicates in the cloud storage. It is less expensive which is easy to be deployed for any business storage services. More so, the system used in this study achieved scalability when large amount of data were involved.

Aghili H [10] did a comparative analysis of symmetric algorithms like DES, triple-DES, AES and Blowfish. The analysis proved that Blowfish takes less time for execution. The author used Blowfish for encryption, applied IBM deduplication service on JPG image, and it was concluded that blowfish is a best suitable algorithm with respect to security and the algorithm processing. The weaknesses of blowfish are that the encryption key may get to the receiver through unsecured transmission channel. In blowfish encryption algorithm each user must have a unique key, so as number of users increasing, to manage the key also becoming very difficult. Blowfish decryption process is also times consuming and processing speed is very slow. An optimization technique was used to augment the Apriori algorithm to eliminate unnecessary processing which can cause delay when searching for item sets and computing the support.

Akhila K, et al. [11] examined various deduplication techniques such as ClouDedup, DupLESS, HEDup, and SecDup etc. and most of the algorithms used were based on convergent key method. One of the drawbacks of convergent key is compromising of the key which hindered the quality of the algorithms. The Apriori augmented with optimization technique used in this study is a simple association techniques with feasible implementation which removes duplicates in the cloud storage. It is less expensive which can also be deployed for any business storage services.

Maragatharajan and Prequiet [12], proposed encrypted data using proxy re-encryption method. The user chooses random number to constitute the symmetric key and the encryption key is communicated to the cloud service provider to check for duplicate data with the help of tokens. If any duplicate file is found, it will be communicated to an authorized party. The scheme allows uploading and downloading of files, checking for duplicates, and deleting the duplicate files. This scheme provides security and privacy of files. But, the drawback of the method is that when it checks and duplicate files were found, this will be communicated to an authorized party. However, the author failed to realize that information communicating to the authorized party can leaks to an unauthorized third party who may compromise the information before getting to an authorized party. The Apriori algorithm used aims to discover patterns which technically detect duplication between input data and the stored data when they were compared. Finding the pattern between input data and the stored data helped the system to quickly remove any duplicate data.

Yang Z, et al. [13] proposed a method which is called droplet strips input data streams onto multiple storage servers, the method capable to limit number of stored data clusters on each server and also ensure that fingerprint index is fitted into the system memory. It was revealed that buffering layer in droplet performs better for only small data clusters and hash function crashes is likely an issue with droplet deduplication. It was also observe that if fingerprint is compromised, it is not possible for the fingerprint identifiers to be replaced. The association augmented with optimization that was used in this study enhanced the system searching for duplicates process and the distribution and finding relationship among data items is not differ from one another.

Ryan NSW, et al. [14] proposed a rapid asymmetric maximum. The method uses bytes value to declare the threshold instead of using hashes. The technique utilizes a fix-sized and a variable-sized window to find a maximum- valued byte which is the point of entry for the system to detect and prevent duplicate. But the drawback is that it likely delivers more metadata, and the process will make the system slow. An optimization technique was used to augment the Apriori algorithm to eliminate unnecessary processing which can cause delay when searching for data item sets and computing the support to the relationship between input data and the stored data.

Qinlu and Bilin [15] proposed RFD-HDFS and FD-HDFS which was based on hash function, the method proved that both RFD-HDFS and FD-HDFS framework not only implement deduplication function but also effectively reduces the disk utilization of duplicate files. But, one of the drawbacks is that the applications require hash function and probability of hash colliding in SHA cannot be overlooked. Therefore, hash crashes is likely to be an issue when using the method for deduplication because in a situation where a bit of information gets hash number that contrasted with the record of existing hash numbers, the bit of information is viewed as a duplicate. The Apriori algorithm used discovered patterns which technically detect duplication between input data and the stored data when they were compared. Finding the pattern helped the system to quickly remove any duplicate data.

AbdulSalam and Assmaa [16] proposed content-based TTTD chunk algorithm as a deduplication system that was decomposed into three stages which include chunking, hashing and indexing, and matching. The hashing and indexing stage computes the hash value for the whole chunk and the value is added to the index table. The technique computes the hash values for each chunk to search for the similarity of the chunks using a file name or file type in the dataset. The literature revealed that the process is a time consuming to complete the matching operations. Furthermore, the content-based TTTD used the histogram to represent the chunk of dataset while using statistical distribution to representing the chunks in one dataset may differ from one another. The association augmented with optimization used in this study enhanced the searching process and the distribution of data items will not differ from one another.

**Materials and Methods**

In this study, let $X$ denotes input data while $Y$ denotes the stored data. The association rule $X \rightarrow Y$ is interpreted as a set of data that satisfies the conditions in $X$ which are also likely to satisfy the conditions in $Y$. This shows that $X$ and $Y$ are conjunctions of attribute value-pairs and $S$ (for support) is the probability that $X$ and $Y$ appear together in data storage and $C$ (for confidence) is the conditional probability that $X$ appears in data storage when $Y$ is present. The association rule used in the study is an if-THEN rule which uses two measures to quantify the support and confidence of the rule to find the relationship between input data and the stored data to predict whether the input data has occurred in the data storage. The association rule used in the study uses inference engine to make decision based on support and used the measure’s engine to ascertain the reliability of the decision made. The association rule detects duplication when input data and the stored data were compared, the conditions in input data must satisfy the conditions in stored data, the input data and the stored data are conjunctions of attribute value-pairs where $S$ (for support) indicates that input data and the stored data appear together in the data storage $(X \cup Y)$. Both equation 1 and 2 and the algorithm 1 were used to model the association rule for support which found relationship between the input data and the stored data.

$$\forall X, Y: (X \subseteq Y) \Rightarrow s(X) \geq s(Y)$$ (1)

$$s(X \rightarrow Y) = \frac{\sigma(X \cup Y)}{N}$$ (2)

Algorithm 1

Step1: $X \subseteq \sum, Y \subseteq \sum$

Step2: $X \cap Y = \emptyset$

Step3: $\exists i \in X, 1 \leq i \leq n$

Step4: $\exists j \in Y, 1 \leq j \leq n$

The $C$ (for confidence) which indicates that conditionally $X$ appears in the data storage when $Y$ is present and the rule shows the level of correlation between an input data and the stored data $(X$ and $Y)$ in the data storage. If the $C$ (for confidence) is low, this indicates that the input data is already occurred in the data center storage but, if the $C$ (for confidence) is high, it indicates that the input data has not occurred in the data storage. The equation 3 modelled confidence by dividing the support for whether the input data has duplicate in the data storage using correlation for measurement.

$$c(X \rightarrow Y) = \frac{\sigma(X \cup Y)}{\sigma(X)}$$ (3)

The equation 4 depicts correlation measurement between input data and the data stored to determine whether the $C$ (for confidence) is high or low before decision is made that input data has occurred or not occurred in the data storage.

$$C(x, y) = \frac{\sum_{i=1}^{m} \left(x_i - \mu_i\right) \left(y_i - \mu_i\right)^2}{\sqrt{\sum_{i=1}^{m} \left(x_i - \mu_i\right) \sum_{i=1}^{m} \left(y_i - \mu_i\right)^2}}$$ (4)

Discussion of Results

Usage of system across enterprise might be varied and if a storage system accommodates multiple virtual machines for different users, a deduplication system should provide significant storage savings. A deduplication system in this study was implemented and tested at Federal Polytechnic, Ilaro. Authentication scheme for data deduplication scheme using Apriori augmented with optimization techniques was implemented in JavaScript on the portable windows-based systems. The study focused on the usage of Apriori algorithm to find the relationship among the data items to detect duplicate and a single edge server was used as an external storage system. Implementation was done on a single storage connector which provides the software functionality through a directory on the disks and also manages external storage systems which includes databases and servers. The JavaScript file system library which was used can be deployed on any file system compatible with the library. The system is flexible to detect duplicates in cloud system and as well as capable of detecting duplicates in any local storage. The same two files were copied into the software directory and the system detected that the two files were the same as it is shown in the Figure 2. As the system used machine learning approach, it did not rely on dataset like other different machine learning approaches which have been used by other studies but it mapped the incoming data with stored data to check for duplicate.

Click to enlarge

Conclusion

Deduplication is a promising technique to optimize storage system. It can greatly minimize the amount of data, energy consumption in the data center, and thereby reduce drastically the storage cost. Based on the extensive review of deduplication techniques, the literature revealed that authentication scheme using Apriori algorithm which was used in this study is a powerful tool for deduplication processes. Ability of the Apriori to compare and finding the relationship between all incoming data and stored data has strengthened the system to remove all duplicate data for the large cloud data center.