# data_poisoning_attacks_on_multitask_relationship_learning__ca29306c.pdf

Data Poisoning Attacks on Multi-Task Relationship Learning

Mengchen Zhao, Bo An, Yaodong Yu, Sulin Liu, Sinno Jialin Pan School of Computer Science and Engineering, Nanyang Technological University, Singapore 639798 zhao0204@e.ntu.edu.sg, {boan,ydyu,liusl,sinnopan}@ntu.edu.sg

Multi-task learning (MTL) is a machine learning paradigm that improves the performance of each task by exploiting useful information contained in multiple related tasks. However, the relatedness of tasks can be exploited by attackers to launch data poisoning attacks, which has been demonstrated a big threat to single-task learning. In this paper, we provide the first study on the vulnerability of MTL. Specifically, we focus on multi-task relationship learning (MTRL) models, a popular subclass of MTL models where task relationships are quantized and are learned directly from training data. We formulate the problem of computing optimal poisoning attacks on MTRL as a bilevel program that is adaptive to arbitrary choice of target tasks and attacking tasks. We propose an efficient algorithm called PATOM for computing optimal attack strategies. PATOM leverages the optimality conditions of the subproblem of MTRL to compute the implicit gradients of the upper level objective function. Experimental results on realworld datasets show that MTRL models are very sensitive to poisoning attacks and the attacker can significantly degrade the performance of target tasks, by either directly poisoning the target tasks or indirectly poisoning the related tasks exploiting the task relatedness. We also found that the tasks being attacked are always strongly correlated, which provides a clue for defending against such attacks.

Introduction

The security of machine learning algorithms has become a great concern in many real-world applications involving adversaries. The threats to machine learning systems can be classified as two kinds: exploratory attacks where attackers modify test examples in order to make machine learning algorithms produce erroneous outputs (Li and Vorobeychik 2014; Biggio et al. 2013), and causative attacks (a.k.a poisoning attacks) where attackers manipulate training examples to subvert the learned model (Barreno et al. 2010; Biggio, Nelson, and Laskov 2012; Xiao et al. 2015). Poisoning attacks usually occur when training data is collected from public sources (e.g., Internet users) and can be very harmful due to its long-lasting effect on the learned model. Studying poisoning attacks provides deep understanding of how well machine learning performs in adversarial training

Copyright c 2018, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved.

environment, which is critical for improving the robustness of real-world machine learning systems. In this paper, we formally analyze optimal poisoning attacks on multi-task learning (MTL) models, where multiple tasks are learned jointly to achieve better performance than single-task learning (Zhang and Yang 2017). Specifically, we focus on multi-task relationship learning (MTRL) models, a popular subclass of MTL models where task relationships are quantized and are learned directly from training data (Zhang and Yeung 2010; Liu, Pan, and Ho 2017). Many MTL-based machine systems collect training data from individual users to provide personalized services, including collaborative spam filtering and personalized recommendations, which makes them vulnerable to poisoning attacks launched by cyber criminals. For example, in an MTL-based recommender system, attackers can control a considerable number of user accounts either by hacking existing user accounts or creating fictitious user accounts. Previous works on poisoning attacks focus on single-task learning (STL) models, including support vector machines (Biggio, Nelson, and Laskov 2012), autoregressive models (Alfeld, Zhu, and Barford 2016) and factorization-based collaborative filterings (Li et al. 2016). However, none of them studies poisoning attacks on MTL models. Computing optimal poisoning attacks on MTL models can be much more challenging than on STL models, because MTL tasks are related with each other and an attack on one task might potentially influence other tasks. This also opens a door for the attacker to attack some accessible tasks and indirectly influence the unaccessible target tasks, which cannot be addressed by existing methods on poisoning STL models. The major contributions of our work are threefold. First, we formulate the optimal poisoning attack problem on MTRL as a bilevel program that is adaptive to any choice of target tasks and attacking tasks. Second, we develop a stochastic gradient ascent based algorithm called PATOM for solving the optimal attack problem, where the gradients are computed based on the optimality conditions of the convex subproblem of MTRL. Third, we demonstrate experimentally that MTRL is very sensitive to data poisoning attacks. The attacker can significantly degrade the performance of target tasks, by either directly poisoning the target tasks or indirectly poisoning the related tasks. Moreover, we study the change of task relationships under attacks and

The Thirty-Second AAAI Conference on Artificial Intelligence (AAAI-18) The Thirty-Second AAAI Conference on Artificial Intelligence (AAAI-18)

found that the attacking tasks usually have strong local correlations, which suggests that a group of strongly correlated tasks could be dangerous to the learner.

Related Work Data poisoning attacks against machine learning algorithms have become an important research topic in adversarial machine learning (Barreno et al. 2006; Huang et al. 2011; Kloft and Laskov 2010; Lowd and Meek 2005). The first work that provides formal study of poisoning attacks investigates the vulnerability of support vector machines, where an attacker progressively injects malicious data points to the training set in order to maximize the classification error (Biggio, Nelson, and Laskov 2012). Xiao et al.(2015) study poisoning attacks on feature selection algorithms and propose an algorithm that repeatedly optimizing the injected data until convergence. Mei and Zhu (2015b) propose an algorithmic framework for computing training set attacks. Recently, poisoning attacks have been analyzed on many important machine learning algorithms, including autoregressive models (Alfeld, Zhu, and Barford 2016), latent Dirichlet allocation (Mei and Zhu 2015a), and matrix factorizationbased collaborative filtering (Li et al. 2016). However, existing works focus only on STL models and the vulnerability of MTL models is left to be explored. Another line of research related to our work is MTL, which has been extensively studied in the literature. In general, MTL can be categorized into four classes: feature learning approaches, low-rank approaches, task clustering approaches, and task relationship approaches. Feature learning approaches aim to learn a common shared feature space among multiple tasks to boost the learning performance of each task (Argyriou, Evgeniou, and Pontil 2007). Low-rank approaches assume that the model parameters of different tasks share a low-rank structure, and discovery of such a low-rank structure could help learning a more precise model for each task (Ando and Zhang 2005). Task clustering approaches assume that different tasks form several task-clusters, each of which consists of similar tasks (Thrun and O Sullivan 1996). Task relationship learning aims to quantify and learn task relationship automatically from data, such that knowledge can be transferred among related tasks (Zhang and Yeung 2010). However, as we discussed, the vulnerability of MTL has never been studied. In this work, we fill the gap by investigating the vulnerability of task relationship learning approaches, which have proven to be effective in MTL.

Multi-Task Relationship Learning We denote by T = {Ti}m i=1 the set of learning tasks. For each task Ti, we are given a set of training data Di = {(xi j, yi j)|xi j Rd, j = 1, ..., ni}. The label yi j R if the task is a regression task and yi j { 1, +1} if the task is a binary classification task. Note that a multi-class classification problem can be easily decomposed to a set of binary classification problems using the one-vs-the-rest strategy (Fan et al. 2008). The goal of MTL is to jointly learn a prediction function fi(x) for each task. In this pa-

per, we consider linear prediction functions where fi(x) = (wi) x + bi, but note that it is easy to extend to nonlinear cases using kernel methods. For the ease of representation, we denote (x, 1) by x and denote (w, b) by w so that fi(x) = (wi) x. We consider a general multi-task relationship learning (MTRL) formulation (Zhang and Yeung 2010) as follows, which includes many existing popular MTL methods as its special cases (Evgeniou, Micchelli, and Pontil 2005; Evgeniou and Pontil 2004; Jacob, Vert, and Bach 2009; Kato et al. 2008).

j=1 l((wi) xi j, yi j) + λ1

2 tr(WΩ 1W ), (1)

s.t. Ω 0, tr(Ω) = 1,

where l( ) is an arbitrary convex loss function, W is a matrix whose i-th column wi is the weight vector of task Ti, Ω Rm m is the covariance matrix that describes positive, negative and unrelated task relationships. The first term in the objective function measures the empirical loss of all tasks with the term 1/ni to balance the different sample sizes of tasks. The second term in the objective function is to penalize the complexity of W, and the last term serves as the task-relationship regularization term. The first constraint ensures that the covariance matrix Ω is positive semi-definite, and the second constraint controls its complexity.

Data Poisoning Attacks on MTRL

In this section, we introduce the problem settings for the data poisoning attack on MTRL. We define three kinds of attacks based on real-world scenarios and propose a bilevel formulation for computing optimal attacks. We assume that the attacker aims to degrade the performance of a set of target tasks Ttar T by injecting data to a set of attacking tasks Tatt T. We denote by Di = {( xi j, yi j)| xi j Rd, j = 1, ..., ni} the set of malicious data injected to task i. Specially, Di = , i.e., ni = 0, if Ti Tatt. We define and study the following three kinds of attacks based on real-world scenarios.

Direct attack: Ttar = Tatt. Attacker can directly inject data to all the target tasks. For example, in product review sentiment analysis, each task is a sentiment classification task that classifies a review as negative or positive. On e-commerce platforms such as Amazon, attackers can directly attack the target tasks by providing crafted reviews to the target products.

Indirect attack: Ttar Tatt = . Attacker cannot inject data to any of the target tasks. However, he can inject data to other tasks and indirectly influence the target tasks. For example, personalized recommendations treat each user as a task and use users feedback to train personalized recommendation models. In such scenarios, attackers usually cannot access the training data of target tasks. However,

attackers can launch indirect attacks by faking some malicious user accounts, which will be treated as attacking tasks, and providing crafted feedback to the systems.

Hybrid attack: A mixture of direct attack and indirect attack where the attacker can inject data to both target tasks and attacking tasks.

We denote by L(D, w) = |D| k=1 l(w xk, yk) the empirical loss incurred by weight vector w on data set D, and define the attacker s utility function as the empirical loss on training data of the target tasks:

U = {i|Ti Ttar} L(Di, wi).

Following the Kerckhoffs principle (Kahn 1998) and existing works on poisoning attacks (Biggio, Nelson, and Laskov 2012; Li et al. 2016), we assume that the attacker has full knowledge of the victim MTRL model. In reality, attackers can either obtain the knowledge of victim models by exploiting insider threats (Greitzer et al. 2008) or probing the machine learning systems by sending queries from the outside (Lowd and Meek 2005). We then formulate the optimal attack problem as the following bilevel optimization problem. Problem (2) is the upper level problem, in which the objective function is the attacker s utility U. The variables of the upper level problem are the injected data points Di, which are usually constrained in real-world scenarios. For example, the injected data should have similar scale with the clean data. The lower level problem (Problem (3)) is an MTRL problem with training set consists of both clean and injected data points. The lower level problem can be regarded as the constraint of the upper level problem. In other words, the variables W used for computing the objective of Problem (2) should be the optimal solution of the lower level problem.

max { Di|Ti Tatt}

{i|Ti Ttar} L(Di, wi), (2)

s.t. Constraints on { Di|Ti Tatt},

1 ni + ni L(Di Di , wi )

2 tr(WW )+λ2

2 tr(WΩ 1W ),

(3) s.t. Ω 0, tr(Ω) = 1.

Computing Optimal Attack Strategies

In this section, we propose an algorithm called PATOM for computing optimal attack strategies. PATOM is a projected stochastic gradient ascent based algorithm that efficiently maximizes the injected data in the direction of increasing the empirical loss of target tasks. Since there is no close-form relation between the empirical loss and the injected data, we compute the gradients exploiting the optimality conditions of the subproblem of MTRL.

General Optimization Framework Bilevel problems are usually hard to solve due to their non-linearity, non-differentiability and non-convexity. In our bilevel formulation, although the upper level problem (2) is relatively simple, the lower level problem (3) is highly non-linear and non-convex. Inspired by (Li et al. 2016; Mei and Zhu 2015b; Xiao et al. 2015; Zhao et al. 2017), we use a projected gradient ascent method to solve our proposed bilevel problem. The idea is to iteratively update the injected data in the direction of maximizing the attacker s utility function U. In order to reduce the complexity of the optimal attack problem, we fix the labels of injected data yi j and optimize over the features of injected data xi j. The update rule is written as follows,

( xi j)t Proj X(( xi j)t 1 + η ( xi j)t 1U), (4)

where η is the step size, t denotes the t-th iteration, and X represents the feasible region of the injected data, which is specified by the first constraint in the upper level problem (2). We consider X as an ℓ2-norm ball with diameter r. Therefore, Proj X can be represented by:

Proj X(x) = x, if ||x||2 r, xr ||x||2 , if ||x||2 > r.

In order to compute the gradients ( xi j)t 1U, we first apply the chain rule to arrive at

( xi j)U = W U ( xi j)W. (5)

However, note that U is the sum of losses incurred by every point in the target tasks, the first term on the right side could be computationally expensive if the number of data points in target tasks is large. Therefore, we instead propose a projected stochastic gradient ascent based algorithm, called PATOM, to improve the scalability of our approach. The details of PATOM is shown in Algorithm 1. We first randomly initialize the injected data Di within the ℓ2-norm ball with diameter r. Using the injected data, we solve the MTRL problem (the lower level problem (3)), and obtain the initial values of the weight matrix W0 and the covariance matrix Ω0. In each iteration, we perform a projected stochastic gradient ascent procedure steps (7-10) on all the injected data. Specifically, for each data point (xp q,yp q) sampled from Dbatch, we compute the gradients of its associate loss l((wp t ) xp q,yp q) with respect to each injected data xi j. Therefore, by replacing U in (4) with l((wp t ) xp q,yp q) we have the stochastic version of the update rule as shown in (6). Then, with the updated injected data Di = Dt i, we solve the lower level problem (3) again to obtain a new weight matrix Wt and a new covariance matrix Ωt, which will be used in the next iteration.

( xi j)t Proj X(( xi j)t 1+η ( xi j)t 1l((wp t 1) xp q,yp q)). (6)

Gradients Computation In order to compute the gradients ( xi j)l((wp) xp q,yp q) in (6), we still apply the chain rule and obtain:

xi jl((wp) xp q,yp q) = wpl((wp) xp q,yp q) xi jwp. (7)

Algorithm 1: computing Poisoning ATtacks On Multi-task relationship learning (PATOM)

1 Input: Ttar, Tatt, step size η, attacker budget ni.

2 Randomly initialize D0 i = {(( xi j)0, ( yi j)0)|j = 1, ..., ni}, i Tatt.

3 Di = D0 i , , i Tatt.

4 Solve lower level problem (3) to obtain W0 and Ω0.

6 while t < tmax do

7 Sample a batch Dbatch from i Ttar Di.

8 for (xp q,yp q) Dbatch do

9 for i Tatt, j = 1... ni do

10 Update ( xi j)t according to (6).

11 Di = Dt i, i Tatt.

12 Solve (3) to obtain Wt and Ωt.

13 t t + 1.

We can see that the first term on the right side depends only on the loss function l( ) and is relatively easy to compute. However, the second term on the right side depends on the optimality conditions of lower level problem (3). In the rest of this section, we show how to compute the gradients with respect to two commonly used loss functions. For regression tasks, we adopt least-square loss: l1(w x, y) = (y w x)2. For classification tasks, we adopt squared hinge loss: l2(w x, y) = (1 yw x)2. We first fix Ω to eliminate the constraints of the lower level problem (3), and obtain the following sub-problem:

1 ni+ ni L(Di Di, wi)+λ1

2 tr(WΩ 1W ). (8)

As shown in (Zhang and Yeung 2010), MTRL problems can be solved by an alternating approach with Ω and W alternatingly fixed in each iteration. Also note that in bilevel optimization, the optimality of the lower level problem can be considered as a constraint to the upper level problem. Therefore, at convergence, we can treat Ω in Problem (8) as a constant-value matrix when computing the gradients. We then substitute the least-square loss function l1( ) into Problem (8) and reformulate it as the following constrained optimization problem:

j=1 (εi j)2+

j =1 (ˆεi j)2

2 tr(WΩ 1W ), (9)

s.t. εi j = yi j (wi) xi j, i, j,

ˆεi j = yi j (wi) xi j , i, j .

The Lagrangian of the problem (9) is:

j=1 (εi j)2+

j=1 (ˆεi j)2

2 tr(WΩ 1W )

j=1 αi j yi j (wi) xi j εi j

j =1 αi j ( yi j (wi) xi j ˆεi j )

The gradient of G with respect to W is:

G W = W(λ1Im + λ2Ω 1)

j=1 αi jxi je i +

j =1 αi j xi j e i

where Im is m m identity matrix, and ei is the i-th column of Im. By setting G

W = 0, we obtain:

j=1 αi jxi j+

j =1 αi j xi j

e i Ω(λ1Ω+λ2In) 1

(12) which implies that each task s weight vector wi can be represented as a linear combination of training data from all tasks. For simplicity in presentation, we denote by Φ = Ω(λ1Ω+λ2In) 1, and reexpress (12) as the following form:

j=1 αi jxi j +

j =1 αi j xi j

, p = 1...m.

Similarly, we substitute the squared hinge loss into the loss function L( ) in Problem (8), and obtain:

j=1 αi jyi jxi j+

j =1 αi j yi j xi j

, p = 1...m.

Given (13) and (14), we can compute the gradient in (6). In case of the least-square loss, we have:

xi jl((wp) xp q,yp q) = 2((wp) xp q yp q)xp q wp

xi j = 2((wp) xp q yp q)xp q αi jΦi,p. (15)

In case of the squared hinge loss, we have:

xi jl((wp) xp q,yp q)

= 2(yp q(wp) xp q 1)yp qxp q wp

xi j = 2(yp q(wp) xp q 1)yp qxp q yi j αi jΦi,p. (16)

Experimental Results

In this section, we first evaluate PATOM in terms of convergence and solution quality. Experimental results show that PATOM converges to local optima in less than 10 iterations and the attack strategies computed by PATOM significantly outperform baselines. Second, we study the task relationships under the data poisoning attacks and found that task relationships are very sensitive to the attacks. We also found that the tasks under attacking form strong correlations.

We use three real-world datasets to validate our proposed methods. The Landmine and the MNIST datasets are used for classification tasks and Sarcos dataset is used for regression tasks. For each dataset, all data points are divided by the maximum ℓ2 norm among them, so that all data points are within a ℓ2-norm ball with diameter 1. We consider this ball as the feasible region of the injected data in order to ensure that the injected data and the clean data are at the same scale. We use the area under the ROC curve (AUC) to evaluate the learning performance for classification tasks, and the normalized mean squared error (NMSE) for regression tasks. The higher AUC corresponds to the better performance for classification and the lower NMSE corresponds to the better performance for regression. The detailed description of the datasets are given below.

Sarcos1 relates to an inverse dynamics problem for a 7 degrees-of-freedom SARCOS anthropomorphic robot arm. The input is a 21-dimensional space that includes 7 joint positions, 7 joint velocities and 7 joint accelerations. Each input instance is associated with 7 joint torques. Following previous work (Zhang and Yeung 2010), each task is to learn a mapping from the 21-dimensional input space to one of the 7 torques. The dataset contains 44,484 training examples and 4,449 test examples.

Landmine2 consists of 29 tasks collected from various landmine fields. A data point in each task is represented by a 9-dimensional feature vector, and associated with a corresponding binary label ( 1 for landmine and - 1 for cluster). The feature vectors are extracted from radar images, concatenating four momentbased features, three correlation-based features, one energy ratio feature and one spatial variance feature. The tasks entail different numbers of data points, varying from 89 to 138 examples.

MNIST3 is a hand-written digit dataset with 10 classes. We use the one-vs-the-rest strategy to decompose the multi-class classification problem to 10 binary classification problems, and treat each binary classification problem as a task. To form the training set for each task, we randomly draw 300 data points of the designated digits and assign label +1 and draw an equal number of instances from other classes randomly and assign

1http://www.gaussianprocess.org/gpml/data/. 2http://people.ee.duke.edu/ lcarin/Landmine Data.zip. 3http://yann.lecun.com/exdb/mnist/.

Figure 1: Convergence of PATOM.

label -1 . The dataset contains 60,000 training examples and 10,000 test examples. We use principal component analysis (PCA) to reduce the feature space to a 128dimensional space.

Evaluating Convergence of PATOM Our first set of experiments study the convergence of PATOM on Sarcos dataset and Landmine dataset, with respect to regression tasks and classification tasks. On Sarcos dataset, we randomly draw 300 training examples and 600 test examples from the associated training and test set. For direct attacks, we select 3 tasks on Sarcos dataset and 15 tasks on Landmine dataset as the respective target tasks, and set the attacking tasks the same as the target tasks. For indirect attacks, we use the same target tasks as in the direct attack, and treat the rest of tasks as the attacking tasks. For hybrid attacks, we randomly select the same number of attacking tasks as in the indirect attack experiments from all tasks. We set the step size η = 100 and the lower level problem parameters λ1 = λ2 = 0.1. The batch size is set to be three times of the clean data. The number of injected data points in each task is set to be 20% of the clean data. Figure 1 shows the results of the convergence experiments on the two datasets, where x-axis represents the number of iterations in PATOM, and y-axis represents the NMSE averaged over target tasks of Sarcos dataset and the AUC averaged over target tasks on Landmine dataset, respectively. We can see that for all the three kinds of attacks on the two datasets, PATOM converges to local optima in less than 10 iterations, where at iteration 0 the injected data points are randomly initialized. Since existing optimization techniques cannot guarantee global optimal solutions for nonconvex programs, all of the solutions we find are approximate. However, we can get an estimation of the global optimal solution by selecting multiple start points and comparing the local optima. In our experiments, we observe very similar local optima values when choosing multiple start points. Based on this observation, we run PATOM with one start point in our remaining experiments.

Evaluating Solution Qualities Our second set of experiments evaluates the performance of MTRL under direct attacks and indirect attacks with respect to different datasets. On each dataset, we select 4 different pairs of target task set and attacking task set. Each pair

Figure 2: Solution quality comparison. Each figure corresponds to a choice of pair (Ttar, Tatt) of the associated dataset. The bold line (dashed line) with circle marker represents direct attacks (random direct attacks); the bold line (dashed line) with square marker represents indirect attacks (random indirect attacks). The budget represents the ratio of the number of injected data points to the number of clean data points.

(Ttar, Tatt) is chosen by randomly selecting half of tasks to form Ttar and the rest of tasks to form Tatt. We have |Ttar| = 4 and |Tatt| = 3 on Sarcos dataset, |Ttar| = 15 and |Tatt| = 14 on Landmine dataset, and |Ttar| = 5 and |Tatt| = 5 on MNIST dataset. For a pair (Ttar, Tatt) of each dataset, we compare the averaged NMSE or averaged AUC over the target tasks under four kinds of attacks: direct attacks, indirect attacks, random direct attacks and random indirect attacks. The last two kinds of attacks are treated as baselines, where the injected data points are randomly chosen. Figure 2 shows the results of quality comparison among the four kinds of attacks. Some interesting findings include:

Direct attacks are more effective than indirect attacks and random attacks given the same budget. From Figure 2, we can see that direct attacks significantly degrade the learning performance on all datasets. For example, on Sarcos dataset, direct attacks with 30% malicious data injected leads to about 50% higher averaged NMSE. However, note that in some scenarios, attackers may have larger budget for launching indirect attacks. Take the recommender system for example, attackers can provide arbitrary number of training data through the malicious accounts created by themselves. In such cases, indirect attacks are also big threats to the learning system.

Both direct attacks and indirect attacks computed by PATOM significantly outperform random attacks, respec-

tively, which demonstrates that the real-world attackers can do much better than just launching random attacks.

Different choices of pairs (Ttar, Tatt) influence the attacks performance. For example, we can see from the second figure of the first row of Figure 2, the indirect attacks lead to a higher loss than random direct attacks. However, in the third figure of the first row, the random direct attacks lead to a higher loss than indirect attacks.

Indirect attacks almost have no effect on MNIST dataset. Since we can easily learn good classifiers on MNIST dataset using only hundreds of training examples, each task does not need much help from other tasks and the task correlations are relatively low. Consequently, it is hard for the attacker to launch effective indirect attacks by exploiting task relationships.

The AUC slightly increases after budget=0.1 on MNIST dataset. Since our formulation maximizes the empirical loss on the training data but the AUC is computed based on the test data, we think the AUC on test data reaches the lower bound near budget=0.1 and slightly increases after that due to the distribution discrepancy between training data and test data. The intuition behind this is similar to overfitting in the convention of machine learning.

(a) No attack

(b) Tatt = {T2, ..., T5}

(c) Tatt = {T3, ..., T6}

(d) Tatt = {T4, ..., T7}

(e) No attack

(f) Tatt = {T5, ..., T19}

(g) Tatt = {T11, ..., T25}

(h) Tatt = {T15, ..., T29}

Figure 3: Visualization of task correlations under attacks. (a) - (d) are the results on Sarcos dataset and (e) - (f) are the results on Landmine dataset. The color of each grid represents the value of the correlation matrix, ranging from 1 (blue) to +1 (yellow) as shown in the color bar at the right side of each figure. The first figure of each row is the ground-truth task correlations learned with clean data. The target task set is set to be Ttar = {T1, T2, T3} on Sarcos dataset and Ttar = {T1, ..., T15} for Landmine dataset and remains the same under different attacks.

Evaluating Task Relationships

Our third set of experiments study the task relationships under different attacks. We fix the target task set as Ttar = {T1, T2, T3} on Sarcos dataset and Ttar = {T1, ..., T15} on Landmine dataset. Then, for each dataset, we select three different attacking task sets and compute three hybrid attacks. We set the amount of injected data to be 30% of the clean data with respect to each task. We convert the learned covariance matrices to correlation matrices and visualize them in Figure 3. Since the Sarcos dataset has 7 tasks and the Landmine dataset has 29 tasks, the learned correlation matrix is a 7 7 symmetric matrix on Sarcos dataset and 29 29 symmetric matrix on Landmine dataset.

Figure 3 shows that on both datasets the ground-truth task correlations are significantly subverted by the data poisoning attacks. For example, in Figure 3(a), the ground-truth correlation between tasks 2 and 3 is 0.99, which suggests that the two tasks are highly negatively correlated. However, in Figure 3(b), the correlation between tasks 2 and 3 becomes 0.99, meaning that the two tasks are highly positively correlated. Similar results can be found on Landmine dataset. Moreover, from Figures 3(b) - 3(d) and 3(f) - 3(h), we observe that the attacking tasks are usually highly positive correlated, in contrast with other tasks. This suggests that the machine learner needs to be aware of a group of tasks that form strong local correlations.

Conclusion and Future Work

This paper studies the data poisoning attacks on MTRL models. To the best of our knowledge, we are the first to study the vulnerability of MTL. We categorize the data poisoning attacks into direct attacks, indirect attacks and hybrid attacks based on the real-world scenarios. We propose a bilevel formulation that includes the three kinds of attacks to analyze the optimal attack problems. We propose PATOM, a stochastic gradient ascent based approach that leverages the optimality conditions of MTRL to compute the gradients. We evaluate PATOM in terms of convergence and solution quality on real-world datasets. Experimental results show that PATOM converges to local optima in less than 10 iterations and the attack strategies computed by PATOM significantly outperform baselines. We also study the task correlations under data poisoning attacks. The ultimate goal of studying data poisoning attacks is to develop effective defense strategies against such attacks. In future work, we will consider two classes of potential defense strategies for protecting MTL: data sanitization and improving the robustness of MTL. First, as shown in our experiments of evaluating task relationships, the tasks under attacking show a strong correlation with only 30% data injected. Therefore, the machine learner can examine the data from tasks that form strong local correlations, perhaps through human verifications. Moreover, once a task is demonstrated to be malicious, the learner can examine the tasks that strongly correlate to it, which will significantly

reduce the learner s effort in examining the data. Second, improving the robustness of MTL could also be an effective approach to defend against data poisoning attacks. MTL exploits the task relatedness to improve the performance of individual tasks, where such relatedness can also be exploited by the attacker to launch indirect attacks. A possible approach to improve the robustness of MTL is to differentiate the helpful relatedness and the harmful task relatedness, so that we can preserve the helpful relatedness and reduce the harmful relatedness during learning.

Acknowledgements

This research is supported by NRF2015 NCR-NCR003-004.

Alfeld, S.; Zhu, X.; and Barford, P. 2016. Data poisoning attacks against autoregressive models. In Proceedings of the 30th AAAI Conference on Artificial Intelligence, 1452 1458. Ando, R. K., and Zhang, T. 2005. A framework for learning predictive structures from multiple tasks and unlabeled data. Journal of Machine Learning Research 6:1817 1853. Argyriou, A.; Evgeniou, T.; and Pontil, M. 2007. Multitask feature learning. In Advances in Neural Information Processing Systems, 41 48. Barreno, M.; Nelson, B.; Sears, R.; Joseph, A. D.; and Tygar, J. D. 2006. Can machine learning be secure? In Proceedings of the 2006 ACM Symposium on Information, Computer and Communications Security, 16 25. Barreno, M.; Nelson, B.; Joseph, A. D.; and Tygar, J. 2010. The security of machine learning. Machine Learning 81(2):121 148.

Biggio, B.; Corona, I.; Maiorca, D.; Nelson, B.; ˇSrndi c, N.; Laskov, P.; Giacinto, G.; and Roli, F. 2013. Evasion attacks against machine learning at test time. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, 387 402. Biggio, B.; Nelson, B.; and Laskov, P. 2012. Poisoning attacks against support vector machines. In Proceedings of the 29th International Conference on Machine Learning, 1807 1814. Evgeniou, T., and Pontil, M. 2004. Regularized multi task learning. In Proceedings of the 10th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 109 117. Evgeniou, T.; Micchelli, C. A.; and Pontil, M. 2005. Learning multiple tasks with kernel methods. Journal of Machine Learning Research 6:615 637. Fan, R.-E.; Chang, K.-W.; Hsieh, C.-J.; Wang, X.-R.; and Lin, C.-J. 2008. Liblinear: A library for large linear classification. Journal of Machine Learning Research 9(Aug):1871 1874. Greitzer, F. L.; Moore, A. P.; Cappelli, D. M.; Andrews, D. H.; Carroll, L. A.; and Hull, T. D. 2008. Combating the insider cyber threat. IEEE Security & Privacy 6(1).

Huang, L.; Joseph, A. D.; Nelson, B.; Rubinstein, B. I.; and Tygar, J. 2011. Adversarial machine learning. In Proceedings of the 4th ACM Workshop on Security and Artificial Intelligence, 43 58. Jacob, L.; Vert, J.-p.; and Bach, F. R. 2009. Clustered multitask learning: A convex formulation. In Advances in neural information processing systems, 745 752. Kahn, D. 1998. Codebreakers: The comprehensive history of secret communication from ancient times to the Internet. Naval War College Review 51(4):153 155. Kato, T.; Kashima, H.; Sugiyama, M.; and Asai, K. 2008. Multi-task learning via conic programming. In Advances in Neural Information Processing Systems, 737 744. Kloft, M., and Laskov, P. 2010. Online anomaly detection under adversarial impact. In Proceedings of the 13th International Conference on Artificial Intelligence and Statistics, 405 412. Li, B., and Vorobeychik, Y. 2014. Feature cross-substitution in adversarial classification. In Advances in Neural Information Processing Systems, 2087 2095. Li, B.; Wang, Y.; Singh, A.; and Vorobeychik, Y. 2016. Data poisoning attacks on factorization-based collaborative filtering. In Prodeedings of the 30th Annual Conference on Neural Information Processing Systems, 1885 1893. Liu, S.; Pan, S. J.; and Ho, Q. 2017. Distributed multitask relationship learning. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 937 946. Lowd, D., and Meek, C. 2005. Adversarial learning. In Proceedings of the 11th ACM SIGKDD International Conference on Knowledge Discovery in Data Mining, 641 647. Mei, S., and Zhu, X. 2015a. The security of latent dirichlet allocation. In Proceedings of the 18th International Conference on Artificial Intelligence and Statistics, 681 689. Mei, S., and Zhu, X. 2015b. Using machine teaching to identify optimal training-set attacks on machine learners. In Proceedings of the 29st AAAI Conference on Artificial Intelligence, 2871 2877. Thrun, S., and O Sullivan, J. 1996. Discovering structure in multiple learning tasks: The TC algorithm. In Proceedings of the 13th International Conference on Machine Learning, 489 497. Xiao, H.; Biggio, B.; Brown, G.; Fumera, G.; Eckert, C.; and Roli, F. 2015. Is feature selection secure against training data poisoning? In Proceedings of the 32th International Conference on Machine Learning, 1689 1698. Zhang, Y., and Yang, Q. 2017. A survey on multi-task learning. ar Xiv preprint ar Xiv:1707.08114. Zhang, Y., and Yeung, D.-Y. 2010. A convex formulation for learning task relationships in multi-task learning. In Proceedings of the 26th Conference on Uncertainty in Artificial Intelligence, 733 742. Zhao, M.; An, B.; Gao, W.; and Zhang, T. 2017. Efficient label contamination attacks against black-box learning models. In Proceedings of the 26th International Joint Conference on Artificial Intelligence, 3945 3951.