# differentiable_rule_induction_from_raw_sequence_inputs__d03aeb9c.pdf

Published as a conference paper at ICLR 2025

DIFFERENTIABLE RULE INDUCTION FROM RAW SEQUENCE INPUTS

Kun Gao1, Katsumi Inoue2, Yongzhi Cao3, Hanpin Wang3, Feng Yang1

1Institute of High Performance Computing, Agency for Science, Technology and Research 2National Institute of Informatics 3Key Laboratory of High Confidence Software Technologies, School of Computer Science Peking University gaok@ihpc.a-star.edu.sg, inoue@nii.ac.jp, {caoyz,whpxhy}@pku.edu.cn, yangf@ihpc.a-star.edu.sg

Rule learning-based models are widely used in highly interpretable scenarios due to their transparent structures. Inductive logic programming (ILP), a form of machine learning, induces rules from facts while maintaining interpretability. Differentiable ILP models enhance this process by leveraging neural networks to improve robustness and scalability. However, most differentiable ILP methods rely on symbolic datasets, facing challenges when learning directly from raw data. Specifically, they struggle with explicit label leakage: The inability to map continuous inputs to symbolic variables without explicit supervision of input feature labels. In this work, we address this issue by integrating a self-supervised differentiable clustering model with a novel differentiable ILP model, enabling rule learning from raw data without explicit label leakage. The learned rules effectively describe raw data through its features. We demonstrate that our method intuitively and precisely learns generalized rules from time series and image data.

1 INTRODUCTION

The deep learning models have obtained impressive performances on tabular classification, time series forecasting, image recognition, etc. While in highly trustworthy scenarios such as health care, finance, and policy-making process (Doshi-Velez & Kim, 2017), lacking explanations for decisionmaking prevents the applications of these complex deep learning models. However, the rule-learning models have interpretability intrinsically to explain the classification process. Inductive logic programming (ILP) is a form of logic-based machine learning that aims to learn logic programs for generalization and interpretability from training examples and background knowledge (Cropper et al., 2022). Traditional ILP methods design deterministic algorithms to induce rules from symbolic data to more generalized formal symbolic first-order languages (Quinlan, 1990; Blockeel & De Raedt, 1998). However, these symbolic ILP methods face robustness and scalability problems when learning from large-scale and ambiguous datasets (Evans et al., 2021; Hocquette et al., 2024). With the sake of robustness of neural networks, the neuro-symbolic ILP by combining neural networks and ILP methods can learn from noisy data (Evans & Grefenstette, 2018; Manhaeve et al., 2018; Gao et al., 2022a) and can be applied to large-scale datasets (Yang et al., 2017; Gao et al., 2024; Phua & Inoue, 2024). However, existing neuro-symbolic ILP methods are mainly learned from discrete symbolic data or fuzzy symbolic data that the likelihoods are generated from a pre-trained neural network module (Evans et al., 2021; Shindo et al., 2023). Learning logic programs from raw data is prevented because of the explicit label leakage problem, which is common in neuro-symbolic research (Topan et al., 2021): The leakage happens by introducing labels of ground objects for inducing rules (Evans & Grefenstette, 2018; Shindo et al., 2023). In fact, generating rules to describe objects in raw data without label information is necessary, especially when some objects are easily overlooked or lack labels yet are important for describing the data.

Corresponding author.

Published as a conference paper at ICLR 2025

In this study, we introduce Neural Rule Learner (Neur RL), a framework designed to learn logic programs directly from raw sequences, such as time series and flattened image data. Unlike prior approaches prone to explicit label leakage, where input feature labels are extracted using pre-trained supervised neural networks and then processed with differentiable ILP methods to induce rules trained with input labels, our method bypasses the need for supervised pre-trained networks to generate symbolic labels. Instead, we leverage a pre-trained clustering model in an unsupervised manner to discretize data into distinct features. Subsequently, a differentiable clustering module and a differentiable rule-learning module are jointly trained under supervision using raw input labels. This enables the discovery of rules that describe input classes based on feature distributions within the inputs. As a result, the model achieves efficient training in a fully differentiable pipeline while avoiding the explicit label leakage issue. The contributions of this study include: (1) We formally define the ILP learning task from raw inputs based on the interpretation transition setting of ILP. (2) We design a fully differentiable framework for learning symbolic rules from raw sequences, including a novel interpretable rule-learning module with multiple dense layers. (3) We validate the model s effectiveness and interpretability on various time series and image datasets.

2 RELATED WORK

Inductive logic programming (ILP), introduced by Muggleton & Feng (1990); Muggleton & De Raedt (1994), learns logic programs from symbolic positive and negative examples with background knowledge. Inoue et al. (2014) proposed learning from interpretation transitions, and Phua & Inoue (2021) applied ILP to Boolean networks. Manhaeve et al. (2018); Evans & Grefenstette (2018) adapted neural networks for differentiable and robust ILP, while Gao et al. (2022b) introduced a neural network-based ILP model for learning from interpretation transitions. This model was later extended for scalable learning from knowledge graphs (Gao et al., 2022a; 2024). Similarly, Liu et al. (2024) proposed a deep neural network for inducing mathematical functions. In our work, we present a novel neural network-based model for learning logic programs from raw numeric inputs.

In the raw input domain, Evans & Grefenstette (2018) proposed ILP to learn rules from symbolic relational data, using a pre-trained neural network to map raw input data to symbolic labels. Unlike ILP, which enforces a strong language bias by predefining logic templates and limiting the number of atoms, Neur RL uses only predicate types as language bias. Similarly, Evans et al. (2021) used pre-trained networks to map sensory data to disjunctive sequences, followed by binary neural networks to learn rules. Shindo et al. (2023) introduced αILP, leveraging object recognition models to convert images into symbolic atoms and employing top-k searches to pre-generate clauses, with neural networks optimizing clause weights. Our approach avoids pre-trained large-scale neural networks for mapping raw inputs to symbolic representations. Instead, we propose a fully differentiable framework to learn rules from raw sequences. Additionally, unlike the memory-intensive rule candidate generation required by ILP and αILP, Neur RL eliminates this step, enhancing scalability.

Adapting autoencoder and clustering methods in the neuro-symbolic domain shows promise. Sansone & Manhaeve (2023) applies conventional clustering on input embeddings for deductive logic programming tasks. Misino et al. (2022) and Zhan et al. (2022) use autoencoders and embeddings to calculate probabilities for predefined symbols to complete deductive logic programming and program synthesis tasks. In our approach, we use an autoencoder to learn representations for sub-areas of raw inputs, followed by a differentiable clustering method to assign ground atoms to similar patterns. The differentiable rule-learning module then searches for rule embeddings with these atoms in a bottom-up manner (Cropper & Dumancic, 2022). Similarly, DIFFNAPS (Walter et al., 2024) also uses an autoencoder to build hidden features and explain raw inputs. Additionally, Bot CL (Wang et al., 2023) uses attention-based features to explain the ground truth class. However, the logical connections between the features used in DIFFNAPS and Bot CL to describe the ground truth class are unclear. In contrast, rule-based explainable models like Neur RL use feature conjunctions to describe the ground truth class.

Azzolin et al. (2023) use a post-hoc rule-based explainable model to globally explain raw inputs from local explanations. In contrast, our model directly learns rules from raw inputs, and Neur RL s performance is unaffected by other explainable models. Das et al. (1998) used clustering to split sequence data into subsequences and symbolize them for rule discovery. Our model combines clustering and rule-learning in a fully differentiable framework to discover rules from sequence and image data,

Published as a conference paper at ICLR 2025

with the rule-learning module providing gradient information to prevent cluster collapse (Sansone, 2023), where very different subsequences are assigned to the same clusters. He et al. (2018) viewed similar subsequences, called motifs, as potential rule bodies. Unlike their approach, we do not limit the number of body atoms in a rule. Wang et al. (2019) introduced SSSL, which uses shapelets as body atoms to learn rules and maximize information gain. Our model extends this by using raw data subsequences as rule body atoms and evaluating rule quality with precision and recall, a feature absent in SSSL.

3 PRELIMINARIES

3.1 LOGIC PROGRAMS AND INDUCTIVE LOGIC PROGRAMMING

A first-order language L = (D, F, C, V ) (Lloyd, 1984) consists of predicates D, function symbols F, constants C, and variables V . A term is a constant, variable, or expression f(t1, . . . , tn) with f as an n-ary function symbol. An atom is a formula p(t1, . . . , tn), where p is an n-ary predicate symbol. A ground atom (fact) has no variables. A literal is an atom or its negation; positive literals are atoms, and negative literals are their negations. A clause is a finite disjunction of literals, and a rule (definite clause) is a clause with one positive literal, e.g., αh α1 α2 αn. A rule r is written as: αh α1, α2, . . . , αn, where αh is the head (head(r)), and {α1, α2, . . . , αn} is the body (body(r)), with each atom in the body called a body atom. A logic program P is a set of rules. In first-order logic, a substitution is a finite set {v1/t1, v2/t2, . . . , vn/tn}, where each vi is a variable, ti is a term distinct from vi, and v1, v2, . . . , vn are distinct (Lloyd, 1984). A ground substitution has all ti as ground terms. The ground instances of all rules in P are denoted as ground(P).

The Herbrand base BP of a logic program P is the set of all ground atoms with predicate symbols from P, and an interpretation I is a subset of BP containing the true ground atoms (Lloyd, 1984). Given I, the immediate consequence operator TP : 2BP 2BP for a definite logic program P is defined as: TP (I) = {head(r) | r ground(P), body(r) I} (Apt et al., 1988). A logic program P with m rules sharing the same head atom αh is called a same-head logic program. A same-head logic program with n possible body atoms can be represented as a matrix MP [0, 1]m n. Each element akj in MP is defined as follows (Gao et al., 2022a): If the k-th rule is αh αj1 αjp, then akji = li, where li (0, 1) and Pp s=1 ls = 1 (1 i p, 1 < p, 1 ji n, 1 k m). If the k-th rule is αh αj, then akj = 1. Otherwise, akj = 0. Each row of MP represents a rule in P, and each column represents a body atom. An interpretation vector v I {0, 1}n corresponds to an interpretation I, where v I[i] = 1 if the i-th ground atom is true in I, and v I[i] = 0 otherwise. Given a logic program with m rules and n atoms, along with an interpretation vector v I, the function DP : {0, 1}n {0, 1}m (Gao et al., 2024) calculate the Boolean value for the head atom of each rule in P. It is defined as::

DP (v I) = θ(MP v I), (1)

where the function θ is a threshold function: θ(x) = 1 if x 1, otherwise θ(x) = 0. For a same-head logic program, the Boolean value of the head atom v(αh) is computed as v(αh) = Wm i=1 DP (v I)[i]. Additionally, Gao et al. (2024) replaces θ(x) with a differentiable threshold function and use fuzzy disjunction Wm i=1x[i] = 1 Qm i=1 x[i] to calculate the Boolean value of the head atom in a same-head logic program with neural networks.

Inductive logic programming (ILP) aims to induce logic programs from training examples and background knowledge (Muggleton et al., 2012). ILP learning settings include learning from entailments (Evans & Grefenstette, 2018), interpretations (De Raedt & Dehaspe, 1997), proofs (Passerini et al., 2006), and interpretation transitions (Inoue et al., 2014). In this paper, we focus on learning from interpretation transitions: Given a set E 2BP 2BP of interpretation pairs (I, J), the goal is to learn a logic program P such that TP (I) = J for all (I, J) E.

3.2 SEQUENCE DATA AND DIFFERENTIABLE CLUSTERING METHOD

A raw input consists of an instance (x, y), where x RT1 T2 Td represents real-valued observations with d variables, Ti indicates the length for i-th variable, and y {0, 1, . . . , u 1} is the class label, with u classes. A sequence input is a type of raw input where x RT1 is an ordered sequence of real-valued observations (Wang et al., 2019). A subsequence si of length l from sequence

Published as a conference paper at ICLR 2025

x = (x1, x2, . . . , x T ) is a contiguous sequence (xi, . . . , xi+l 1) (Das et al., 1998). All possible subsequence with length l include s1, ..., s T l+1.

Clustering can be used to discover new categories (Rokach & Maimon, 2005). In the paper, we adapt the differentiable k-means method (Fard et al., 2020) to group raw data x X. First, an autoencoder A generates embeddings hγ for x, where γ represents the parameters. Then, we set K clusters to discretize all raw data x. The representation of the k-th cluster is rk Rp, with p as the dimension, and R = {r1, . . . , r K} as the set of all cluster representations. For any vector y Rp, the function cf(y; R) returns the closest representation based on a fully differentiable distance function f. Then, the differentiable k-means problems are defined as follows:

x X f(x, A(x; γ)) + λ

k=1 f(hγ(x), rk)Gk,f(hγ(x), α; R), (2)

where the parameter λ regulates the trade-off between seeking good representations for x and the representations that are useful for clustering. The weight function G is a differentiable minimum function proposed by Jang et al. (2017):

Gk,f(hγ(x), α; R) = e αf(hγ(x),rk) PK k =1 e αf(hγ(x),rk ) , (3)

where α [0, + ). A larger value of α causes the approximate minimum function to behave more like a discrete minimum function. Conversely, a smaller α results in a smoother training process1.

4.1 PROBLEM STATEMENT AND FORMALIZATION

In this subsection, we define the learning problem from raw inputs using the interpretation transition setting of ILP and describe the language bias of rules for sequence data. We apply a neuro-symbolic description method from Marconato et al. (2023) to describe a raw input: (i) assume the label y depends entirely on the state of K symbolic concepts B = (a1, a2, . . . , a K), which capture highlevel aspects of x, (ii) the concepts B depend intricately on the sub-symbolic input x and are best extracted using deep learning, (iii) the relationship between y and B can be specified by prior knowledge B, requiring reasoning during the forward computing of deep learning models.

In this paper, we treat each subset of raw input as a constant and each concept as a ground atom. For example, in sequence data x X, if a concept increases from the point 0 to 10, the corresponding ground atom is increase(x[0 : 10]). We define the body symbolization function Lb to convert a continuous sequence x into discrete symbolic concepts, so that the set of all symbolic concepts B across all raw inputs X holds B = Lb(X). For binary class raw inputs, we use target atom ht to represents the target class, where ht being true means the instance belongs to the positive class. Then, a head symbolization function Lh maps an binary input x to a set of atoms: if the class of x is the target class t, then Lh(x) = {ht}; otherwise, Lh(x) = . Hence, a logic program P describing binary class raw inputs consists of rules with the same head atom ht2. We define the Herbrand base BP of a logic program as the set of all symbolic concepts B in all raw inputs and the head atom ht. For a raw input x, Lb(x) represents a set of symbolic concepts I BP , which can be considered an interpretation. The task of inducing a logic program P from raw inputs based on learning from interpretation transition is defined as follows:

Definition 1 (Learning from Binary Raw Input) Given a set of raw binary label inputs X, learn a same-head logic program P, where TP (Lb(x)) = Lh(x) holds for all raw inputs x X.

In the paper, we aim to learn rules to describe the target class with the body consisting of multiple sequence features. Each feature corresponds to a subsequence of the sequence data. Besides, each feature includes the pattern information and region information of the subsequence. Based on the

1We set α to 1000 in the whole experiments. 2We can transfer multiple class raw inputs to the binary class by setting the class of interest as the target class and the other classes as the negative class.

Published as a conference paper at ICLR 2025

Encoder Decoder

Differentiable

All subsquences s

of a series x

Reconstructed subquences s

Deep rule learning

Logic programs P r1r1

Label of the series x

Predicted label

Differentiable path

Loss for autoencoder

Loss for Deep FOL

(supervised)

Loss for differentiable k-

means (unsupervised)

layer Dense

v(ht) v(ht)

(a) Learning pipeline.

a1 a2 a1 a2 a3 a4 a3 a4

h (a1 a2 a3) (a3 a4) h (a1 a2 a3) (a3 a4)

a2 a2 a3 a3 a4 a4

Input nodes

First layer

Second layer

Fuzzy disjunction

a1 a2 a3 a1 a2 a3 0.34 0.23 0.36

a3 a4 a3 a4 0.41 0.45

(b) Deep rule-learning module.

Figure 1: The learning pipeline of Neur RL and the rule-learning module.

pattern information, we further infer the mean value and tendency information of the subsequence. Using the region predicates, we can apply Neur RL to the dataset, where the temporal relationships between different patterns play a crucial role in distinguishing positive from negative examples. Specifically, we use the following rules to describe the sequence data with the target class: ht patterni1(Xj1), regionk1(Xj1), . . . , patternin1(Xjn2), regionkn3(Xjn2), (4) where the predicate patterni indicates the i-th pattern in all finite patterns within all sequence data, the predicate regionk indicates the k-th region in all regions in a sequence, and the variable Xj can be substituted by a subsequence of the sequence data. For example, pattern1(x[0 : 5]) and region0(x[0 : 5]) indicate that the subsequence x[0 : 5] matches the pattern with index one and belongs to the region with index zero, respectively. A pair of atoms, patterni(X) regionk(X), corresponds to a feature within the sequence data. In this pair, the variables are identical, with one predicate representing a pattern and the other representing a region. We infer the following information from the rules in format (4): In a sequence input x, if all pairs of ground patterns and regions atoms substituted by subsequences in x are true, then the sequence input x belongs to the target class represented by the head atom ht.

4.2 DIFFERENTIABLE SYMBOLIZATION PROCESS

In this subsection, we design a differentiable body symbolization function Lb, inspired by differentiable k-means (Fard et al., 2020), to transform numeric sequence data x into a fuzzy interpretation vector v I [0, 1]n. This vector encodes fuzzy values of ground patterns and region atoms substituted by input subsequences. A higher value in v I[i] indicates the i-th atom in the interpretation I is likely true. Using the head symbolization function J = Lh(x), we determine the target atom s Boolean value v(ht), where v(ht) = 1 if J = {ht} and v(ht) = 0 if J = . Building on (Gao et al., 2024), we learn a logic program matrix MP [0, 1]m n such that Wm i=1DP (v I)[i] = v(ht) holds. The rules in format (4) are then extracted from MP , generalized from the most specific clause with all pattern and region predicates.

The architecture of Neur RL is shown in Fig. 1a. To learn a logic program P from sequence data, each input sequence x is divided into shorter subsequences s of length l with a unit step stride. An encoder maps subsequences s to an embedding space z, and a decoder reconstructs s from z3. The differentiable k-means algorithm described in Section 3.2 clusters embeddings z, grouping subsequences s with similar patterns into groups r. This yields fuzzy interpretation vectors v I and Boolean target atom value v(ht) for each sequence. Finally, the differentiable rule-learning module uses v I as inputs and v(ht) as labels to learn high-level rules describing the target class.

Now, we describe the method to build the differentiable body symbolization function Lb from sequence x to interpretation vector v I as follows: Let K be the maximum number of clusters based on the differentiable k-means algorithm, each subsequence s with the length l in x and the corresponding embedding z has a cluster index c (1 c K), and each sequence input x can be transferred

3We use fully connected layers as the encoder and decoder structures.

Published as a conference paper at ICLR 2025

into a vector of cluster indexes c {0, 1}K (|x| l+1). Additionally, to incorporate temporal or spatial information into the predicates of the target rules, we divide the entire sequence data into R equal regions. The region of a subsequence s is determined by the location of its first point, s[0]. For each subsequence s, we calculate its cluster index vector cs [0, 1]K using the weight function Gk,f as defined Eq. (2), where cs[k] = Gk,f(hγ(s), α; R) (1 k K and PK i=1 cs[i] = 1). A higher value in the i-th element of cs indicates that the subsequence s is more likely to be grouped into the i-th cluster. Hence, we can transfer the sequence input x RT to cluster index tensor with the possibilities of cluster indexes of subsequences in all regions cx [0, 1]K lp R, where lp is the number of subsequence in one region of input sequence data |x| /R . To calculate the cluster index possibility of each region, we sum the likelihood of cluster index of all subsequence within one region in cluster index tensor cx and apply softmax function to build region cluster matrix cp RK R as follows:

k=1 cx[i, k, j], cp[i, j] = ec[i,j] PK i=1 ec[i,j] . (5)

Since the region cluster matrix cp contains clusters index for each region. Besides, each cluster corresponds to a pattern. Hence, we flatten cp into a fuzzy interpretation vector v I, which serves as the input to the deep rule-learning module of Neur RL.

4.3 DIFFERENTIABLE RULE-LEARNING MODULE

In this section, we define the novel deep neural network-based rule-learning module denoted as NR to induce rules from fuzzy interpretation vectors. Based on the label of the sequence input x, we can determine the Boolean value of target atom v(ht). Then, using fuzzy interpretation vectors v I as inputs and Boolean values of the head atom v(ht) as labels y, we build a novel neural networkbased rule-learning module as follows: Firstly, each input node receives the fuzzy value of a pattern or region atom stored in v I. Secondly, one output node in the final layer reflects the fuzzy values of the head atom. Then, the neural network consists of k dense layers and one disjunction layer. Lastly, let the number of nodes in the k-th dense layer be m, then the forward computation process is formulated as follows:

i=1 (gk gk 1 g1(v I)) [i], (6)

where the i-th dense layer is defined as:

gi(xi 1) = 1 1 d Re LU (Mixi 1 d) , (7)

with d as the fixed bias4. The matrix Mi is the softmax-activated of trainable weights Mi Rnout nin in i-th dense layer of the rule-learning module:

Mi[j, k] = e Mi[j,k] Pnin u=1 e Mi[j,u] . (8)

With the differentiable body symbolization function Lb and NR, we now define a target function that integrates the autoencoder module, clustering module, and rule-learning module as follows:

min R,γe,γl P

s x,x X f1 (s, A (s; γe)) + λ1 PK k=1 f1 (hγe(s), rk) Gk,f1(hγe(s), α; R) + λ2f2 NR Lb (x) ; γl , y , (9) where γe and γl represent the trainable parameters in the encoder and rule-learning module, respectively. The loss function f1 and f2 are set to mean square error loss and cross-entropy loss correspondingly. The parameters λ1 and λ2 regulate the trade-off between finding the concentrated representations of subsequences, the representations of clusters for obtaining precise patterns, and the representations of rules to generalize the data5. Fig. 1a illustrates the loss functions defined in

4In our experiments, we set the fixed bias as 0.5. 5In our experiments, we assigned equal weights to finding representations, identifying clusters, and discovering rules by setting λ1 = λ2 = 1.

Published as a conference paper at ICLR 2025

the target function (9). The supervised loss functions are applied to the autoencoder (highlighted in orange boxes) and the rule-learning module (highlighted in blue boxes), respectively. The unsupervised loss function is applied to the differentiable k-means method (highlighted in gray box).

We analyze the interpretability of the rule-learning module NR. In the logic program matrix MP defined in Section 3.1, the sum of non-zero elements in each row, Pn j=1 MP [i, j], is normalized to one, matching the threshold in the function θ in Eq. (1). The conjunction of the atoms corresponding to non-zero elements in each row of MP can serve as the body of a rule. Similarly, in the rulelearning module of Neur RL, the sum of the softmax-activated weights in each layer is also one. Due to the properties of the activation function Re LU(x d)/(1 d) in each node, a node activates only when the sum of the weights approaches one; otherwise, it deactivates. This behavior mimics the threshold function θ in Eq. (1). Similar to the logic program matrix MP , the softmax-activated weights in each layer also have interpretability. When the fuzzy interpretation vector and Boolean value of the target atom fit the forward process in Eq. (6), the atoms corresponding to non-zero elements in each row of the i-th dense softmax-activated weight matrix Mi form a conjunction. From a neural network perspective, the j-th node in the i-th dense layer, denoted as ni j, represents a conjunction of atoms from the previous (i 1)-th dense layer (input layer). The likelihood of these atoms appearing in the conjunction is determined by the softmax-activated weights Mi[j, :] connecting to node ni j. In the final k-th dense layer, the disjunction layer computes the probability of the target atom ht. The higher likelihood conjunctions, represented by the nodes in the final k-th layer, form the body of the rule headed by ht. To interpret the rules headed by ht, we compute the product of all softmax-activated weights Mi as the program tensor: MP = Qk i=1 Mi, where MP [0, 1]m n. The program tensor has the same interpretability as the program matrix, with high-value atoms in each row forming the rule body and the target atom as the rule head. The number of nodes in the last dense layer m determines the number of learned rules in one learning epoch. Fig. 1b shows a neural network with two dense layers and one disjunction layer in blue. The weights in orange represent significant softmax-activated values, with input nodes as atoms and hidden nodes as conjunctions. Multiplying the softmax weights identifies the atoms forming the body of a rule headed by the target atom.

We use the following method to extract rules from program tensor MP : We set multiple thresholds τ [0, 1]. When the value of the element in a row of MP is larger than a threshold τ, then we regard the atom corresponding to these elements as the body atom in a rule with the format (4). We compute the precision and recall of the rules based on the discretized fuzzy interpretation vectors generated from the test dataset. The discretized fuzzy interpretation vector is derived as the flattened version of cp[i, j] = 1(cp[i, j] = maxk cp[k, j]), where maxk cp[k, j] represents the maximum value in the j-th column of cp. Then, we keep the high-precision rules as the output. The precision and recall of a rule are defined as: precision = nbody head/nbody and recall = nbody head/nhead, where nbody head denotes the number of discretized fuzzy interpretation vectors v I that satisfy the rule body and have the target class as the label. Similarly, nbody represents the number of discretized fuzzy interpretation vectors that satisfy the rule body, while nhead refers to the number of instances with the target class. When obtaining rules in the format (4), we can highlight the subsequences satisfying the pattern and region predicates above on the raw inputs for more intuitive interpretability.

To train the model, we first pre-train an autoencoder to obtain subsequence embeddings, then initialize the cluster embeddings using k-means clustering (Lloyd, 1982) based on these embeddings. Finally, we jointly train the autoencoder, clustering model, and rule-learning model using the target function (9) to optimize the embeddings, clusters, and rules simultaneously.

5 EXPERIMENTAL RESULTS

5.1 LEARNING FROM SYNTHETIC DATA

In this subsection, we evaluate the model on synthetic time series data based on triangular pulse signals and trigonometric signals. Each signal contains two key patterns, increasing and decreasing, with each pattern having a length of five units. To test Neur RL s learning capability on a smaller dataset, we set the number of inputs in both the positive and negative classes to two for both the training and test datasets. In each class, the difference between two inputs at each time point is a random number drawn from a normal distribution with a mean of zero and a variance of 0.1. The

Published as a conference paper at ICLR 2025

hp pattern0(X) region1(X) (p = 1, r = 1) hp pattern0(X) region1(X) (p = 1, r = 1)

(a) The signals based on triangular pulse.

hp pattern0(X) region2(X) (p = 1, r = 1) hp pattern0(X) region2(X) (p = 1, r = 1)

(b) The signals based on trigonometry function.

Figure 2: The synthetic data and the learned rules.

hp pattern2(X) region1(X) pattern1(Y) region2(Y) (p = 0.83, r = 0.89) hp pattern2(X) region1(X) pattern1(Y) region2(Y) (p = 0.83, r = 0.89)

(a) A rule from ECG dataset.

hp pattern0(X) region6(X) (p = 0.99, r = 0.90) hp pattern0(X) region6(X) (p = 0.99, r = 0.90)

(b) A rule from Italy Pow.Dem. dataset.

Figure 3: Selected rules from two UCR datasets.

positive test inputs are plotted in blue, and the negative test inputs in orange in Fig. 2. We set both the length of each region and the subsequence length to five units, and the number of clusters is set to three in this experiment. Neur RL is tasked with learning rules to describe the positive class, represented by the head atom hp. If the ground body atoms, substituted by subsequences of an input, hold true, the head atom hp is also true, indicating that the target class of the input is positive.

The rules with 1.0 precision (p) and 1.0 recall (r ) are shown in Fig. 2. The rule in Fig. 2a states that when pattern0 (cluster index 0) appears in region1 (from time points 5 to 9), the time series label is positive. Similarly, the rule in Fig. 2b indicates that when pattern0 appears in region2 (from time points 10 to 14), the label is positive. We highlight subsequences that satisfy the rule body in red in Fig. 2, inferring that the pattern0 indicates decreasing. These red patterns perfectly distinguish positive from negative inputs.

5.2 LEARNING FROM UCR DATASETS

In this subsection, we experimentally demonstrate the effectiveness of Neur RL on 13 randomly selected datasets from UCR (Dau et al., 2019), as used by Wang et al. (2019). To evaluate Neur RL s performance, we consider the classification accuracy from the rules extracted by the deep rulelearning module (denoted as Neur RL(R)) and the classification accuracy from the module itself (denoted as Neur RL(N)). The number of clusters in this experiment is set to five. The subsequence length and the number of regions vary for each task. We set the number of regions to approximately 10 for time series data. Additionally, the subsequence length is set to range from two to five, depending on the specific subtask.

The baseline models include SSSL (Wang et al., 2019), Xu (Xu & Funaya, 2015), and Bo W (Wang et al., 2013). SSSL uses regularized least squares, shapelet regularization, spectral analysis, and pseudo-labeling to auto-learn discriminative shapelets from time series data. Xu s method constructs a graph to derive underlying structures of time series data in a semi-supervised way. Bo W generates a bag-of-words representation for time series and uses SVM for classification. Statistical details, such as the number of classes (C.), inputs (I.), series length, and comparison results are shown in

Published as a conference paper at ICLR 2025

Table 1: Classification accuracy on 13 binary UCR datasets with different models.

Dataset C. I. Length Xu Bo W SSSL Neur RL(R) Neur RL(N)

Coffee 2 56 286 0.588 0.620 0.792 0.964 1.000 ECG 2 200 96 0.819 0.955 0.793 0.820 0.880 Gun point 2 200 150 0.729 0.925 0.824 0.760 0.873 Italy Pow.Dem. 2 1096 24 0.772 0.813 0.941 0.926 0.923 Lighting2 2 121 637 0.698 0.721 0.813 0.689 0.748 CBF 3 930 128 0.921 0.873 1.000 0.909 0.930 Face four 4 112 350 0.833 0.744 0.851 0.914 0.964 Lighting7 7 143 319 0.511 0.677 0.796 0.737 0.878 OSU leaf 6 442 427 0.642 0.685 0.835 0.844 0.849 Trace 4 200 275 0.788 1.00 1.00 0.833 0.905 Words Syn 25 905 270 0.639 0.795 0.875 0.932 0.946 Oliver Oil 4 60 570 0.639 0.766 0.776 0.768 0.866 Star Light Curves 3 9236 2014 0.755 0.851 0.872 0.869 0.907

Mean accuracy 0.718 0.801 0.859 0.842 0.891

Table 2: Comparisons with non-differentiable k-means clustering algorithm.

Dataset Non-differentiable k-means Differentiable k-means accuracy running time (s) accuracy running time (s)

Coffee 0.893 313 0.964 42 ECG 0.810 224 0.820 65 Gun point 0.807 102 0.740 35 Italy Pow.Dem. 0.845 114 0.926 63 Lighting2 0.672 1166 0.689 120

Tab. 1, with the best results in bold and second-best underlined. The Neur RL(N) achieves the most best results, with seven, and Neur RL(R) achieves five second-best results.

The learned rules from the ECG and Italy Pow.Dem. datasets in the UCR archive are shown in Fig. 3. In Fig. 3a, red highlights subsequences with the shape pattern2 in the region region1, while green highlights subsequences with the shape pattern1 in the region region2. The rule suggests that when data decreases between time points 15 to 25 and then increases between time points 30 to 40, the input likely belongs to the positive class. The precision and recall for this rule are 0.83 and 0.89, respectively. In Fig. 3b, red highlights subsequences with the shape pattern0 in the region region6. The rule indicates that a lower value around time points 18 to 19 suggests the input belongs to the positive class, with precision and recall of 0.99 and 0.90, respectively.

positive class negative class

PLMFEMXsr Ii Os MDE2o4INw Vt+e ZW0Lstet Vy9r5Tq N1kce Ti BUzg HD6g Dnf Qg CYQEPAMr/Dm KOf Fe Xc+Fq05J5s5hj9w Pn8Aqf KPA=</latexit>p = 1, r = 1

Figure 4: Learned rules from MNIST datasets.

To demonstrate the benefits of the fully differentiable learning pipeline from raw sequence inputs to symbolic rules, we compare the accuracy and running times (in seconds) between Neur RL and its deep rule-learning module using the non-differentiable k-means clustering algorithm (Lloyd, 1982). We use the same hyperparameter to split time series into subsequences for two methods. Results in Tab. 2 show that the differentiable pipeline significantly reduces running time without sacrificing rule accuracy in most cases.

5.3 LEARNING FROM IMAGES

In this subsection, we ask the model to learn rules to describe and discriminate two classes of images from MNIST datasets. We divide the MNIST dataset into five independent datasets, where each dataset contains one positive class and one negative class. For twodimensional image data, we first flatten the image data to one-dimensional sequence data. Then, the

Published as a conference paper at ICLR 2025

Number of Clusters Length of Regions Length of Subsequence

(a) On ECG dataset.

Number of Clusters Length of Regions Length of Subsequence

(b) On Italy Pow.Dem. dataset.

Figure 5: Results of ablation study. Hyperparameter values vs. accuracy.

sequence data can be the input for the Neur RL model to learn the rules. The lengths of subsequence and region are both set to three. Besides, the number of clusters is set to five. After generating the highlighted patterns based on the rules, we recover the sequence data to the image for interpreting these learned rules.

We present the rule for learning the digit one from the digit zero in Fig. 4, and the rules for learning other positive class from the negative class are shown in Fig. 6 in Appendix C. We present the rule with the precision larger than 0.9 and the highlight features (or areas) defined in the rules. Each highlight feature corresponds to a pair of region and pattern atoms in a learned rule. We can interpret rules like importance attention (Zhang et al., 2019), where the colored areas include highly discriminative information for describing positive inputs compared with negative inputs. For example, in Fig. 4, if the highlighted areas are in black at the same time, then the image class is one. Otherwise, the image class is zero. Compared with attention, we calculate the precision and recall to evaluate these highlight features quantitatively.

5.4 ABLATION STUDY

We conducted ablation studies using default hyperparameters, except for the one being explored. In this study, the number of clusters and regions influences the number of atoms in the learned rules, while the length of subsequences depends on the length of potential patterns. Hence, these three hyperparameters collectively describe the sensitivity of Neur RL. We present the accuracy of both Neur RL(N) and Neur RL(R) on the time series tasks ECG and Italy Pow.Dem. from the UCR archive.

From Fig. 5, we observe that Neur RL s sensitivity varies across tasks, and the subsequence length is a sensitive hyperparameter when the sequence length is small, as seen in the Italy Pow.Dem. dataset. Properly chosen hyperparameters can achieve high consistency in accuracy between rules and neural networks. Notably, increasing the number of clusters or length of regions does not reduce accuracy linearly. This suggests that the rule-learning module effectively adjusts clusters within the model. In addition, we conduct another ablation study in Tab. 3 of Appendix A to show that pre-training the autoencoder and clustering prevents cluster collapse (Sansone, 2023) and improves accuracy without increasing significant training time.

6 CONCLUSION

Inductive logic programming (ILP) is a rule-based machine learning method that supporting data interpretability. Differentiable ILP offers advantages in scalability and robustness. However, label leakage remains a challenge when learning rules from raw data, as neuro-symbolic models require intermediate feature labels as input. In this paper, we propose a novel fully differentiable ILP model, Neural Rule Learner (Neur RL), which learns symbolic rules from raw sequences using a differentiable k-means clustering module and a deep neural network-based rule-learning module. The differentiable k-means clustering algorithm groups subcomponents of inputs based on the similarity of their embeddings, and the learned clusters are used as input for the rule-learning module to induce rules that describe the ground truth class of input based on its features. Compared to other rule-based models, Neur RL achieves comparable classification accuracy while offering interpretability through quantitative metrics. Future goals include variables representing entire inputs to explain handwritten digits without label leakage (Evans et al., 2021). Another focus is learning from incomplete data (e.g., healthcare applications) to address real-world challenges.

Published as a conference paper at ICLR 2025

ACKNOWLEDGMENTS

We express our gratitude to the anonymous Reviewers for their valuable feedback, which has contributed significantly to enhancing the clarity and presentation of our paper. We also thank Xingyao Wang, Yingzhi Xia, Yang Liu, Jasmine Ong, Justina Ma, Jonathan Tan, Yong Liu, and Rick Goh for their supporting.

This work has been supported by AI Singapore under Grant AISG2TC2022006, Singapore. This work has been supported by the NII international internship program, JSPS KAKENHI Grant Number JP21H04905 and JST CREST Grant Number JPMJCR22D3, Japan. This work has also been supported by the National Key R&D Program of China under Grant 2021YFF1201102 and the National Natural Science Foundation of China under Grants 61972005 and 62172016.

Krzysztof R. Apt, Howard A. Blair, and Adrian Walker. Towards a theory of declarative knowledge. In Foundations of Deductive Databases and Logic Programming, pp. 89 148. Morgan Kaufmann, 1988.

Steve Azzolin, Antonio Longa, Pietro Barbiero, Pietro Li o, and Andrea Passerini. Global explainability of gnns via logic combination of learned concepts. In Proceedings of the 11th International Conference on Learning Representations, ICLR-23, 2023.

Hendrik Blockeel and Luc De Raedt. Top-down induction of first-order logical decision trees. Artif. Intell., 101(1-2):285 297, 1998.

Andrew Cropper and Sebastijan Dumancic. Inductive logic programming at 30: A new introduction. J. Artif. Intell. Res., 74:765 850, 2022.

Andrew Cropper, Sebastijan Dumancic, Richard Evans, and Stephen H. Muggleton. Inductive logic programming at 30. Mach. Learn., 111(1):147 172, 2022.

Gautam Das, King-Ip Lin, Heikki Mannila, Gopal Renganathan, and Padhraic Smyth. Rule discovery from time series. In Proceedings of the Fourth International Conference on Knowledge Discovery and Data Mining (KDD-98), pp. 16 22. AAAI Press, 1998.

Hoang Anh Dau, Anthony Bagnall, Kaveh Kamgar, Chin-Chia Michael Yeh, Yan Zhu, Shaghayegh Gharghabi, Chotirat Ann Ratanamahatana, and Eamonn Keogh. The UCR time series archive. IEEE/CAA Journal of Automatica Sinica, 6(6):1293 1305, 2019. https://www.cs.ucr. edu/ eamonn/time_series_data_2018/.

Luc De Raedt and Luc Dehaspe. Clausal discovery. Mach. Learn., 26(2-3):99 146, 1997.

Finale Doshi-Velez and Been Kim. Towards a rigorous science of interpretable machine learning. ar Xiv preprint ar Xiv:1702.08608, 2017.

Richard Evans and Edward Grefenstette. Learning explanatory rules from noisy data. J. Artif. Intell. Res., 61:1 64, 2018.

Richard Evans, Matko Bosnjak, Lars Buesing, Kevin Ellis, David P. Reichert, Pushmeet Kohli, and Marek J. Sergot. Making sense of raw input. Artif. Intell., 299:103521, 2021.

Maziar Moradi Fard, Thibaut Thonet, and Eric Gaussier. Deep k-means: Jointly clustering with k-means and learning representations. Pattern Recognit. Lett., 138:185 192, 2020.

Kun Gao, Katsumi Inoue, Yongzhi Cao, and Hanpin Wang. Learning first-order rules with differentiable logic program semantics. In Proceedings of the 31st International Joint Conference on Artificial Intelligence, IJCAI-22, pp. 3008 3014, 2022a.

Kun Gao, Hanpin Wang, Yongzhi Cao, and Katsumi Inoue. Learning from interpretation transition using differentiable logic programming semantics. Mach. Learn., 111(1):123 145, 2022b.

Kun Gao, Katsumi Inoue, Yongzhi Cao, and Hanpin Wang. A differentiable first-order rule learner for inductive logic programming. Artif. Intell., 331:104108, 2024.

Published as a conference paper at ICLR 2025

Yuanduo He, Xu Chu, Guangju Peng, Yasha Wang, Zhu Jin, and Xiaorong Wang. Mining rules from real-valued time series: A relative information-gain-based approach. In 2018 IEEE 42nd Annual Computer Software and Applications Conference, COMPSAC-18, pp. 388 397. IEEE Computer Society, 2018.

C eline Hocquette, Andreas Niskanen, Matti J arvisalo, and Andrew Cropper. Learning MDL logic programs from noisy data. In Proceedings of the 39th Annual AAAI Conference on Artificial Intelligence, AAAI-24, pp. 10553 10561. AAAI Press, 2024.

Katsumi Inoue, Tony Ribeiro, and Chiaki Sakama. Learning from interpretation transition. Mach. Learn., 94(1):51 79, 2014.

Eric Jang, Shixiang Gu, and Ben Poole. Categorical reparameterization with gumbel-softmax. In Proceedings of the 5th International Conference on Learning Representations, ICLR-17, 2017.

Ziming Liu, Yixuan Wang, Sachin Vaidya, Fabian Ruehle, James Halverson, Marin Soljacic, Thomas Y. Hou, and Max Tegmark. KAN: kolmogorov-arnold networks. Co RR, abs/2404.19756, 2024.

John W. Lloyd. Foundations of Logic Programming, 1st Edition. Springer, 1984.

Stuart P. Lloyd. Least squares quantization in PCM. IEEE Trans. Inf. Theory, 28(2):129 136, 1982.

Robin Manhaeve, Sebastijan Dumancic, Angelika Kimmig, Thomas Demeester, and Luc De Raedt. Deepproblog: Neural probabilistic logic programming. In Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems, Neur IPS-18, pp. 3753 3763, 2018.

Emanuele Marconato, Gianpaolo Bontempo, Elisa Ficarra, Simone Calderara, Andrea Passerini, and Stefano Teso. Neuro-symbolic continual learning: Knowledge, reasoning shortcuts and concept rehearsal. In Proceedings of the 40th International Conference on Machine Learning, ICML-23, volume 202, pp. 23915 23936. PMLR, 2023.

Eleonora Misino, Giuseppe Marra, and Emanuele Sansone. VAEL: bridging variational autoencoders and probabilistic logic programming. In Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems, Neur IPS-22, 2022.

Stephen H. Muggleton and Luc De Raedt. Inductive logic programming: Theory and methods. J. Log. Program., 19/20:629 679, 1994.

Stephen H. Muggleton and Cao Feng. Efficient induction of logic programs. In Algorithmic Learning Theory, First International Workshop, ALT-90, pp. 368 381. Springer/Ohmsha, 1990.

Stephen H. Muggleton, Luc De Raedt, David Poole, Ivan Bratko, Peter A. Flach, Katsumi Inoue, and Ashwin Srinivasan. ILP turns 20 - biography and future challenges. Mach. Learn., 86(1): 3 23, 2012.

Andrea Passerini, Paolo Frasconi, and Luc De Raedt. Kernels on prolog proof trees: Statistical learning in the ILP setting. J. Mach. Learn. Res., 7:307 342, 2006.

Yin Jun Phua and Katsumi Inoue. Learning logic programs using neural networks by exploiting symbolic invariance. In Proceedings of the 30th international conference on Inductive Logic Programming, ILP-21, volume 13191 of Lecture Notes in Computer Science, pp. 203 218. Springer, 2021.

Yin Jun Phua and Katsumi Inoue. Variable assignment invariant neural networks for learning logic programs. In Proceedings of the 18th International Conference on Neural-Symbolic Learning and Reasoning, Ne Sy-24, volume 14979 of Lecture Notes in Computer Science, pp. 47 61. Springer, 2024.

J. Ross Quinlan. Learning logical definitions from relations. Mach. Learn., 5:239 266, 1990.

Lior Rokach and Oded Maimon. Clustering methods. In The Data Mining and Knowledge Discovery Handbook, pp. 321 352. Springer, 2005.

Published as a conference paper at ICLR 2025

Emanuele Sansone. The triad of failure modes and a possible way out. In Proceedings of the 37th Annual Conference on Neural Information Processing Systems on the 4th Workshop on Self Supervised Learning: Theory and Practice (SSL, Neur IPS-23), 2023.

Emanuele Sansone and Robin Manhaeve. Learning symbolic representations through joint generative and discriminative training. In Proceedings of the 11th International Conference on Learning Representations on Neurosymbolic Generative Models Workshops (Ne Sy-Ge Ms, ICLR-23), 2023.

Hikaru Shindo, Viktor Pfanschilling, Devendra Singh Dhami, and Kristian Kersting. αILP: thinking visual scenes as differentiable logic programs. Mach. Learn., 112(5):1465 1497, 2023.

Sever Topan, David Rolnick, and Xujie Si. Techniques for symbol grounding with SATNet. In Marc Aurelio Ranzato, Alina Beygelzimer, Yann N. Dauphin, Percy Liang, and Jennifer Wortman Vaughan (eds.), Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems, Neur IPS-21, pp. 20733 20744, 2021.

Nils Philipp Walter, Jonas Fischer, and Jilles Vreeken. Finding interpretable class-specific patterns through efficient neural search. In Proceedings of the 38th AAAI Conference on Artificial Intelligence, AAAI-24, volume 38, pp. 9062 9070, 2024.

Bowen Wang, Liangzhi Li, Yuta Nakashima, and Hajime Nagahara. Learning bottleneck concepts in image classification. In Proceedings of the 34th IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR-23, pp. 10962 10971, 2023.

Haishuai Wang, Qin Zhang, Jia Wu, Shirui Pan, and Yixin Chen. Time series feature learning with labeled and unlabeled data. Pattern Recognit., 89:55 66, 2019.

Jin Wang, Ping Liu, Mary Fenghua She, Saeid Nahavandi, and Abbas Z. Kouzani. Bag-of-words representation for biomedical time series classification. Biomed. Signal Process. Control., 8(6): 634 644, 2013.

Zhao Xu and Koichi Funaya. Time series analysis with graph-based semi-supervised learning. In 2015 IEEE International Conference on Data Science and Advanced Analytics, DSAA-15, pp. 1 6. IEEE, 2015.

Fan Yang, Zhilin Yang, and William W. Cohen. Differentiable learning of logical rules for knowledge base reasoning. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems, NIPS-17, pp. 2319 2328, 2017.

Eric Zhan, Jennifer J. Sun, Ann Kennedy, Yisong Yue, and Swarat Chaudhuri. Unsupervised learning of neurosymbolic encoders. Trans. Mach. Learn. Res., 2022, 2022.

Han Zhang, Ian J. Goodfellow, Dimitris N. Metaxas, and Augustus Odena. Self-attention generative adversarial networks. In Proceedings of the 36th International Conference on Machine Learning, ICML-19, volume 97 of Proceedings of Machine Learning Research, pp. 7354 7363. PMLR, 2019.

Published as a conference paper at ICLR 2025

A ABLATION STUDY ON PRE-TRAINING

In this section, we conduct experiments with and without pre-training the autoencoder and clustering model to investigate whether pre-training improves accuracy. The results, including accuracy and running time (in seconds), are presented in Tab. 3.

Table 3: Ablation study: With vs. without pre-training autoencoder and clustering model. The first accuracy is achieved using Neur RL(N), while the second accuracy is obtained with Neur RL(R).

Dataset With pre-training Without pre-training Accuracy Running time (s) Accuracy Running time (s)

Coffee 1.00, 0.96 42 0.83, 0.81 30 ECG 0.88, 0.82 65 0.87, 0.64 53 Italy Pow.Dem. 0.92, 0.93 63 0.75, 0.80 61 Gun Point 0.87, 0.76 35 0.86, 0.43 31 Lighting2 0.75, 0.69 120 0.64, 0.60 63

The experiment results indicate the pre-training autoencoder and clustering model can improve the accuracy of both Neur RL(N) and Neur RL(R) without increasing significant training time.

B THE LINK OF THE MODEL

The model and data can be found here: https://github.com/gaokun12/Neur RL

C CLASSIFYING AND EXPLAINING MNIST DATA

Each column in Fig. 6 corresponds to a learning task. There are six rules presented in Fig. 6 to describe the positive class from the negative class.

p = 0.94, r = 0.31 p = 0.94, r = 0.31

p = 0.99, r = 0.43 p = 0.99, r = 0.43

p = 0.90, r = 0.76 p = 0.90, r = 0.76

p = 1, r = 1 p = 1, r = 1

p = 98, r = 89 p = 98, r = 89

Figure 6: Rules from MNIST datasets.