# causal_abstraction_inference_under_lossy_representations__0ee13be8.pdf

Causal Abstraction Inference under Lossy Representations

Kevin Xia 1 Elias Bareinboim 1

The study of causal abstractions bridges two integral components of human intelligence: the ability to determine cause and effect, and the ability to interpret complex patterns into abstract concepts. Formally, causal abstraction frameworks define connections between complicated low-level causal models and simple high-level ones. One major limitation of most existing definitions is that they are not well-defined when considering lossy abstraction functions in which multiple low-level interventions can have different effects while mapping to the same high-level intervention (an assumption called the abstract invariance condition). In this paper, we introduce a new type of abstractions called projected abstractions that generalize existing definitions to accommodate lossy representations. We show how to construct a projected abstraction from the low-level model and how it translates equivalent observational, interventional, and counterfactual causal queries from low to high-level. Given that the true model is rarely available in practice we prove a new graphical criteria for identifying and estimating high-level causal queries from limited low-level data. Finally, we experimentally show the effectiveness of projected abstraction models in high-dimensional image settings.

1. Introduction

The ability to determine cause and effect, and the ability to interpret complex patterns into abstract concepts, are two integral components of human intelligence. From the causality perspective, causal reasoning is vital in planning courses of actions, determining blame and responsibility, and generalizing across changing environments. From the abstraction perspective, humans generally grasp better intuition when understanding something at a high-level. For

1Causal AI Lab, Columbia University. Correspondence to: Kevin Xia <kmx2000@columbia.edu>.

Proceedings of the 42 nd International Conference on Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

example, a human can easily parse the object in an image as a dog or a car instead of interpreting it as a collection of pixel values. Combining these two modes of reasoning is vital for building more advanced AI systems.

Causal inference is often studied under the semantics of structural causal models (SCMs) (Pearl, 2000). An SCM models reality with a collection of mechanisms and exogenous distributions. Each SCM induces a collection of distributions categorized into three successively more descriptive layers known as the Ladder of Causation or Pearl Causal Hierarchy (PCH) (Pearl & Mackenzie, 2018; Bareinboim et al., 2022). These three layers refer to the observational (L1), interventional (L2), and counterfactual (L3) distributions. In many causal inference tasks, the goal is to infer a quantity from a higher layer using data from lower layers, a problem known as cross-layer inference. It is understood that it is generally impossible to infer higher layer information without additional assumptions (a result known as the Causal Hierarchy Theorem or CHT (Bareinboim et al., 2022)), so understanding the necessary assumptions for performing inferences is a key component of any causal inference task.

Existing works on causal abstractions have made significant progress in defining abstraction principles, proving insightful properties, and learning abstraction functions in practice (Rubenstein et al., 2017; Beckers & Halpern, 2019; Beckers et al., 2019; Geiger et al., 2023; Massidda et al., 2023; Zennaro et al., 2023; Felekis et al., 2024). Causal abstractions are typically studied by comparing a high-level model MH, defined over high-level variables VH, with its low-level counterpart ML, defined over VL. An abstraction function τ maps from VL to VH, and MH is formally defined as an abstraction of ML if it satisfies key properties with respect to τ such as commutativity with interventions. More recently, this notion has been relaxed to only enforcing properties between distributions of MH and ML from the PCH (Xia & Bareinboim, 2024). For example, rather than saying MH is a full abstraction of ML, one can say that MH is an abstraction of ML specifically for interventional quantities in L2 or for a single causal effect P(y | do(x)) L2. Xia & Bareinboim (2024) also shows the synergy between causal abstraction theory and representation learning (Bengio et al., 2013), which has shown great success in many deep learning applications by mapping high-dimensional data like images or text to simpler representation spaces. These definitions of

Causal Abstraction Inference under Lossy Representations

causal abstractions have accomplished formalizing a broad topic of human intelligence into mathematical language.

One particular limitation of existing definitions of abstractions is known as the Abstract Invariance Condition (AIC), which states, informally, that two values cannot be abstracted together if they have different downstream impacts. This is illustrated in Fig. 1. For example, a nutritionist may have collected data on two types of cholesterol, HDL and LDL, and are studying their impact on heart disease (Steinberg, 2007; Truswell, 2010). They would like to abstract the two together by summing them as total cholesterol (TC). However, this violates the AIC, as it is known that HDL decreases rate of heart disease while LDL increases it, so the sum is ambiguous (a lossy representation).1 Nonetheless, it may still be desirable to have a consistent formalism in which these kinds of ambiguous abstractions are welldefined, since in many practical settings (where representation learning or dimensionality reduction is needed), the AIC is clearly violated or is impossible to verify.

In this paper, we study this extension of causal abstractions, which we later define as projected abstractions, referring to the idea that an abstraction that violates the AIC results in a loss of information that is then characterized in the exogenous space. The proposed formalism generalizes abstractions both on the SCM and on the PCH level to allow for mathematically consistent abstractions even with AIC violations. Projected abstractions have many uses in practice, resulting in tractable causal inference and high-quality causal sampling even in the presence of extreme dimensionality reduction, a result which we show in the experiments.

To summarize, in Sec. 2, we generalize abstractions to settings which the AIC does not hold and provide an algorithm for constructing the high-level model. In Sec. 3, we show how to perform causal inference from data within this class of abstractions when the true model is not observed. In Sec. 4, we empirically demonstrate the power of abstractions at performing causal inference in high-dimensional image settings. All proofs can be found in App. A. Appendices can be found in the full technical report, Xia & Bareinboim (2025).

1.1. Preliminaries

We now introduce the notation and definitions used throughout the paper. We use uppercase letters (X) to denote random variables and lowercase letters (x) to denote corresponding values. Similarly, bold uppercase (X) and lowercase (x) letters denote sets of random variables and values respectively. We use DX to denote the domain of X and DX = DX1 DXk for the domain of X = {X1, . . . , Xk}. We denote P(X = x) (often short-

1See App. C Ex. 7 for a more concrete explanation.

Figure 1: An illustration of AIC violations. On the low level, two different interventions may be performed (e.g., X x1 and X x2). However, after applying the abstraction function τ to obtain the high-level model, both interventions are mapped to the same result (τ(x1) = τ(x2) = x H). If ML behaves differently under x1 compared to x2, MH cannot stay consistent with both models.

ened to P(x)) as the probability of X taking the values x under the distribution P(X).

We utilize the basic semantic framework of structural causal models (SCMs) (Pearl, 2000), following the presentation in Bareinboim et al. (2022).

Definition 1 (Structural Causal Model (SCM)). An SCM M is a 4-tuple U, V, F, P(U) , where U is a set of exogenous variables (or latents ) that are determined by factors outside the model; V is a set {V1, V2, . . . , Vn} of (endogenous) variables of interest that are determined by other variables in the model that is, in U V; F is a set of functions {f V1, f V2, . . . , f Vn} such that each f Vi is a mapping from (the respective domains of) UVi Pa Vi to Vi, where UVi U, Pa Vi V \ Vi, and the entire set F forms a mapping from U to V. That is, for i = 1, . . . , n, each f Vi F is such that vi f Vi(pa Vi, u Vi); and P(U) is a probability function defined over the domain of U.

Each M induces a causal diagram G, where every Vi V is a vertex, there is a directed arrow (Vj Vi) for every Vi V and Vj Pa Vi, and there is a dashed-bidirected arrow (Vj L9999K Vi) for every pair Vi, Vj V such that UVi and UVj are not independent (Markovianity is not assumed). Our treatment is constrained to recursive SCMs, which implies acyclic causal diagrams, with finite discrete domains over endogenous variables V.

Counterfactual (and also interventional and observational) quantities can be computed from SCM M as follows:

Definition 2 (Layer 3 Valuation (Bareinboim et al., 2022, Def. 7)). An SCM M induces layer L3(M), a set of distributions over V, each with the form P(Y ) = P(Y1[x1], Y2[x2],...) such that

P M(y1[x1], y2[x2], . . . ) = (1) Z

DU 1 Y1[x1](u) = y1, Y2[x2](u) = y2, . . . d P(u)

Causal Abstraction Inference under Lossy Representations

where Yi[xi](u) is evaluated under Fxi := {f Vj : Vj V \ Xi} {f X x:X Xi}. L2 is the subset of L3 for which all xi are equal, and L1 is the subset for which all Xi = .

Each Yi corresponds to a set of variables in a world where the original mechanisms f X are replaced with constants xi for each X Xi; this is also known as the mutilation procedure. This procedure corresponds to interventions, and we use subscripts to denote the intervening variables (e.g. Yx) or subscripts with brackets when the variables are indexed (e.g. Y1[x1]). For instance, P(yx, y x ) is the probability of the joint counterfactual event Y = y had X been x and Y = y had X been x .

We use the notation Li(M) to denote the set of Li distributions from M. We use Z to denote a set of quantities from Layer 2 (i.e. Z = {P(Vzk)}ℓ k=1), and Z(M) denotes those same quantities induced by SCM M (i.e. Z(M) = {P M(Vzk)}ℓ k=1).

The theory of causal abstractions developed in this paper build on the foundations of constructive abstraction functions, under which individual distributions of the PCH are well-defined between low and high-level models.

Definition 3 (Inter/Intravariable Clusterings (Xia & Bareinboim, 2024, Def. 5)). Let M be an SCM over V.

1. A set C is said to be an intervariable clustering of V if C = {C1, C2, . . . Cn} is a partition of a subset of V. C is further considered admissible w.r.t. M if for any Ci C and any V Ci, no descendent of V outside of Ci is an ancestor of any variable in Ci. That is, there exists a topological ordering of the clusters of C relative to the functions of M.

2. A set D is said to be an intravariable clustering of variables V w.r.t. C if D = {DCi : Ci C}, where DCi = {D1 Ci, D2 Ci, . . . , Dmi Ci } is a partition (of size mi) of the domains of the variables in Ci, DCi (recall that DCi is the Cartesian product DV1 DV2 DVk for Ci = {V1, V2, . . . , Vk}, so elements of Dj Ci take the form of tuples of the value settings of Ci).

Definition 4 (Constructive Abstraction Function (Xia & Bareinboim, 2024, Def. 6)). A function τ : DVL DVH is said to be a constructive abstraction function w.r.t. inter/intravariable clusters C and D iff

1. There exists a bijective mapping between VH and C such that each VH,i VH corresponds to Ci C;

2. For each VH,i VH, there exists a bijective mapping between DVH,i and DCi such that each vj H,i DVH,i corresponds to Dj Ci DCi; and

3. τ is composed of subfunctions τCi for each Ci C such that v H = τ(v L) = (τCi(ci) : Ci C), where τCi(ci) = vj H,i if and only if ci Dj Ci. We also apply the same notation for any WL VL such that WL is a union of clusters in C (i.e. τ(w L) = (τCi(ci) : Ci C, Ci WL)).

Finally, we state the AIC formally below.

Definition 5 (Abstract Invariance Condition (AIC)). Let ML = UL, VL, FL, P(UL) be an SCM and τ : DVL DVH be a constructive abstraction function relative to C and D. The SCM ML is said to satisfy the abstract invariance condition (AIC, for short) with respect to τ if, for all v1, v2 DVL such that τ(v1) = τ(v2), u DUL, Ci C, the following holds:

τCi f L V (pa(1) V , u V ) : V Ci

= τCi f L V (pa(2) V , u V ) : V Ci , (2)

where pa(1) V and pa(2) V are the values corresponding to v1 and v2.

A table summarizing the notation can be found in App. A.1, detailed explanations of these definitions can be found in App. A.2, and additional useful definitions from prior work can be found in App. A.3.

2. Abstractions under AIC Violations

The abstract invariance condition (AIC) states, in words, that two low-level values cannot map to the same high-level value if they have different downstream effects. This is a critical property that must hold for existing definitions of abstractions to be well-defined. In this paper, we will use the following running example to illustrate the key points.

Example 1. For concreteness, consider a setting in which different insurance companies (Z) offer various insurance plans (X), which affect whether an insurance claim is approved (Y ). For simplicity, suppose there are two insurance companies (z1 and z2) that offer three insurance plans (x1, x2, and x3), and the claim is either approved (Y = 1) or not approved (Y = 0). Suppose the true model M = ML = UL, VL, FL, P(UL) is described as

UL = {UZ, U z1 X , U z2 X , U x1 Y , U x2 Y , U x3 Y }

VL = {Z, X, Y }

f L Z (u Z) = u Z f L X(z, uz1 X , uz2 X ) = uz X f L Y (x, ux1 Y , ux2 Y , ux3 Y ) = ux Y

Causal Abstraction Inference under Lossy Representations

P(UZ = z1) = 0.5 P(U z1 X )={x1 0.4; x2 0.1; x3 0.5} P(U z2 X )={x1 0.1; x2 0.4; x3 0.5} P(U x1 Y = 1) = 0.9, P(U x2 Y = 1) = 0.1, P(U x3 Y = 1) = 0.9

The interpretation of the model is as follows: Insurance plans x1 and x3 are very effective, with 0.9 probability of claim acceptance, while x2 is very ineffective at only 0.1 probability. Insurance company z1 is more reputable than z2 and is more likely to offer plan x1 over x2, while company z2 prefers to offer plan x2 over x1.

Suppose an important factor of consideration not shown in the model is that x1 and x2 are cheaper insurance plans, while x3 is more expensive. A data scientist who is studying this model may choose to abstract the different plans away, categorizing them simply as cheap and expensive plans. Formally, they would study a set of higher-level variables VH = {ZH, XH, YH}, where ZH = Z, YH = Y , and XH has a domain DXH = {x C, x E} corresponding to cheap and expensive plans respectively. There exists an abstraction function τ : DVL DVH such that τ maps x1 and x2 to x C (cheap) and maps x3 to x E (expensive). We will sometimes use the notation XL to describe X to disambiguate from XH, and we will use the notation Z and Y instead of ZH and YH since the variables are the same on both levels.

This immediately brings the AIC into question. If the data scientist is interested in the causal effect of cheap plans on claim acceptance (i.e., P(YXH=x C = 1)), whether x C refers to x1 or x2 is ambiguous. To witness, note that

P(YXL=x1 = 1) = 0.9 (4)

P(YXL=x2 = 1) = 0.1. (5)

Since τ(x1) = τ(x2) = x C, but P(Yx1) = P(Yx2), the AIC is clearly violated, leaving the intervention on x C ambiguous.

Fundamentally, the issue with AIC violations is clear: formal definitions of abstractions expect an equality between low-level and corresponding high-level quantities, but it is not well-defined when one high-level quantity corresponds to multiple differing low-level quantities. In practice, the AIC can be a difficult restriction. Generally, it is assumed to be true whenever abstractions are applied, but it is difficult to verify given that the true SCM and functions are rarely available in real-world settings. The assumption is also likely to be incorrect when applying abstractions naïvely, for example, by performing representation learning or dimensionality reduction without taking the AIC into account. By definition, dimensionality reduction is a lossy transformation of the original data, and the AIC is violated if any of the lost information is relevant for downstream functions.

Figure 2: Comparison between (a) full SCM projections and (b) partial SCM projections. When X is fully projected away, its function is subsumed by its child s function f Y . When X is partially projected, it is split into observed portion Xo and unobserved portion Xu. The role of Xo is preserved, while Xu is subsumed into the function f Y .

Even when the AIC does not hold, it does not necessarily mean that these lossy transformations should not be used. Representation learning and dimensionality reduction are often performed to improve tractability or interpretability at the cost of some lost information. Hence, it would still be desirable to perform causal inferences in the high-level space even under AIC violations. To address the issue of different low-level quantities matching the same high-level quantity, one can reinterpret the high-level quantity as a distribution over its corresponding low-level quantities, where the randomness in the distribution results from the lost information from the abstraction (i.e., a hard intervention on the high-level translates to a soft intervention on the low-level).

2.1. Projected Abstractions

The discussion on relaxing the AIC begins with the concept of SCM projections (Lee & Bareinboim, 2019), which can be viewed as a primitive form of abstraction. An SCM M projected to a subset of variables W V is a functionally identical SCM defined over W, where the functions of V \ W are subsumed by other downstream functions (see App. A Def. 4 for the full definition and App. C Ex. 8 for an example). In the context of constructive abstraction functions, the act of projecting away a variable can be viewed as excluding the variable from all intervariable clusters. This brings the first major insight in addressing AIC violations. In general, when reducing the granularity of a variable, some parts of the variable deemed less important are abstracted away while others are retained. While by definition, SCM projections only allow for entire variables to be included or excluded, one could conceive of SCM projections in which variables are only partially projected away (see App. C Ex. 9

Causal Abstraction Inference under Lossy Representations

for an example). Formally, partial SCM projections can be defined as follows.

Proposition 1 (Partial SCM Projection). Let V be a set of variables and W V be a subset. For each Wi W, let δi : DW o i DW u i DWi be a surjective function mapping new variables W o i and W u i to Wi. W o i and W u i are called the observed and unobserved projections of Wi respectively. Denote δ(Wo, Wu) = W, where Wo = {W o i : Wi W} and Wu = {W u i : Wi W}. For any SCM M = U, V, F, P(U) , there exists an SCM M = U = U Wu, V = Wo, F , P(U ) such that, for all u DU, X W, and x DX,

wo x = M [xo](u, xu, zu), (6)

where δ(wo x, wu x) = Wx(u), δ(xo, xu) = x, Zu = Wu \ Xu, and zu are the corresponding values from wu x. M is called a partial SCM projection of M over Wo.

In words, a partial SCM projection of M over Wo is essentially a smaller version of M defined only on the variables of W V, where each Wi W is only partially represented in the projection. A function δ splits Wi s domain into its observed (W o i ) and unobserved (W u i ) portions. Eq. 6 ensures that any value of Wo obtained from an intervention on the original SCM Mx will match the corresponding output from M , when the observed portion of the intervention xo is applied to M , while the unobserved portions of xu

and wu are passed as unobserved arguments to the functions. A comparison between regular SCM projections and partial SCM projections is shown in Fig. 2. The definition of projected abstractions follow.

Definition 6 (Projected Abstraction). An SCM MH is a projected abstraction of ML if and only if it is a partial SCM projection of a τ-abstraction (Beckers & Halpern, 2019, Def. 3.13) (also Def. 14 in App. A) of ML.

To provide intuition for projected abstractions, consider the following example.

Example 2. Continuing Example 1, given the setup of Eq. 3, suppose XH {x C, x E} is given the function

f H X (z, uz1 X , uz2 X ) =

( x C uz X {x1, x2} x E uz X = x3 , (7)

and define Xu H {x1, x2} as a random variable with distribution

P(Xu H = xi) = P(XL = xi | XL {x1, x2}, z). (8)

Suppose now Y is now given a high-level function

f H Y (x H, xu H, uxi Y ) =

ux1 Y x H = x C, Xu H = x1 ux2 Y x H = x C, Xu H = x2 ux3 Y x H = x E

Observe the intuition from constructing these functions from the perspective of projected abstractions. f H X behaves identically to f L X, except the output remaps the value of XL to the corresponding XH (i.e. f H X = τ(f L X)). However, due to the AIC violation, f H Y is unable to disambiguate between x1 and x2 if XH = x C. The solution is to introduce a new exogenous variable Xu H which represents information in XL that is not captured in XH and disambiguates between x1 and x2. f H Y then uses both XH and Xu H to mimic the behavior of XL. It is clear that XL can be constructed as δ(XH, Xu H), defined as

δ(x H, xu H) =

x1 x H = x C, Xu H = x1 x2 x H = x C, Xu H = x2 x3 x H = x E

which matches Eq. 9. Indeed, MH = UH = UL {Xu H}, VH, FH = {f L Z , f H X , f H Y }, P(UH) is a partial SCM projection (and also projected abstraction) of ML over VH. The graph corresponding to ML is clearly the top graph of Fig. 2(b), but note that through Eq. 8, there is now a dependence from Z to Y , so the graph for MH is instead the bottom graph of Fig. 2(b).

It is easy to see that Eq. 6 holds in this example. For instance, fix UZ = z1, U z1 X = x2, U x2 Y = 1. Clearly, evaluating ML with these values results in Z = z1, X = x2, Y = 1. Note that x2 = δ(x C, x2), and this is the only set of values of XH, Xu H that map to x2. Indeed, on the high level, with UZ = z1, U z1 X = x2, U x2 Y = 1, Xu H = x2, it must also be the case that Z = z1, XH = x C, Y = 1.

Projected abstractions make an important step to working around the AIC as Eq. 6 allows for quantities to be welldefined between low and high-level variables by simply obtaining a partial projection of the original SCM ML over the high-level variables VH. However, unlike full SCM projections, partial SCM projections are not unique in terms of the induced PCH distributions. Prop. 1 guarantees its existence but is underspecified in a couple of ways. First, P(U ) is not fully defined, and it is not clear how Wu

should be sampled (e.g., it is not clear how Eq. 8 is chosen in Ex. 2). Second, Eq. 6 does not specify what behavior M should follow when zu does not match wu x (e.g., How should Y depend on Xu H in Ex. 2 if XH = x E?).

The specific choice of partial SCM projection that best serves as an abstraction can be determined by understanding how low-level interventions relate to high-level interventions. In other words, given a high-level intervention XH x H, it is important to define the corresponding lowlevel soft-intervention σXL, which is a distribution over all possible interventions x L that map to x H. The consequence of the underspecification of partial SCM projections is that there are many possible choices of defining σXL. For a full

Causal Abstraction Inference under Lossy Representations

discussion on how σXL should be decided, see App. B. A useful general form of σXL is defined as follows. Split σXL into individual soft interventions σCi for each intervariable cluster Ci XL. Then define each σCi as

P(σCi = ci) = P(ci | τ(ci) = v H,i, pa VH,i, uc VH,i). (11) In words, a high-level intervention should be equivalent to a distribution over the corresponding low-level interventions that assigns probability to each possible intervention based on their prior probabilities given their parents.2

Example 3. Continuing Example 1, suppose the data scientist is interested in the causal effect of choosing a cheap insurance plan on claim approval. In other words, she would like to study the intervention XH x C, which is ambiguous on the low-level as it could refer to either XL x1 or XL x2. More specifically, according to Eq. 11, XH x C corresponds to a soft intervention σXC on the low level, defined as

( x1 w.p. P(x1 | XL {x1, x2}, z) x2 w.p. P(x2 | XL {x1, x2}, z) (12)

While there are many ways to disambiguate whether x C is referring to x1 or x2, this choice of σXL will assign probabilities based on the prior probabilities of XL being one of x1 or x2. Moreover, the probabilities change depending on the value of z. This makes intuitive sense, since under the intervention XH x C, we expect that if Z = z1, then XL is more likely to be x1 than x2, or vice-versa when Z = z2. From a query perspective, this implies that

P(YXH=x C = 1 | Z = z1) (13)

= P(YσXL(x C,Z) = 1 | Z = z1)

xi {x1,x2} P(xi | XL {x1, x2}, z1)P(Yxi = 1) = 0.74

Likewise, P(YXH=x C = 1 | Z = z2) = 0.26 (14)

While projected abstractions are defined over the entire SCM, the mapping between low and high-level interventions are more clear at the query-level (i.e., individual interventional and counterfactual distributions of interest). Such quantities can be defined as follows.

Definition 7 (Generalized Query). Denote YL, as a set of counterfactual variables over VL. That is,

YL, = YL,1[σXL,1], YL,2[σXL,2], . . . , (15)

2Here, uc VH,i can informally be thought of as the confounded exogenous parents of VH,i. The full definition is somewhat involved, and the subtleties are discussed in App. B.2. Due to space constraints, the main body provides intuition in Markovian settings, where unobserved confounding is not present.

Algorithm 1 Constructing MH from ML.

input ML = UL, VL, FL, P(UL) , constructive abstraction function τ from clusters C and D 1: UH UL, P(UH) P(UL) 2: VH C, DVH D 3: for W VL do 4: W o, W u project(W) {construct δ from Prop. 1} 5: UH UH {W u} 6: end for 7: for Ci C (and corresponding Vi VH) do 8: P(δ(co i , Cu i ) = ci | UL) P(Ci = ci | τ(ci) = vi, pa Vi, uc VH,i) {from Eq. 11} 9: f H i τ(f L V (δ(pao V , pau V ), u V ) : V Ci) 10: end for 11: FH {f H i : Ci C} 12: return MH = UH, VH, FH, P(UH)

where each YL,i[σXL,i] corresponds to the potential outcomes of the variables YL,i under the (possibly soft) intervention σXL,i over XL,i. Each YL,i and XL,i must be unions of clusters from C (i.e. YL,i = S

C C C for some C C) such that τ(YL,i) and τ(XL,i) are well-defined (i.e. τ(YL,i) = V C C τC(C) ). For the high-level counterpart, denote

YH, = τ(YL, ) = YH,1[x H,1], YH,2[x H,2], . . . , (16)

such that YH,i = τ(YL,i), and XH,i = τ(XL,i) for all i. For any value y H, DYH, , denote

DYL, (y H, ) = {y L, : y L, DYL, , τ(y L, ) = y H, }, (17) that is, the set of all values y L, such that τ(y L, ) = y H, .

For any high-level query

τ(Q) = P(YH, = y H, ), (18)

of the form of Eq. 16, its low-level counterpart is

y L, DYL, (y H, ) P(YL, = y L, ), (19)

of the form of Eq. 15.

This query definition connects the distributions of L3(MH) to corresponding distributions of L3(ML). Compared to earlier definitions, Eq. 15 has been generalized to account for soft interventions in addition to hard interventions. Under constructive abstractions functions τ, a notion of Q-τ consistency was established for certain queries Q L3(ML) (App. A Def. 17), which still apply under this generalized definition. In short, given a low level query Q (Eq. 19) and its high-level counterpart τ(Q) (Eq. 18), MH is said to be Q-τ consistent with ML if QML = τ(Q)MH. One can then say that MH is an abstraction of ML specifically for the query Q, even if MH may not be Q -τ consistent with ML for other query choices Q . If MH is Q-τ

Causal Abstraction Inference under Lossy Representations

consistent with ML for all τ(Q) Li(MH), then MH is said to be Li-τ consistent with ML.

With σXL,i defined in Eq. 11, one can then algorithmically construct a projected abstraction consistent in all queries. Given ML and a constructive abstraction function τ (which may not satisfy the AIC), Alg. 1 can be used to construct the high-level abstraction MH. In line 4, each W VL is split into its observed and unobserved counterparts W o and W u. Line 8 assigns each W u a distribution based on Eq. 11. Line 9 builds the high-level function using the low-level function with inputs reconstructed using δ. Finally, the full high-level model MH is assembled and returned in line 10. Under these inputs, Alg. 1 constructs a projected abstraction MH that is Q-τ consistent with ML for all possible high-level L3 queries, as shown by the following result.

Theorem 1. The SCM MH constructed by Alg. 1 is a projected abstraction of ML that is Q-τ consistent with ML for all τ(Q) L3(MH).

As an example, it can be verified that running Alg. 1 on ML in Ex. 1 results in the SCM MH from Ex. 2.

3. Projected Abstraction Inference

Alg. 1 finds an abstraction model MH that is consistent with its low-level counterpart ML for all queries, but it requires the full specification of ML. In practice, ML typically represents the true model of reality and will not be observed. Inferences of L2 and L3 queries must be made through limited available data, usually observational (L1).

The Causal Hierarchy Theorem (Bareinboim et al., 2022, Thm. 1) states that cross-layer inference, or inferring higher layer quantities (e.g., L2, L3) from lower layer data (e.g., L1), is generally impossible without additional assumptions. Many such assumptions take the form of a graphical model, such as a causal diagram (Pearl, 1995), which imply constraints between causal distributions from causal (Bareinboim et al., 2022) and counterfactual Bayesian networks (Correa & Bareinboim, 2024). In the context of abstractions, when τ is a constructive abstraction function that satisfies the AIC, it has been shown that one can avoid assuming the entire causal diagram of the low-level model in favor of a cluster causal diagram (C-DAG) (Anand et al., 2023) w.r.t. the intervariable clusters C. Unfortunately, this graphical model is insufficient for the case when the AIC is violated.

Proposition 2 (C-DAG Insufficiency (Informal)). For a constructive abstraction function τ over intervariable clusters C in which the AIC does not hold, the C-DAG GC implies constraints that may be unsound.

To witness why this is the case, Fig. 2(b) shows the issue clearly. Attempting an abstraction in violation of the AIC is akin to performing a partial SCM projection, which may

Figure 3: Examples of C-DAGs (left) and their corresponding projected C-DAGs (right), with AIC violation variables V H outlined in red.

introduce new dependencies between SCM functions, therefore implying new edges in the graph. Ex. 3 explains this dependence numerically. Since no variables are clustered together in the example, both the original causal diagram G and the C-DAG GC are represented by the top graph in Fig. 3. However, this graph implies that P(Yx H | z) = P(Yx H). Evidently, this is not true since Eq. 13 is not equal to Eq. 14. As hinted by the construction in Alg. 1, the high-level function f H Y requires some additional information from Z to decide between interpreting x C as x1 or x2. This information adds a dependence from Z to the function of f H Y , which requires adding a directed edge from Z to Y .

While the original C-DAG construction is not valid for projected abstraction inferences, one can use a modified version that adds the new required dependencies into the C-DAG.

Definition 8 (Partially Projected C-DAG). Let τ : DVL DVH be a constructive abstraction function w.r.t. intervariable clusters C and intravariable clusters D. Let GC = VH, EC be a C-DAG (with nodes VH and edges EC), of graph G w.r.t. C. Let V H VH be the set of AIC violation variables (App. A Def. 19). Then, construct G C = VH, E C as follows. Start by setting E C EC. Then apply the following rules for all X V H. (1) If Z X Y in EC, then add Z Y into E C. (2) If Z X Y in EC, then add Z Y and X Y into E C. (3) If Z X Y in EC, then add Z Y into E C. Repeat iteratively to accommodate new edges.3 G C is called the partially projected C-DAG of G w.r.t. C and V H.

The steps correspond to the intuition discussed earlier when performing a partial projection, parts of the variables in V H

3Procedure can be applied algorithmically in one pass by applying all rules for each node in V H in topological order.

Causal Abstraction Inference under Lossy Representations

are projected into the exogenous space, resulting in additional dependences that require additional edge connections. Examples of C-DAGs and their corresponding projected C-DAGs are shown in Fig. 3. In the figure, rows (a), (b), and (c) correspond to examples of steps 1, 2, 3 respectively. It turns out that this new definition is precisely what is needed for abstraction inference in the absence of the AIC.

Theorem 2 (Projected C-DAG Sufficiency and Necessity (Informal)). Let ML be an SCM over variables VL, τ : DVL DVH be a constructive abstraction function w.r.t. clusters C and D, and V H be the AIC violation set. The partially projected C-DAG G C w.r.t. C and V H completely describes all constraints over VH.

In other words, the projected C-DAG provides exactly the constraints necessary to solve the task of performing causal inferences across abstractions, even when the AIC is violated. In particular, certain interventional and counterfactual distributions may be inferrable from a combination of the projected C-DAG G C and the available datasets from ML. Determining precisely which queries can be inferred is known as the identification problem, which is defined below in the context of abstract identification.

Definition 9 (Abstract Identification (General)). Let τ : DVH DVL be a constructive abstraction function. Consider projected C-DAG G C, and let Z = {P(VL[zk])}ℓ k=1 be a collection of available interventional (or observational if Zk = ) distributions over VL. Let ΩL and ΩH be the space of SCMs defined over VL and VH, respectively, and let ΩL(G C) and ΩH(G C) be their corresponding subsets that induce G C. A query Q is said to be τ-ID from G C and Z iff for every ML ΩL(G C), MH ΩH(G C) such that MH is Z-τ consistent with ML, MH is also Q-τ consistent with ML.

In words, a query Q is considered τ-ID if, for any pair of models ML and MH such that both are compatible with G C and Z, they also match in Q. In contrast, Q is not τ-ID if there exist ML and MH that are compatible with both G C and Z but disagree on Q (i.e., QML = τ(Q)MH). Abstract identification may seem like a difficult property to check, but it turns out that there is a natural connection with the classical identification problem, as shown below.

Theorem 3 (Dual Abstract ID (General)). Consider a counterfactual query Q over VL, a constructive abstraction function τ w.r.t. clusters C and D, a projected C-DAG G C, and data Z from VL. Q is τ-ID from G C and Z if and only if τ(Q) is ID from G C and τ(Z).

In words, τ-identification across abstractions is equivalent to classic identification on the high-level space.

Example 4. Continuing Ex. 1, note that XH is the only AIC violator in VH, since x1 and x2 both map to x C but

have different effects on Y . Hence, V H = {XH}, and the C-DAG GC and projected C-DAG G C are the two graphs in Fig. 3(a). To answer the query of interest P(YXH=x C = 1), one can apply Thm. 3 to simply identify the quantity w.r.t. P(VH) and G C. In this case, note that the causal effect of XH on Y can be computed via backdoor adjustment on Z, so P(YXH=x C = 1) is equal to X

z P(Y = 1 | XH = x C, Z = z)P(Z = z) (20)

z P(Y = 1 | XL {x1, x2}, z)P(z) (21)

= (0.7)(0.74) + (0.3)(0.26) = 0.596. (22)

Thm. 3 implies that, in practice, τ-ID can be checked by performing any classical ID procedure on the high-level space. This may include algorithmic approaches or other optimization-based approaches.

4. Experiments

We perform two experiments to demonstrate the benefits of projected abstractions. The models in the experiments leverage Neural Causal Models (NCMs) (Xia et al., 2021; 2023), specifically the generative adversarial implementation called GAN-NCMs. Details of the experiment setup can be found in App. D, and code can be found at https://github.com/Causal AILab/ Projected Causal Abstractions.

In the first experiment, we test the necessity of the projected C-DAGs when the AIC does not hold. The high-level query τ(Q) = P(yx | z) is estimated in the graph setting shown in Fig. 3(a), where Z is a digit from 0 to 9, X is a corresponding colored MNIST image, and Y is a label denoting the color prediction of X. τ(X) maps the image to a binary variable representing the shade (light or dark) of X.

The results are shown in Fig. 6. Three different GAN-NCMs are trained: one directly on the low-level data that does not use abstractions (red), an abstracted one constrained by the C-DAG (yellow), and an abstracted one constrained by the projected C-DAG (blue). 95% confidence intervals of the errors are plotted in the figure. Note that the abstractionless model and the projected C-DAG model have decreasing error with more samples, but the regular C-DAG model is unable to learn the correct query. The abstractionless model has higher error than the projected C-DAG model since it operates in a higher-dimensional space.

In the second experiment, we test an interesting consequence of the projected abstraction theory: the soft intervention definition in Eq. 11 can be directly modeled and sampled if attempting to reconstruct the low-level data. We call this

Causal Abstraction Inference under Lossy Representations

Figure 4: Colored MNIST results. Samples from different causal queries (top) are collected from competing approaches (left). The expressions in parentheses are the representation sizes. The left column shows direct image samples from each of the models, while the second, third, and fourth columns show samples generated from an L1, L2, and L3 query, respectively.

Figure 5: (Left) Graph of Colored MNIST experiment. (Right) Correlation shown between color and digit.

approach projected sampling and explain it in more detail in App. B.3. We show this in the causal colored MNIST experiment (Xia & Bareinboim, 2024). In the model, digit D and color C both cause the image I, but they are confounded (e.g., 0 s are red, 5 s are cyan, see Fig. 5). Three different queries are tested (the right three columns of Fig. 4). P(I | D = 0) is an L1 query representing images conditioned on digit = 0, resulting in red 0 s. P(ID=0) is an L2 query representing images with the digit intervened as 0, cutting the confounding and resulting in 0 s of all colors. P(ID=0 |D =5) is an L3 query representing images with digit intervened as 0, conditioned on the digit originally being 5. This results in 0 s with colors of images that were originally 5 s, resulting in cyan 0 s.

Four methods are compared on these queries in Fig. 4, with the ground truth shown on row 5. The non-causal approach (row 1) simply directly models the conditional distribution between digit and image and therefore fails to model anything higher than L1. The representational NCM or RNCM (Xia & Bareinboim, 2024) (row 2) is able to decently reproduce all queries, but it uses a 16-dimensional representation space, which cannot shrink much further due to AIC limitations. When forced to take a binary representation (row 3), the RNCM clearly lacks the representation power to properly generate images. In contrast, using a projected sampling approach (row 4) can reproduce the images even with a representation size as small as a binary digit.

Figure 6: Mean absolute error (MAE) v. number of samples for the MNIST estimation task. Comparisons between an abstractionless approach (red), a C-DAG approach (yellow), and a projected C-DAG approach (blue).

5. Conclusion

This paper introduced projected abstractions (Def. 6), which can be constructed algorithmically (Alg. 1, Thm. 1), to overcome the AIC limitation. When the full model was not available, we leveraged a new graphical model (Def. 8, Thm. 2) that allowed for causal inferences through the abstract-ID problem (Def. 9, Thm. 3). Finally, we demonstrated the ability of projected abstractions to leverage representation learning within difficult causal inference settings through high-dimensional image experiments.

Impact Statement

This paper presents work whose goal is to advance the field of causal inference, a subfield of machine learning. The results in this paper may have implications bringing together strong practical results in representation learning and computer vision research with the explainability and generalizability of causal inference results. The trend is that this will lead to smarter AI, which itself has many consequences out

Causal Abstraction Inference under Lossy Representations

of the scope of this work, but the benefit of understanding causal inference is that it can lead to less bias and more accountability of AI models.

Acknowledgements

This research is supported in part by the NSF, ONR, AFOSR, Do E, Amazon, JP Morgan, and The Alfred P. Sloan Foundation.

Anand, T. V., Ribeiro, A. H., Tian, J., and Bareinboim, E. Causal effect identification in cluster dags. In Proceedings of the 37th AAAI Conference on Artificial Intelligence. AAAI Press, 2023.

Bareinboim, E., Correa, J. D., Ibeling, D., and Icard, T. On pearl s hierarchy and the foundations of causal inference. In Probabilistic and Causal Inference: The Works of Judea Pearl, pp. 507 556. Association for Computing Machinery, New York, NY, USA, 1st edition, 2022.

Beckers, S. and Halpern, J. Y. Abstracting causal models. In Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence and Thirty-First Innovative Applications of Artificial Intelligence Conference and Ninth AAAI Symposium on Educational Advances in Artificial Intelligence, AAAI 19/IAAI 19/EAAI 19. AAAI Press, 2019. ISBN 978-1-57735-809-1. doi: 10.1609/aaai.v33i01.33012678. URL https://doi. org/10.1609/aaai.v33i01.33012678.

Beckers, S., Eberhardt, F., and Halpern, J. Y. Approximate causal abstraction. In Proceedings of the 35th Conference on Uncertainty in Artificial Intelligence. 2019.

Bengio, Y., Courville, A., and Vincent, P. Representation learning: A review and new perspectives. IEEE Trans. Pattern Anal. Mach. Intell., 35(8):1798 1828, aug 2013. ISSN 0162-8828. doi: 10.1109/TPAMI.2013.50. URL https://doi.org/10.1109/TPAMI.2013.50.

Correa, J. and Bareinboim, E. Counterfactual graphical models: Constraints and inference. Technical Report R-115, Causal Artificial Intelligence Lab, Columbia University, August 2024.

Felekis, Y., Zennaro, F. M., Branchini, N., and Damoulas, T. Causal optimal transport of abstractions. In Conference on Causal Learning and Reasoning, CLea R 2024, 2024.

Geiger, A., Potts, C., and Icard, T. Causal abstraction for faithful model interpretation, 2023.

Lee, S. and Bareinboim, E. Structural Causal Bandits with Non-manipulable Variables. In Proceedings of the 33rd AAAI Conference on Artificial Intelligence, 2019.

Massidda, R., Geiger, A., Icard, T., and Bacciu, D. Causal abstraction with soft interventions. In van der Schaar, M., Zhang, C., and Janzing, D. (eds.), Proceedings of the Second Conference on Causal Learning and Reasoning, volume 213 of Proceedings of Machine Learning Research, pp. 68 87. PMLR, 11 14 Apr 2023. URL https://proceedings.mlr.press/ v213/massidda23a.html.

Pearl, J. Causal diagrams for empirical research. Biometrika, 82(4):669 688, 1995.

Pearl, J. Causality: Models, Reasoning, and Inference. Cambridge University Press, New York, NY, USA, 2nd edition, 2000.

Pearl, J. and Mackenzie, D. The Book of Why. Basic Books, New York, 2018.

Rubenstein, P. K., Weichwald, S., Bongers, S., Mooij, J., Janzing, D., Grosse-Wentrup, M., and Schölkopf, B. Causal Consistency of Structural Equation Models. In Proceedings of the Thirty-Third Conference on Uncertainty in Artificial Intelligence, 2017.

Steinberg, D. Copyright. In The Cholesterol Wars. Academic Press, Oxford, 2007. ISBN 9780-12-373979-7. doi: https://doi.org/10.1016/ B978-0-12-373979-7.50003-0. URL https: //www.sciencedirect.com/science/ article/pii/B9780123739797500030.

Truswell, A. Cholesterol and Beyond: The Research on Diet and Coronary Heart Disease 1900-2000. 01 2010. ISBN 978-90-481-8874-1. doi: 10.1007/978-90-481-8875-8.

Xia, K. and Bareinboim, E. Neural causal abstractions. In Proceedings of the 38th AAAI Conference on Artificial Intelligence. AAAI Press, 2024.

Xia, K. and Bareinboim, E. Causal Abstraction Inference under Lossy Representations. Technical Report Technical Report R-124, Columbia University, Department of Computer Science, New York, 2025.

Xia, K., Lee, K.-Z., Bengio, Y., and Bareinboim, E. The causal-neural connection: Expressiveness, learnability, and inference. In Ranzato, M., Beygelzimer, A., Dauphin, Y., Liang, P., and Vaughan, J. W. (eds.), Advances in Neural Information Processing Systems, volume 34, pp. 10823 10836. Curran Associates, Inc., 2021.

Xia, K., Pan, Y., and Bareinboim, E. Neural causal models for counterfactual identification and estimation. In Proceedings of the 11th International Conference on Learning Representations (ICLR-23), 2023.

Causal Abstraction Inference under Lossy Representations

Zennaro, F. M., Drávucz, M., Apachitei, G., Widanage, W. D., and Damoulas, T. Jointly learning consistent causal abstractions over multiple interventional distributions. In van der Schaar, M., Zhang, C., and Janzing, D. (eds.), Conference on Causal Learning and Reasoning, CLea R 2023, 11-14 April 2023, Amazon Development Center, Tübingen, Germany, April 11-14, 2023, volume 213 of Proceedings of Machine Learning Research, pp. 88 121. PMLR, 2023. URL https://proceedings.mlr. press/v213/zennaro23a.html.