JSTOR

Introduction

Entropy maximization (EM) is an inference technique that originated in statistical mechanics (Jaynes 1957, 2003). The philosophy behind EM inference is to provide the probability distribution (which we denote P(x)) over system configurations (which we denote x) that corresponds best to the available information. Because a probability distribution with higher entropy encodes less information, the probability distribution that corresponds best to the available information, formulated in terms of constraints, can be found by maximizing the entropy subject to these constraints.

Recently, several studies have explored EM inference in ecological problems (Shipley et al. 2006; Banavar and Maritan 2007; Pueyo et al. 2007; Dewar and Porté 2008; Haegeman and Loreau 2008; Harte et al. 2008). Most attention has been paid to EM inference of species abundance distributions (Banavar and Maritan 2007; Pueyo et al. 2007; Dewar and Porté 2008), but Harte et al. (2008) provide an exception: they apply EM to simultaneously predict, with a minimal number of assumptions (constraints), several macroecological patterns, such as species abundance distributions and species‐level spatial abundance distributions, that together give species‐area relationships. In this article, we zoom in on EM inference of spatial abundance distributions.

We treat the spatial abundance distribution of a given species in a simple, spatially implicit manner. We divide a spatial region in M cells and then describe the arrangement of N individuals over these cells. This description is spatially implicit because we do not take into account the correlations that might exist between neighboring cells. As a consequence, all abundance distributions that we consider are unchanged with random permutation of the cells. Note that this is also the approach taken in different mechanistic models (Coleman 1981; Harte et al. 2005, 2008; Conlisk et al. 2007).

However, this description of spatial abundance distribution by itself does not suffice to apply the EM algorithm. The complete specification of an EM problem requires a number of additional assumptions. These assumptions might appear incidental on first sight, but we show here that they have a major effect on the EM prediction: it turns out that there is not a single EM prediction for the spatial abundance distribution but a myriad of distributions that are obtained under various assumptions, none of which seems to stand out as the most plausible. Interestingly, most of these distributions were obtained as the outcome of process‐based models, including the discrete broken stick model (MacArthur 1960), the random‐placement model (Coleman 1981), and the single‐division model (Conlisk et al. 2007).

To introduce the general framework of the EM approach, we distinguish the formulation of the EM problem from its solution. Whereas the solution of an EM problem can be found with a purely technical recipe, the formulation of the EM problem requires a number of assumptions that can have an important effect on the solution. We first present different EM assumptions for the prediction of the abundance distribution over spatial cells. The corresponding EM problems are solved systematically in the subsequent section.

Formulating the EM Problem for Spatial Distributions

Formulating the EM problem consists of three steps (Haegeman and Loreau 2009): specifying (1) system configurations, (2) the prior distribution over the system configurations, and (3) the constraints on the system configurations. We will discuss these three steps in order.

General Recipe for Solving the EM Problem

Once the set of system configurations, the prior distribution, and the hard and/or soft constraints have been specified, the EM problem can readily be solved. The solution consists of finding the probability distribution P(x) that maximizes the relative entropy $H( P\vert P_{0}) $ subject to the constraints. The relative entropy with respect to the prior distribution P₀(x) is given by For an uninformative prior (i.e., P₀(x) independent of x), maximizing relative entropy $H( P\vert P_{0}) $ is equivalent to maximizing Shannon entropy H(P),

Solution methods for this maximization problem are well known. If all constraints are of the hard type, maximizing (relative) entropy is particularly simple. Configurations that satisfy all constraints have a probability proportional to the prior distribution P₀(x); configurations that do not satisfy all constraints have zero probability. Hence, the EM solution reads if x satisfies all constraints, where Z is a normalization constant given by where the sum is over all vectors x that satisfy the hard constraint. This guarantees that

If there are soft constraints, one can use the technique of Lagrange multipliers. For the soft constraint (6), the EM solution can be written in terms of a corresponding Lagrange multiplier α, if x satisfies all hard constraints, with the normalization constant given by where the sum runs over vectors x that satisfy all hard constraints. The Lagrange multiplier α must be determined by imposing the soft constraint (6). The latter constraint can be rewritten as which often allows one to solve explicitly for α.

It should be noted that the same EM problem can be written in different ways: different combinations of system configurations, prior distribution, and constraints can yield equivalent EM problems. A first example concerns exchangeability of constraints and system configurations: instead of imposing a hard constraint on the set of system configurations, one could equivalently start out with a smaller set of system configurations. A second, less trivial, example concerns the exchangeability of the prior distribution P₀ and the specification of the system configurations, to which we already alluded in the previous section: the EM problem formulated in terms of labeled configurations m with an uninformative prior $P_{0}( \mathbf{m}) \propto 1$ is equivalent to the EM problem formulated in terms of unlabeled configurations n with an informative prior $P_{0}( \mathbf{n}) \propto \mathstrut{\cal M} ( \mathbf{n}) $ . Thus, an alternative definition of the set of system configurations can be mimicked by introducing an appropriate prior distribution.

The latter fact is particularly important for the next section. It implies that the EM problem with labeled configurations and an uninformative prior is not equivalent to the EM problem with unlabeled configurations and an uninformative prior. The ratio between the two EM solutions is given by the multiplicity factor (eq. [3]). To see that this factor drastically modifies the EM distribution, consider the example of $N=6$ individuals distributed over $M=3$ cells. The multiplicity factors for the most and least even distributions, $\mathbf{n}_{1}=( 2,\: 2,\: 2) $ and $\mathbf{n}_{2}=( 6,\: 0,\: 0) $ , respectively, are $\mathstrut{\cal M} ( \mathbf{n}_{1}) =90$ and $\mathstrut{\cal M} ( \mathbf{n}_{2}) =1$ . In other words, whereas the EM problem in terms of unlabeled configurations assigns the same prior probability to n₁ and n₂, the EM problem in terms of labeled configurations assumes that the even configuration n₁ is a priori 90 times as probable as the clustered configuration n₂. Therefore, these two EM problems will lead to very different predictions (the EM prediction for labeled configurations will give relatively more weight to evenly distributed configurations than will the EM prediction for unlabeled configurations). Obviously, these differences become even more pronounced for larger M and N.

EM Solutions for Spatial Distributions

In this section, we study all four EM problems for spatial distributions that result from different combinations of (1) working with either labeled or unlabeled configurations and (2) using either hard or soft constraints to impose a total number of individuals. In all cases we assume an uninformative prior on the system configuration. Here we summarize the results and discuss similarities and differences between different EM solutions; we refer to the appendixes for the formal derivations.

Link between Hard‐ and Soft‐Constraint Solutions

For both labeled and unlabeled configurations, the EM problems with hard and soft constraints on the number of individuals yield related results. The relationship is given in equations (15) and (21): the soft‐constraint solution conditioned on the total number of individuals $N( \mathbf{n}) =K$ equals the hard‐constraint solution for the total number of individuals K. This conditioning property is generally valid for EM solutions.

The difference between hard‐ and soft‐constraint EM solutions resides in their distribution for the total number of individuals N(n). For the hard constraint, the distribution for N(n) is concentrated at the constraint N. For the soft constraint, we know that the distribution for N(n) has its mean at the constraint N, by construction. If the variation around the mean is small, that is, if the soft‐constraint distribution for N(n) is sharply peaked at N, then the EM solutions for hard and soft constraints are practically equivalent.

The EM solutions for labeled and unlabeled configurations behave quite differently in this respect. For labeled configurations, the distribution $P^{( \mathrm{lab}\,,\,\mathrm{soft}\,) }_{M,\, N}( \mathbf{n}) $ is geometric (see eq. [14]), with a large variation around the mean N. This can be verified by computing the coefficient of variation, which is greater than 1 for all N and M. The EM distributions P^{(lab, hard)} and P^{(lab, soft)} are therefore quite different.

For unlabeled configurations, the distribution $P^{( \mathrm{unl}\,,\,\mathrm{soft}\,) }_{M,\, N}( N( \mathbf{n}) ) $ is a negative binomial (see eq. [20]) and sharply peaked at N. Indeed, In most cases of interest, M and N are large (say, $M> 10$ and $N> 10$ ), and the coefficient of variation is much less than 1. Hence, the EM distributions P^{(unl, hard)} and P^{(unl, soft)} can be considered equivalent.

These conclusions are illustrated in figure 1, which compares the one‐cell abundance distributions for the four EM distributions we have analyzed: $P^{( \mathrm{lab}\,,\,\mathrm{hard}\,) }$ , $P^{( \mathrm{lab}\,,\,\mathrm{soft}\,) }$ , $P^{( \mathrm{unl}\,,\,\mathrm{hard}\,) }$ , and $P^{( \mathrm{unl}\,,\,\mathrm{soft}\,) }$ . Note that the one‐cell abundance distributions $P^{( \mathrm{lab}\,,\,\mathrm{soft}\,) }$ and $P^{( \mathrm{unl}\,,\,\mathrm{soft}\,) }$ are mathematically identical; see equations (13) and (19). Formula (17) for P^{(unl, hard)} is not identical to these, but its curve almost always coincides with the two soft‐constraint solutions. However, distribution P^{(lab, hard)} has a completely different one‐cell abundance distribution; see equation (11).

(38 KB)

Figure 1: Comparison between different entropy maximization (EM) predictions for the one‐cell abundance distribution: the EM solution (eq. [11]) for labeled configurations with hard constraint (dash‐dotted line), the EM solution (eq. [13]) for labeled configurations with soft constraint (dotted line), the EM solution (eq. [17]) for unlabeled configurations with hard constraint (dashed line), and the EM solution (eq. [19]) for unlabeled configurations with soft constraint (dotted line; one‐cell abundance distributions P^{(lab, soft)} and P^{(unl, soft)} coincide). Also plotted is the EM solution computed by Harte et al. (2008; solid line). The panels show these distributions for different values of M and N. A, $M=4$ and $N=10$ ; B, $M=16$ and $N=10$ ; C, $M=4$ and $N=100$ ; D, $M=16$ and $N=100$ . The solutions for unlabeled configurations are very close for all values of M and N and are visually indistinguishable for $M=16$ . The solution for labeled configurations with hard constraint is very different.

Open New Window

Scale Dependence of EM Solutions

We have shown that several spatial distributions can be obtained from the EM algorithm with different assumptions in the formulation of the EM problem. In this section, we consider the scale on which the EM algorithm is applied. Indeed, the scale of the EM problem, measured by the number of cells M, requires close scrutiny. We investigate whether the outcome of an EM computation depends on the scale on which the problem was formulated. This is particularly important when combining EM distributions on different scales, for example, to compute species‐area relationships (Harte et al. 2008). Distributions on different scales should be combined only if they are consistent. We demonstrate that this condition is not necessarily satisfied by EM distributions.

For a region of fixed size, the number of cells into which the region is partitioned determines the scale of the problem. The larger the number of cells, the finer the scale. We consider two different scales, M₁ and M₂, and we assume that M₁ is the finer scale and is related to the coarser scale M₂ by an integer factor $\ell =M_{1}/ M_{2}$ . In other words, a cell on scale M₂ consists of ℓ cells on scale M₁.

We introduce a scale transformation from the fine scale M₁ to the coarse scale M₂. For any configuration on scale M₁, there is a corresponding configuration on scale M₂. However, there are several configurations on scale M₂ that correspond to a given configuration on scale M₁. To find the probability of a configuration on scale M₂, we sum the probabilities of all configurations on scale M₁ compatible with the configuration on scale M₂.

To illustrate the scale transformation, consider unlabeled configurations with $N=2$ individuals. The fine scale has $M_{1}=4$ cells; the coarse scale has $M_{2}=2$ cells. The scale transformation regroups the first two cells on scale $M_{1}=4$ in the first cell on scale $M_{2}=2$ , and the last two cells on scale $M_{1}=4$ in the second cell on scale $M_{1}=4$ . This leads to the following correspondence:

Scale consistency can then be defined as follows. Apply the EM algorithm on fine scale M₁, and use the scale transformation from M₁ to M₂ to obtain a spatial distribution on coarse scale M₂. If the latter distribution corresponds to the distribution obtained by applying the EM algorithm directly on scale M₂ then the EM distributions are called “scale consistent.”

In appendix C, we show that for EM problems stated in terms of labeled configurations, the resulting spatial abundance distributions are scale consistent. However, for EM problems stated in terms of unlabeled configurations, the distributions are not scale consistent. As a consequence, a new set of EM distributions can be obtained by, first, applying the EM procedure for unlabeled configurations on scale M₁ and, second, computing averages of the EM solution to obtain a consistent probability distribution for configurations on a coarser scale M₂. We again distinguish hard and soft constraints for the number of individuals.

Link between Solution for Labeled Configurations and Averaged Solution for Unlabeled Configurations

The averaged EM solutions are constructed from the solution of the EM problems with unlabeled configurations. One can verify that EM distributions P^{(unl, hard)} and P^{(unl, soft)} are recovered from P^{(avg, hard)} and P^{(avg, soft)} by setting $\ell =1$ . Here we establish a link between the averaged EM solutions and the solution of the EM problems with labeled configurations.

To do so, we consider the limit $\ell \rightarrow \infty $ . It is shown in appendix C that (see eq. [C7]) However, we also find that (see eq. [C8]), The reason for this asymmetry is that the distributions are the same, depending on the number of individuals (see eq. [C10]), but their distribution for the number of individuals is different. For the first, $\mathrm{\lim}_{\ell \rightarrow \infty }P^{( \mathrm{avg}\,,\,\mathrm{soft}\,) }_{M,\, N,\,\ell }$ , the number of individuals has a Poisson distribution (eq. [C9]); for the second, $P^{( \mathrm{lab}\,,\,\mathrm{soft}\,) }_{M,\, N}( \mathbf{n}) $ , the number of individuals has a geometric distribution (eq. [13]).

We conclude that the family of averaged EM distributions ($P^{( \mathrm{avg}\,,\,\mathrm{hard}\,) }_{M,\, N,\,\ell }( \mathbf{n}) $ and $P^{( \mathrm{avg}\,,\,\mathrm{soft}\,) }_{M,\, N,\,\ell }( \mathbf{n}) $ ), parameterized by the scale factor ℓ, comprises many of the other EM solutions. For $\ell =1$ , we recover the EM solution for labeled configurations with both hard and soft constraints. For $\ell \rightarrow \infty $ , we recover the EM solution for unlabeled configurations with hard constraint but not that with soft constraint. For intermediate values of ℓ, we find interpolating spatial abundance distributions; see figure 2.

(32 KB)

Figure 2: One‐cell abundance distributions for different entropy maximization solutions with hard constraint. We consider a species with $N=300$ individuals and a spatial domain with $M=10$ cells. The distribution P^{(avg, hard)} is plotted for different scale factors: $\ell =1$ , 2, 4, 10, and 40 and $\ell \rightarrow \infty $ . Note that P^{(avg, hard)} with $\ell =1$ corresponds to P^{(lab, hard)} and that P^{(avg, hard)} for $\ell \rightarrow \infty $ corresponds to P^{(lab, hard)}.

Open New Window

Discussion

Entropy maximization (EM) is a mathematical framework that can be used to infer a probability distribution on the set of system configurations, given partial information about the system configuration. Several EM applications in ecology can be imagined and have been studied recently. Here we studied the EM problem for spatial abundance distributions. More precisely, we considered a region divided into a number of cells and derived probability distributions for the arrangement of individuals over the cells without taking into account the spatial location of cells.

We showed that an EM problem formulation requires several assumptions or choices and that the outcome of the EM algorithm depends strongly on these choices. There is not a unique EM prediction for the spatial abundance distribution. On the contrary, we obtained a variety of EM predictions, depending on what might look like details in the EM problem: do we formulate the EM problem in terms of labeled or unlabeled configurations; what prior distribution do we assume; do we impose the number of individuals as a hard or a soft constraint; and on what scale is the EM problem formulated?

The fact that EM allows for a wide range of spatial abundance distributions should not come as a surprise. The EM procedure is an inference technique that depends crucially on the information used in the inference. This information is contained not only in the constraints but also in the way we define system configurations and in the prior distribution over the configurations. Our study indicates that these implicit assumptions should be made explicit in any application of EM procedure, because they can radically change the predicted probability distributions.

An analogous situation exists in physics. Consider a system of N noninteracting particles, each occupying one of M energy levels. This physical system is comparable to our ecological example of distributing N individuals over M spatial cells. Labeled (distinguishable) particles give rise to the classical Maxwell‐Boltzmann (MB) distribution, whereas unlabeled (indistinguishable) particles give rise to the quantum mechanical Bose‐Einstein (BE) distribution. A third distribution exists, the Fermi‐Dirac (FD) distribution, which is also quantum mechanical but has the additional constraint that no more than one particle can be in any one state. In our ecological example, this would mean that one cell cannot contain more than one individual. It is well known that a coarse‐grained description of both BE and FD distributions (i.e., by taking together many quantum mechanical energy levels) tends toward the MB distribution. Similarly, our EM solution for unlabeled species at scale M₁ averaged at scale M₂ becomes the EM solution for labeled individuals when $M_{1}\rightarrow \infty $ (fig. 1).

The equivalence of the EM solutions with hard and soft constraints is a property that is generally satisfied in statistical mechanics (except in phase transitions). The lack of equivalence between P^{(lab, hard)} and P^{(lab, soft)} seems to be pathological and related to a similar problem in statistical mechanics for classical systems. To fix this problem, one must introduce an appropriate prior distribution, the so‐called Boltzmann counting. In appendix D, we present an alternative computation for the EM problem in terms of labeled configurations, using as a prior distribution the analog of Boltzmann counting. This yields distributions, P^{(lab, alt, hard)} and P^{(lab, alt, soft)}, that are practically equivalent. If we accept the replacement of the pathological distribution P^{(lab, soft)} with P^{(lab, alt, soft)}, then all EM distributions derived in this article are part of the family of averaged EM distributions P^{(avg, hard)} and P^{(avg, soft)}.

Harte et al. (2008)’s EM application for spatial abundance distributions is different from ours. Their EM problem is written directly in terms of one‐cell abundance distributions: their system configuration is simply the abundance in a single cell. They implicitly assume a prior that assigns equal probability to each abundance. Their constraints are (1) that the number of individuals in a cell is smaller than the total number of individuals N in the entire region (a hard constraint because it rules out any configuration with abundance greater than N) and (2) that the mean number of individuals in a cell equals $N/M$ (a soft constraint). The solution for the one‐cell abundance distribution is different from but close to our solutions P^{(lab, soft)}, P^{(unl, hard)}, and P^{(unl, soft)}; see figure 1. One may wonder what multicell abundance distribution underlies the one‐cell abundance distribution of Harte et al. (2008). In appendix E, we solve the EM problem for the multicell abundance distribution under their constraints 1 and 2, and we show that the corresponding one‐cell abundance distribution is identical to that of Harte et al. (2008) but not scale consistent. However, this does not mean that Harte et al. (2008)’s one‐cell abundance cannot be embedded in a scale‐consistent multicell abundance distribution. Marginal probability distributions do not, in general, completely determine the joint probability distributions, so it is possible that a scale‐consistent multicell abundance distribution exists that yields the same one‐cell abundance distribution. The unlabeled configurations that we have studied are all invariant under permutations of the cells, thanks to the fact that spatial location is not taken into account. A multicell abundance distribution that is permutation invariant and scale consistent and has Harte et al. (2008)’s one‐cell abundance distribution does not seem to exist. One must incorporate space to find such a distribution. How such a scale‐consistent multicell abundance distribution should result from a properly formulated spatial EM problem remains an open problem.

Note that the discussion of multicell versus one‐cell abundance distribution has an analogy in neutral theory’s predictions for species abundance distributions (Chave et al. 2006), where sampling formulas have been derived that are multispecies abundance distributions (Etienne 2005, 2007), in contrast to one‐species abundance distributions (Volkov et al. 2003). Sampling formulas are required for a detailed comparison between theory and observation, because there may be several sampling formulas that are compatible with a single one‐species abundance distribution (Chave et al. 2006). Here we have also seen that the same one‐cell abundance distribution can correspond to several multicell abundance distributions: compare the joint distributions (18) and (12), which give rise to the identical marginals (19) and (13). Given the central role of data comparison in the EM modeling approach, multicell abundance distributions are therefore required for more powerful EM applications.

It is remarkable that the simple EM applications we have considered yield spatial abundance distributions that have been obtained previously by studying more detailed and often dynamical mechanistic models. We encountered the RP model as the solution of the EM problem with labeled configurations and hard constraint. The DBS model was found as the solution of the EM problem with unlabeled configurations and soft constraint, while scale transforming the latter distribution yields the SD model. The fact that these distributions can be obtained from simple EM applications might indicate a certain robustness. For example, one can expect that a (weakly) perturbed mechanistic model would lead to the same EM distribution.

In fact, even more previously studied spatial abundance distributions can be written as EM solutions. For example, the model based on the hypothesis of equal allocation probabilities (HEAP; Harte et al. 2005) is related to the unlabeled‐configurations solution P^{(unl, hard)}. We have shown that this distribution is not scale consistent, which naturally led us to the averaged distribution P^{(avg, hard)}. Similarly, the HEAP model can be interpreted as an iterative averaging approach, applying a scale factor $\ell =2$ in every iteration step. We remark that this iterative approach is not equivalent to the one‐step scale transformation on which our distribution P^{(avg, hard)} is based.

This suggests that almost any reasonable spatial abundance distribution can be written as the solution of an EM problem. The choice of assumptions in the EM problem formulation is indeed wide. One could consider other ways to define system configurations (different from labeled and unlabeled configurations), one could work with informative prior distributions, or other consistency requirements could be imposed. We believe that the present understanding of the problem of allocating individuals to spatial cells does not allow us to decide which EM problem formulation is most appropriate. This should caution ecologists that applying the EM method does not automatically yield useful results; the accuracy of EM predictions can be determined only by comparison with empirical data.

This illustrates both a strength and a weakness of the EM procedure. Entropy maximization applications are based on a minimal number of assumptions (e.g., labeled or unlabeled individuals, scale consistency), yielding an efficient formalism to generate predictions that can be compared with empirical data. However, the EM problem formulation does not directly establish a link with an underlying mechanistic model. In fact, different process‐based models will typically yield similar ecological patterns. Although translating ecological processes into an EM problem formulation (i.e., a set of configurations, a prior distribution, and constraints) can be a nontrivial problem, the EM procedure might develop into a valuable tool to extract from a detailed mechanistic model a minimal set of assumptions that determine the model predictions.

We considered only a subproblem of the spatial distribution of a species in an ecological community. First, we considered only one species at a time, neglecting the effects of other species and their spatial distribution. Second, we considered only the abundance distribution over cells, without taking into account the spatial location of these cells. Stronger correlations can be expected with abundance distributions for nearby cells than with those for distant cells (see Maddux 2004 and Ostling et al. 2004 for a consistency problem related to the spatial structure). In turn, this might influence the predicted species abundance distribution. Whether the EM approach can be usefully applied for several species at once and/or for spatially structured communities is an interesting set of topics for future research.