Approximate Counting with M Counters: a Probabilistic Analysis

Motivated by a recent paper by Cicho´n and Macyna [1], who introduced m counters (instead of just one) in the approximate counting scheme first analysed by Flajolet [2], we analyse the moments of the sum of the m counters, using techniques that proved to be successful already in several other contexts [11].

A counter C is kept, and each time an item arrives and needs to be counted, a random experiment is performed; if the current value of the counter is i, then with probability 2 −i the counter is increased by 1, otherwise it keeps its value; at the beginning, the counter value is C = 1.After n random increments, the value of the counter is typically close to log 2 n, and the cited papers contain exact and asymptotic values for average and variance.For instance, Flajolet [2] gives the dominant constant part of mean and variance and the periodic part of the mean.
Recently, Cichoń and Macyna [1] used this idea as follows: Instead of one counter, they keep m counters, where m ≥ 1 is an integer.For our subsequent analysis we will assume that m is fixed.When a new element arrives (and needs to be counted), it is randomly (with probability 1 m ) assigned to one The most important motivation of the paper is to compute the asymptotic distribution and the moments of J(m, n).The asymptotic distribution is related to the extreme-value Gumbel distribution function (DF): exp(− exp(−x)).The moments are usually given by a constant part and a small fluctuating part.There we use Laplace and Mellin transforms and singularity analysis.
Our aim is to derive an (almost) purely mechanical computation of constant and fluctuating components, with the help of computer algebra systems (we use Maple here).As an example, we provide the first four moments, (even the third moment is very rarely computed in the literature) but the treatment is completely automatic (with some human guidance of course).The fourth moment is particularly interesting: it presents a wide variety of combinatorial and mathematical constants as well as several types of Fourier series (including products of them).
A last but not least motivation is to simplify the analytic treatment by using only easy complex analysis: only simple poles are needed, and we do not use alternating series (so we do not need Rice's formula).A small number of analytic functions are the only tools we need.This should be compared with the complicated techniques sometimes used in previous papers.
We have uniform integrability for the moments of our RV's.To show that the limiting moments are equivalent to the moments of the limiting distributions, we need a suitable rate of convergence.This is related to a uniform integrability condition (see Loève [8,Section 11.4]).
The total error term related to our asymptotics of moments is detailed in [11]; it is given by O(n −C ), where C is some constant.
Another technical point of interest are the periodic oscillations that always occur in approximate counting and related questions: When one goes for higher moments, there are many extra terms coming in, making the Fourier coefficients very complicated.In particular, there are high powers of some (simple) periodic functions.We present a technique to bring the Fourier coefficients of these into some standard form, using residue calculus.
To summarize, we had several motivations to write this paper: • show the power of the method we introduced in [11] • present, with great details, the analysis of classical Approximate counting • give new simplifications of product of some Fourier series

• analyze, with precision, Approximate counting with m counters
The paper is organized as follows: Definitions, notations and known properties are recalled in Section 2. Classical approximate counter (1 counter) is analyzed in Section 3. The case of m counters is considered in Section 4. The asymptotic moments of J(n) are given in Section 5. Section 6 concludes the paper.

Definitions, notations and known properties
Let us first give the notations we will need throughout the paper.They will be used in Section 3, but we prefer to summarize them here instead of introducing them one by one.
The following facts will be frequently used: For the integer-valued RV J (from now on, we drop i and n from J i (n) in order to ease the notation; J is now related to one counter, with n items), we set p(j) := Pr(J = j), P (j) := Pr(J ≤ j).
Setting η = j − log n, we will first compute f and F such that p(j) ∼ f (η), P (j) ∼ F (η), n → ∞, and, of course, Asymptotically, the distribution will be a periodic function of log n in the following sense: fix n; Set log n = log n + 1 (hence n = 2n, the base could of course be changed).Then The asymptotic distribution of J(n ) is the same as the one of J(n), only shifted by 1.All n with the same {log n} lead to the same points of F (η).The distribution P (j) does not converge in the weak sense, it does however converge along subsequences n m for which the fractional part of log n m is constant.This type of convergence is not uncommon in the Analysis of Algorithms.Many examples are given in [11].
Next, we must check that by computing a suitable rate of convergence.This is related to a uniform integrability condition (see Loève [8,Section 11.4].) 3. Classical approximate counting (one counter).Analysis of J(n) In this section, we provide the asymptotic distribution and the first four asymptotic moments of the RV J(n) based on n items.From Flajolet [2, Proposition 1], we have Letting η = j − log n, it is proved in [2] that (Compare also [6,13,14].)It has been pointed out in [9] that it has some similarities with the Digital Search Tree distribution.Actually, as noticed by S. Janson (private communication), the distribution for approximate counting is the same as for unsuccessful search in Digital Search Trees (not only asymptotically).
The rate of the convergence problem is completely solved in Flajolet [2].Also, we obtain by summing We note that the algorithm can be generalized by changing the base.The analysis is quite similar, and we won't provide details here.
The first three asymptotic moments are given in our unpublished report [10].We take the opportunity to present them here, with some complements.In particular we analyze the product of Fourier series, which leads to convolutions of the coefficients.In order to show the power of the methods we use, we also give the fourth moment.Using the techniques we described in our published paper [11], we proceed as follows.

Some preliminary identities
Some preliminary identities are necessary.They appear in the asymptotic moments, but we prefer to avoid overloading Sections 3.3 and 3.4.
Using a classical Euler identity, we will derive several summation formulae.This identity is (for a simple proof, see Knuth [7,).It holds for general q, once the definition of Q(i) it adapted by replacing 1  2 by q.Set It is not hard to see that, with z = −1, q = 1/2, we get the following expansions with the abbreviations

Now we want to compute sums of type
We obtain and setting z = −1, q = 1/2, we derive U 1 = Q.Similarly, set T 1 := zΠ , compute T 1 , etc.With the same procedure, we obtain: Also this is Equation (21) in Flajolet [2].More generally, setting

Slow increase property
It is necessary that the functions we need decrease exponentially in the direction i∞.Indeed, (see ( [11] for details), some integration in the complex plane (along a rectangle) must be convergent.
It will appear that all functions we use here are analytic (in some domain), depending on classical functions such as Γ, ζ, ψ(k, s) (the (k + 1)-gamma function).
We know that Γ(s) decreases exponentially in the direction i∞: Also, we have a "slow increase property" for all functions we encounter: We will also use the function H 2 (−Ls), with To analyze this function, we use the "sum splitting technique" as described in Knuth [7, p. 131], and used in Flajolet [2]: let σ > −1 and ρ(x) be an increasing function.

The asymptotic moments
The detailed proofs of relations ( 8) to (17) are given in [11].Let an (integer-valued) RV K be such that Pr(K − log n ≤ η) ∼ F (η), where F (η) is the DF of a continuous RV Z with mean m 1 , second moment m 2 , variance σ 2 and centered moments µ k .Assume that F (η) is either an extreme-value DF or a convergent series of such and that (1) is satisfied.Let say, with Also We have here, for J, But we need a larger range for α.But, by (7), we can continue φ(α) analytically for all α: all singularities of Γ(−α) are cancelled by the successive roots of H 2 (α).Moreover, this implies Now, we have This gives This leads to (we give only the first two expressions for m i ) Let w, κ's (with or without subscripts) denote periodic functions of log n, with period 1 and 0 mean (for w) or non-zero small mean (for κ) and small (of order 10 −4 ) amplitude.Actually, these functions depend on the fractional part of log n: {log n}.
The moments of J(n) − log n are asymptotically given by mi + w i .The constant part mi comes from the residue of some function at 0 (see [11]) and w i comes from the residues of the same function at some complex poles.The generating function of mi is given by This leads to More generally, the centered moments of J(n) are asymptotically given by µ i = μi + κ i , with the asymptotic dominant constant centered moment generating function given by The neglected part is of order 1/n β with 0 < β < 1.We derive, with (4), the following result about these centered moments.
Theorem 3.1.The asymptotic dominant constant parts of the centered moments of J(n) are given by Note that and With ( 5), we check that we have no singularity of φ (k) , k > 0, at α = 0.The fundamental strip for ( 15) is (s) ∈ −1, 0 .We first obtain Γ(χ l )e −2lπi log n ; m1 , μ2 and w 1 are identical to the expressions given in Flajolet [2].
To compute the periodic components κ i (with non-zero small mean) to be added to the centered moments μi , we first set We replace mk by mk + w k , leading to But it is easy to check that l =0 φ(−Lχ l )e −2lπi log n = 0, so we obtain Finally, we compute leading to the (exponential) generating function of κ k .By expansion and taking differences, we have a result about the oscillating parts.
Theorem 3.2.The asymptotic oscillating parts of the centered moments of J(n) are given by All algebraic manipulations of this paper are mechanically performed by Maple. 1ote that the non-zero κ i mean must be added to the dominant constant parts of the centered moments.This will be considered in the next section.

The corrections
Products of Fourier series may have a constant term, even if the factors do not.This term must be included in the dominant constant part of our moments.This is the object of the present subsection.
We denote by [f ] k the coefficient of e −2kπi log n in the Fourier expansion of f .
In [11], we have proved the following relations .
The coefficient c 1 [k] of e −2kπi log n in the Fourier expansion of w 2 1 is given by This has been checked numerically and gives the tiny value −9.428177 × 10 −25 .
The coefficient c 2 [k] of e −2kπi log n in the Fourier expansion of w 3 1 is given by For a complete description of the Fourier coefficients of the oscillations occurring in the third and fourth moment, we need the following expressions (note that Maple splits the higher derivatives of the Gamma function; if one could rework that, one could reduce the number of necessary expressions): We will show now how to "compute" some Fourier coefficients we need."Computing" is perhaps a very ambitious word, it might be better replaced by "rewriting".We have and higher powers have convolutions as coefficients: and none of the k 1 , . . ., k 4 is allowed to be zero.The only thing that we are able to achieve is to have only one Gamma-term in the (multiple) sum, where a typical term might look like Such a representation is not a priori better than the straight-forward convolution, but we will sketch now how to achieve them.One (small) advantage is that the zeroth term can be explicitly determined, and extracted, and what is left is then oscillating around zero.
As an example, we consider Let us discuss W 2 .It is clear that the convolution of W with itself contains already several terms.For instance, We would like to demonstrate how to rewrite the k-sums in these expressions.The survey paper [15] has many similar examples.The approach we found most versatile is via residue calculus.One writes a suitable function, computes all the residues in the complex plane, and the sum of them is zero.There are of course some technical subtleties, like showing that integral tends to zero for larger and larger radii, and also there are usually some series that do not converge absolutely.The suitable limit of them is the Abel limit, i.e., consider a power series in x, and let x tend to a point at the boundary of convergence.
Here, we want to concentrate on the computational part only.The function that is suitable for the first part is A first contribution comes from the poles at z = χ k : The next contribution stems from the poles at z = j, j + 1, . . .: Now we look at the poles at z = −l, −l − 1, . . .
This series does not converge absolutely.It is best to pull out the "bad" part, which leads to two contributions: As announced, we must interpret S 4 as a limit: The proper function is The poles at z = χ k , k = 0, lead to The pole at z = 0 leads to The poles at z = l, l + 1, . . .lead to The poles at z = −1, −2, . . .lead to Finally, let us consider The function of interest is The poles at z = χ k , k = 0, lead to The pole at z = 0 leads to The poles at z = 1, 2, . . .lead to .
totic constant and oscillating parts of the centered moments of J(n) are given by Note that μ2,c fits with the result given in [16].

m counters: Asymptotic independence of the m counters
In this section, we analyze the asymptotic properties of the RV J(m, n).
We will prove that, asymptotically, the counters are independent with n/m items each.We must analyze the random variable J(m, n) = m 1 J i (n) where J i (n) has the distribution p η (j) with η is now given by ν i : the number of items arriving in counter i.The quantity ν i is bin[ñ, ñ(1 − 1/m))] with ñ := n/m.Actually, {ν 1 , . . ., ν m } is given by a multinomial distribution.We know that we can construct a "box" such that, by large deviation analysis, the probability that {ν 1 , . . ., ν m } is outside this box is bounded by exp(−C ñ2θ−1 ).We will analyze The rate of convergence is analyzed as follows.

4.1.
Let us first assume that ν i is exactly given by its mean ñ.As mentioned in the previous section, the rate of convergence problem is solved in Flajolet [2].

4.2.
Now we assume that we are inside the box (18).We drop the ˜sign from ñ for convenience.Let 0 < ε < 1.We must bound • For j < β log n, we have • For β log n ≤ j < 2 log n, we have We use again the "sum splitting technique."Set r = √ 2 log n.
1. Truncating the sum in (2) to k ≥ r leads to an error E 1 : We proceed now as in the second range 1. 2.
Now we come to S 2 .We get (not with the same ε, of course).

4.3.
Now we consider the case ν > n + n θ .We will show that is small.We notice that 1 < ν/n < m.But, by the rate of convergence proved in [2], j p ν (j)j k is asymptotically bounded by O((log ν) k ) = O((log(nm)) k ) and S 3 is asymptotically small.

4.4.
The last case to consider is 0 < ν < n − n θ .We have here 0 < ν/n < 1.The analysis proceeds like in the previous subsection.We therefore omit the details.
In conclusion, as S 2 and S 3 are asymptotically small, we can assume that ν i can be deterministically chosen as ñ for all i, and that the counters are asymptotically independent.

m counters: Asymptotic moments
If J i are iid RV, with asymptotic centered moments µ k = μk + κ k , then J = m i=1 J i has asymptotic distribution given by the convolution f (η) (m) The corrected moments must now be computed.The interesting case is the fourth moment, since here the dependency on m is more involved: we obtain our last theorem.[κ 2 2 ] l e −2lπi log n + 3(m − 1)2μ 2 κ 2,c .

Conclusion
If we compare the approach in this paper with other ones that appeared previously, then we can notice the following.Traditionally, one would stay with exact enumerations as long as possible, and only at a late stage move to asymptotics.Doing this, one would, in terms of asymptotics, carry many unimportant contributions around, which makes the computations quite heavy, especially when it comes to higher moments.Here, however, approximations are carried out as early as possible, and this allows for streamlined (and often automatic) computations of asymptotic distributions and higher moments.