LLL reduction

Overview

There are a few issues that one may encounter when attempting to generalize Lagrange's algorithm to higher dimensions. Most importantly, one needs to figure what is the proper way to swap the vectors around and when to terminate, ideally in in polynomial time. A rough sketch of how the algorithm should look like is

def LLL(B):
    d = B.nrows()
    i = 1
    while i<d:
        size_reduce(B)
        if swap_condition(B):
            i += 1
        else:
            B[i],B[i-1] = B[i-1],B[i]
            i = max(i-1,1)
    return B

There are two things we need to figure out, in what order should we reduce the basis elements by and how should we know when to swap. Ideally, we also want the basis to be ordered in a way such that the smallest basis vectors comes first. Intuitively, it would also be better to reduce a vector by the larger vectors first before reducing by the smaller vectors, a very vague analogy to filling up a jar with big stones first before putting in the sand. This leads us to the following size reduction algorithm:

def size_reduce(B):
    d = B.nrows()
    i = 1
    while i<d:
        Bs,M = B.gram_schmidt()
        for j in reversed(range(i)):
            B[i] -= round(M[i,j])*B[j]
            Bs,M = B.gram_schmidt()
    return B

We can further improve this by optimizing the Gram Schmidt computation as this algorithm does not modify $\mathcal B^*$at all. Furthermore$\mu$changes in a very predictable fasion and when vectors are swapped, one can write explicit formulas for how$\mathcal B^*$changes as well.

Next, we need to figure a swapping condition. Naively, we want

$$\left\lVert b_i\right\rVert\leq\left\lVert b_{i+1}\right\rVert$$

for all $i$. However, such a condition does not guarantee termination in polynomial time. As short basis vectors should be almost orthogonal, we may also want to incorperate this notion. Concretely, we want $\left|\mu_{i,j}\right|$to be somewhat small for all pairs of $i,j$, i.e. we may want something like

$$|\mu_{i,j}|\leq c$$

However, since $\mu_{i,j}=\frac{\langle b_i,b_j^*\rangle}{\langle b_j^*,b_j^*\rangle}$, this condition is easily satisfied for a sufficiently long $b_j^*$, which is not what we want. The key idea is to merge these two in some way and was first noticed by Lovász - named the Lovász condition:

$$\delta\left\lVert b_i^*\right\rVert^2\leq\left\lVert b_{i+1}^*+\mu_{i+1,i}b_i^*\right\rVert^2\quad\delta\in\left(\frac14,1\right)$$

It turns out that using this condition, the algorithm above terminates in polynomial time! More specifically, it has a time complexity of $O\left(d^5n\log^3B\right)$where we have$d$basis vectors as a subset of $\mathbb R^n$and $B$is a bound for the largest norm of $b_i$. $\frac14<\delta$ ensures that the lattice vectors are ordered roughly by size and $\delta<1$ensures the algorithm terminates.

Polynomial time proof

This follows the proof provided by the authors of the LLL paper. We first prove that the algorithm terminates by showing it swaps the vectors finitely many times. Let$d$be the number of basis vectors as a subset of $\mathbb R^n$. Let $d_i$be the volume of the lattice generated by $\left\{b_j\right\}_{j=1}^i$at each step of the algorithm. We have $d_i=\prod_{j=1}^i\left\lVert b_j^*\right\rVert$. Now consider the quantity

$$D=\prod_{i=1}^dd_i$$

This quantity only changes whenever some $b_i^*$changes, i.e when swaps happen. Let's consider what happens when we swap $b_i$and $b_{i+1}$. Recall the Gram-Schmidt algorithm:

$$b_i^*=b_i-\sum_{j=1}^{i-1}\mu_{i,j}b_j^*\quad\mu_{i,j}=\frac{\langle b_i,b_j^*\rangle}{\langle b_j^*,b_j^*\rangle}$$

From this, see that when we swap $b_i$and $b_{i+1}$, $b_i^*$is replaced by $b_{i+1}^*+\mu_{i+1,i}b_i^*$. Now using the Lovász condition, we see that we have$\left\lVert b_{i+1}^*+\mu_{i+1,i}b_i^*\right\rVert^2<\delta\left\lVert b_i^*\right\rVert^2$, hence the value of $d_i$must decrease by at least $\delta$, i.e. the new $d_i$is less than $\frac{d_i}\delta$. All other $d_j,j\neq i$must remain the same as the volume remains fixed when we swap basis vectors around. Hence at each swap, $D$decreases by $\delta$. This is why we need $\delta<1$.Now we are left with showing $d_i$is bounded from below then we are done.

Let $\lambda_1(L)$be the length of the shortest (nonzero) vector in the lattice. We can treat $d_i$as the volume of the lattice $L_i$generated by$\left\{b_j\right\}_{j=1}^i$. Let $x_i$be the shortest vector in the lattice in $L_i$. By using Minkowski's lattice point theorem, we have

$$\begin{align*} \lambda_1(L)\leq x_i&\leq\underbrace{\frac2{\sqrt\pi}\Gamma\left(\frac i2+1\right)^{\frac1i}}_{C_i}d_i^\frac1i\\ d_i&\geq\frac{\lambda_1(L)^i}{C_i^i}=d_{i,\min} \end{align*}$$

(Note that the value of $C_i$isn't particularly important, one can use a easier value like $\sqrt i$)

Hence we see that $d_i$, and hence $D$has a (loose) lower bound $D_{\min}=\prod_{i=1}^dd_{i,\min}$, meaning that there are at most $\frac{\log D}{\log D_{\min}\delta}$swaps. Since at each iteration,$k$either increases by$1$when there is no swaps or decreases by at most$1$when there is swaps and $k$ranges from$2$to$d$, the number of time the loop runs must be at most $2\frac{\log D}{\log D_{\min}\delta}+d$, hence the algorithm terminates.

This proof also gives us a handle on the time complexity of the operation. Let$B$is the length of the longest input basis vector. Since we have $d_i\leq B^i$, $D\leq B^{\frac{m^2+m}2}$and the algorithm loops $O\left(d^2\log B\right)$times. The Gram-Schmidt orthogonalization is the most expensive part in the entire process, taking up $O\left(d^2n\right)$arithmetic operations. By using classical algorithm for arithmetic operations, each takes $O\left(n\log B\right)$time. From this, we deduce that the time complexity of the LLL algorithm is $O\left(d^5m\log^2B\right)$, a somewhat reasonable polynomial time algorithm.

Let $b_i$be the output of the LLL algorithm, it turns out that we have the bound

$$\left\lVert b_1\right\rVert\leq\left(\frac4{4\delta-1}\right)^{\frac{d-1}4}\text{vol}(L)^\frac1d$$

which requires $\delta>\frac14$. Such bounds for the shortest vector will be elaborated in more detail in the section on reduced basis.

Exercises

1) Implement the LLL in sage and experimentally verify that $D$does indeed decrease by$\delta$each time.

2) Show that the time complexity analysis is correct, and indeed each loop takes at most $O\left(d^2n\right)$operations.