Gram-Schmidt orthogonalization is an algorithm that takes in a basis {biβ}i=1nβ as an input and returns a basis {biββ}i=1nβwhere all vectors are orthogonal, i.e. at right angles. This new basis is defined as
where ΞΌi,jβis the Gram-Schmidt coefficients.
One can immediately check that this new basis is orthogonal, meaning
Let be the matrix where the th row is given by andbe the matrix where the th row is given by , then the Gram-Schmidt orthogonalization gives us where and is the Gram-Schmidt coefficient. As an example, consider the basis of a subspace of :
Instead of doing the Gram-Schmidt orthogonalization by hand, we can get sage to do it for us:
This outputs two matrices, and :
One can quickly verify that and that the rows of are orthogonal to each other.
A useful result is that
Intuitively, this tells us that the more orthogonal a set of basis for a lattice is, the shorter it is as the volume must be constant.
1) Show that the basis is orthogonal.
2) Verify that the output of sage is indeed correct.
3) Show that and is a diagonal matrix whose entries are . Conclude that .
4*) Given the Iwasawa decomposition where is a lower diagonal matrix with on its diagonal, is a diagonal matrix and an orthogonal matrix, meaning , show that and . Furthermore, prove that such a decomposition is unique.
B = Matrix([
[-1, -2, 3, 1],
[-6, -4, 5, 1],
[5, 5, 1, -3]])
B.gram_schmidt()(
[-1 -2 3 1] [ 1 0 0]
[-4 0 -1 -1] [ 2 1 0]
[ 0 3 3 -3], [-1 -1 1]
)Lagrange's algorithm, often incorrectly called Gaussian reduction, is the 2D analouge to the Euclidean algorithm and is used for lattice reduction. Intuitively, lattice reduction is the idea of finding a new basis that consists of shorter vectors. Before going into Lagrange's algorithm, we first recap the Euclidean algorithm:
The algorithm primarily consists of two steps, a reduction step where the size of mis brought down by a multiple of nand a swapping step that ensures mis always the largest number. We can adapt this idea for lattices:
Here is actually the Gram-Schmidt coefficient and it turns out that this algorithm will always find the shortest possible basis! Using the basis
the Lagrange reduction looks like
and here we see it clearly gives the shortest vectors.
Let be a lattice. The basis is defined to be the shortest for any other basis , we have and . Note that this generally cannot be generalized to other dimensions, however in dimension 2, this is possible and is given by Lagrange's algorithm. The proof is a somewhat messy sequence of inequalities that eventually lead to the conclusion we want.
Let be the output of the Lagrange reduction for some lattice . To prove that Lagrange reduction gives the shortest basis, we first show that is the shortest vector in .
We know that from the algorithm directly. Let be any element in . We first show that :
Since , this quantity is only when and is a positive integer for all other cases, hence and is a shortest vector of . Note that we can have multiple vectors with the same norm as , for instance . So this is not a unique shortest vector.
Suppose there exists some basis for such that . We show that . Let .
If , then as must form a basis. This means that and by the inequality above, we must have or . The first case tells us that . By squaring the second case, we get
but since is the shortest vector, .
Otherwise, we have and , so
Hence proving Lagrange's algorithm indeed gives us the shortest basis vectors.
1) Show that the output of Lagrange's algorithm generate the same lattice as the input.
2) Find a case where . Notice that the vectors here is the equality case for the bound given in Exercise 4 of the introduction, this actually tells us that the optimal lattice circle packing in 2D is given by this precise lattice! It turns out that this is actually the optimal circle packing in 2D but the proof is significantly more involved. (See for the details)
3*) Let , show that
and show that for all steps in the algorithm except the first and last, hence decreases by at least at each loop and the algorithm runs in polynomial time.
There are a few issues that one may encounter when attempting to generalize Lagrange's algorithm to higher dimensions. Most importantly, one needs to figure what is the proper way to swap the vectors around and when to terminate, ideally in in polynomial time. A rough sketch of how the algorithm should look like is
There are two things we need to figure out, in what order should we reduce the basis elements by and how should we know when to swap. Ideally, we also want the basis to be ordered in a way such that the smallest basis vectors comes first. Intuitively, it would also be better to reduce a vector by the larger vectors first before reducing by the smaller vectors, a very vague analogy to filling up a jar with big stones first before putting in the sand. This leads us to the following size reduction algorithm:
def euclid(m,n):
while n!=0:
q = round(m/n)
m -= q*n
if abs(n) > abs(m):
m, n = n, m
return abs(m)def lagrange(b1,b2):
mu = 1
while mu != 0:
mu = round((b1*b2) / (b1*b1))
b2 -= mu*b1
if b1*b1 > b2*b2:
b1, b2 = b2, b1
return b1, b2We can further improve this by optimizing the Gram Schmidt computation as this algorithm does not modify Bβat all. FurthermoreΞΌchanges in a very predictable fasion and when vectors are swapped, one can write explicit formulas for howBβchanges as well.
Next, we need to figure a swapping condition. Naively, we want
for all i. However, such a condition does not guarantee termination in polynomial time. As short basis vectors should be almost orthogonal, we may also want to incorperate this notion. Concretely, we want β£ΞΌi,jββ£to be somewhat small for all pairs of i,j, i.e. we may want something like
However, since ΞΌi,jβ=β¨bjββ,bjβββ©β¨biβ,bjβββ©β, this condition is easily satisfied for a sufficiently long bjββ, which is not what we want. The key idea is to merge these two in some way and was first noticed by LovΓ‘sz - named the LovΓ‘sz condition:
It turns out that using this condition, the algorithm above terminates in polynomial time! More specifically, it has a time complexity of O(d5nlog3B)where we havedbasis vectors as a subset of Rnand Bis a bound for the largest norm of biβ. 41β<Ξ΄ ensures that the lattice vectors are ordered roughly by size and Ξ΄<1ensures the algorithm terminates.
This follows the proof provided by the authors of the LLL paper. We first prove that the algorithm terminates by showing it swaps the vectors finitely many times. Letdbe the number of basis vectors as a subset of Rn. Let diβbe the volume of the lattice generated by {bjβ}j=1iβat each step of the algorithm. We have diβ=βj=1iββbjβββ. Now consider the quantity
This quantity only changes whenever some biββchanges, i.e when swaps happen. Let's consider what happens when we swap biβand bi+1β. Recall the Gram-Schmidt algorithm:
From this, see that when we swap biβand bi+1β, biββis replaced by bi+1ββ+ΞΌi+1,iβbiββ. Now using the LovΓ‘sz condition, we see that we haveβbi+1ββ+ΞΌi+1,iβbiβββ2<Ξ΄β₯biβββ₯2, hence the value of diβmust decrease by at least Ξ΄, i.e. the new diβis less than Ξ΄diββ. All other djβ,jξ =imust remain the same as the volume remains fixed when we swap basis vectors around. Hence at each swap, Ddecreases by Ξ΄. This is why we need Ξ΄<1.Now we are left with showing diβis bounded from below then we are done.
Let Ξ»1β(L)be the length of the shortest (nonzero) vector in the lattice. We can treat diβas the volume of the lattice Liβgenerated by{bjβ}j=1iβ. Let xiβbe the shortest vector in the lattice in Liβ. By using Minkowski's lattice point theorem, we have
(Note that the value of Ciβisn't particularly important, one can use a easier value like iβ)
Hence we see that diβ, and hence Dhas a (loose) lower bound Dminβ=βi=1dβdi,minβ, meaning that there are at most logDminβΞ΄logDβswaps. Since at each iteration,keither increases by1when there is no swaps or decreases by at most1when there is swaps and kranges from2tod, the number of time the loop runs must be at most 2logDminβΞ΄logDβ+d, hence the algorithm terminates.
This proof also gives us a handle on the time complexity of the operation. LetBis the length of the longest input basis vector. Since we have diββ€Bi, Dβ€B2m2+mβand the algorithm loops O(d2logB)times. The Gram-Schmidt orthogonalization is the most expensive part in the entire process, taking up O(d2n)arithmetic operations. By using classical algorithm for arithmetic operations, each takes O(nlogB)time. From this, we deduce that the time complexity of the LLL algorithm is O(d5mlog2B), a somewhat reasonable polynomial time algorithm.
Let biβbe the output of the LLL algorithm, it turns out that we have the bound
which requires Ξ΄>41β. Such bounds for the shortest vector will be elaborated in more detail in the section on reduced basis.
1) Implement the LLL in sage and experimentally verify that Ddoes indeed decrease byΞ΄each time.
2) Show that the time complexity analysis is correct, and indeed each loop takes at most O(d2n)operations.
def LLL(B):
d = B.nrows()
i = 1
while i<d:
size_reduce(B)
if swap_condition(B):
i += 1
else:
B[i],B[i-1] = B[i-1],B[i]
i = max(i-1,1)
return B
def size_reduce(B):
d = B.nrows()
i = 1
while i<d:
Bs,M = B.gram_schmidt()
for j in reversed(range(i)):
B[i] -= round(M[i,j])*B[j]
Bs,M = B.gram_schmidt()
return BIn this section, we hope to bring some intuitive understanding to the LLL algorithm and how it works. The LLL algorithm is a lattice reduction algorithm, meaning it takes in a basis for some lattice and hopefully returns another basis for the same lattice with shorter basis vectors. Before introducing LLL reduction, we'll introduce 2 key algorithms that LLL is built from, Gram-Schmidt orthogonalization and Gaussian Reduction. We give a brief overview on why these are used to build LLL.
As the volume of a lattice is fixed, and is given by the determinant of the basis vectors, whenever our basis vectors gets shorter, they must, in some intuitive sense, become more orthogonal to each other in order for the determinant to remain the same. Hence, Gram-Schmidt orthogonalization is used as an approximation to the shortest basis vector. However, the vectors that we get are in general not in the lattice, hence we only use this as a rough idea of what the shortest vectors would be like.
Lagrange's algorithm can be thought as the GCD algorithm for 2 numbers generalized to lattices. This iteratively reduces the length of each vector by subtracting some amount of one from another until we can't do it anymore. Such an algorithm actually gives the shortest possible vectors in 2 dimensions! Unfortunately, this algorithm may not terminate for higher dimensions, even in 3 dimensions. Hence, it needs to be modified a bit to allow the algorithm to halt.
