This is Words and Buttons Online — a collection of interactive #tutorials, #demos, and#quizzes about #mathematics, #algorithms and #programming.

The problem with iterative methods for solving linear systems is that they are usually taught straight after the non-iterative methods. So you spend a few weeks playing with something called Gaussian elimination and the whole point of it is that you can perform a few row-wise operations that transform a system into a more likeable equivalent system, and in the end, if you’ve done everything right, after some predefined number of operations you get to transform your system into a diagonal matrix and a column of answers. Neat!

So you learn that you’re allowed to add rows, multiply them by any non-null scalar value, and swap them. None of these operations affects the solution of the system.

Sometimes, there is no solution at all, but that’s okay; that usually indicates that the system was not defined properly. It’s not your fault.

But then you start learning about the iterative methods, and you start with something called Gauss-Seidel, so you’d expect it has something to do with the Gaussian method from before, right? And it doesn’t. It’s so different that it hurts. First of all, the computation is now iterative. You do the same operation over and over, hoping that you’re getting closer to the answer. And sometimes you do. But sometimes you don’t. You can’t find the solution, but this time it may actually exist. It is now your fault that you can’t find it.

What’s worse, this time, swapping your rows changes the outcome of your computation drastically. Unsolvable systems become solvable, and solvable ones now cause the algorithm to crash and burn. Isn’t it annoying?

Here’s the interactive visualisation. The red and blue lines are two equations, and the black arrows are the iterations. The thick margins near the lines are the exit condition for the algorithm. By default, the widget is set up so the two equations form a system that is easily solvable. But if you click «Swap lines», the very same system will cause the algorithm to diverge. Here, try it!

none yet

The key to understanding the Gauss-Seidel method is in its geometry. When we compute an xi coordinate from the i-th equation, this is exactly like putting an arrow from the current solution to the line representing the i-th equation along the coordinate’s axis.

In our visualization, the red line always defines the x coordinate, and the blue one — the y. And sometimes this makes the algorithm spiral towards the solution. But when you swap the lines, the very same algorithm, starting from the very same point, starts to diverge because the spiral starts to go another way. Isn’t it obvious when you see it?

There are also configurations in which the system, and now it is the property of the system, causes the algorithm to neither converge nor diverge while being obviously solveable. Can you find one by moving the lines-equations around?

I’m sure you can! You can also make a system that doesn’t have a solution or has an infinite number of solutions instead. If you think you can’t, please visit my Programmer’s guide to linear equations tutorial.