In machine learning, data is often represented in vector spaces so that mathematical operations such as combination and scaling are possible.

Field: the numbers to compute with

Let’s first understand what a field is because vector space is defined over a field.

Field is defined as a set of elements where it guarantees 4 basic arithmetic operations: addition, subtraction, multiplication and division (by non-zero elements). The result of performing these operations over the elements of the set must remain within the same set — this property is called a closure.

In addition, a field must satisfy certain rules (called axioms) such as commutativity, associativity, distributivity, and the existence of additive and multiplicative identities and inverses (for all non-zero elements). Intuitively, a field allows us to perform “normal arithmetic”.

The sets of Real numbers (R), rational numbers (Q), complex numbers (C) are fields because they satisfy all the above mentioned properties. Integers (Z), however, are not a field because division does not remain within the set. For example 2÷3 is not an integer as it results in a fraction. As we know fractions are not part of integers set and for that reason integers are not closed under division. Integers form a ring which guarantees addition, subtraction, and multiplication, but not division.

You can imagine a field as a kitchen where you have all the tools needed to cook anything (except dividing by zero).

A field F is the set with two operations (+,⋅) such that (F,+) is an abelian group, (F∖{0},⋅) is an abelian group and ∀a,b,c∈F, a⋅(b+c)=a⋅b+a⋅c

Space: a set with a structure

Next, let’s understand what space is.

Space is defined as a set of elements together with specific structure. Structure is nothing but the set of rules which depicts what operations are allowed on the elements and what properties those operations must hold. Different structures give rise to different types of spaces.

Intuitively, we think of a vector as coordinates, arrows or data representation. However, formally a vector is an element of the vector space. Therefore, without defining a vector space first, a vector cannot be precisely defined.

Vector space: environment for vector

Vector space is a structure defined over the field (usually the real or complex numbers). It consists of a set of elements together with two operations:

• vector addition
• scalar multiplication These operations must be closed within the set and must satisfy specific addition and scalar axioms. A vector space must also contain a zero vector, which represents the absence of any contribution.

You can imagine a vector space as a spice rack in your kitchen, and the structure as the rules that define which spices can be mixed, how they can be combined, and what counts as a valid mixture.

A vector space V over a field F is the set equipped with operations + : V×V→V and ⋅ : F×V→V

For all u,v∈V and α∈F , we have u+v∈V , αv∈V , u+0=u , and α(u+v)=αu+αv

Vector: an element of the vector space

So what is vector? Vector is an element of the vector space. In many common cases (like R^n), they can be represented as ordered lists of field elements. Vectors have many interpretations like

• In physics, they represent direction and magnitude
• In mathematics, they represent coordinates
• In machine learning, they represent data or information

You can imagine a vector as a particular combination of ingredients chosen from the spice rack.

Let v∈R^n be a vector, where v=(v1,v2,v3,…,vn) - This is just a representation and not the definition.

Why vector space matters in ML?

It matters because vector spaces allow data to be

added (combine information) scaled by normalization or weighting, distances and angles can be defined (we will cover this in a future blog post) Many machine learning algorithms rely on these properties. If data does not form a vector space, some ML techniques fail or require special handling.

Vector space is a mathematical environment that makes ML possible.

What is NOT a vector space?

Not everything that looks like a collection of elements forms a vector space. For example -

Categorical values Example: {black, white, blue}

These do not form the vector space because adding and scaling the categorical values are not meaningful and does not make sense. These values require encoding before being used in ML.

Probability distribution Example: [0.2, 0.5, 0.3]

List of probabilities that sums to 1.0. Adding two probability vectors does not preserve the sum to 1 and scaling by a negative scalar will result in negative probability which does not make sense.

Sets

Example: { black, white, blue}

Sets do not support vector addition, scalar multiplication and operations. Operations like union, intersections are not equivalent to vector operations.

So far the vector space that we have discussed only supports the addition and scalar multiplication operations. The length of a vector cannot be measured because there is no notion of norm.

Bringing it all together: the kitchen analogy

Think of the entire setup like a kitchen.

Field is the kitchen with all the basic tools. It guarantees that you can perform the fundamental operations — add, subtract, multiply, and divide (except by zero) — and that everything behaves consistently. Without a field, you don’t even have reliable arithmetic to work with.

Space is the organization and rules of the kitchen. It defines what kind of kitchen this is and what operations are allowed. Different rules give you different kitchens — just as different structures give you different kinds of spaces.

Vector space is a specific kitchen setup where mixing and scaling ingredients is allowed and always makes sense. It guarantees that you can combine ingredients (vector addition), scale recipes up or down (scalar multiplication), and always stay within the same kitchen. There is also a “zero recipe” — doing nothing at all.

Vector is a particular recipe or dish made using the ingredients and rules of that kitchen. It’s a concrete instance — a specific combination of ingredients that follows all the rules of the vector space.

In machine learning, raw data is rarely a ready-made recipe. Instead, we transform data so it can live in a well-defined kitchen — a vector space — where combining, scaling, and learning all make sense.

A note on scope

So far, the vector spaces discussed here only support addition and scalar multiplication. There is no notion of vector length, distance, or angle yet. Those concepts require additional structure (such as norms and inner products), which will be introduced in a future post.

The animation below builds intuition for vector spaces using a simple kitchen analogy, complementing the math discussed in this article.

Thanks for reading. Vector spaces are foundational in machine learning, and intuition goes a long way in understanding them.