Math for AI 101

Acknowledgement: This post is based mainly on Dr. Quang-Vinh Dinh’s presentation and slides and partly on on TA, STA, and peers’ responses from the course AIO2025 offered by AI VIETNAM.

Relevance: The audience of this lecture were those who have read Python for AI 106, had some coding experience, and known a bit of AI knowledge. Absolute beginners are encouraged to read the material and repeatedly practice coding without AI assistance to strengthen their coding and code-reading skills. Note that only necessary Python knowledge is introduced here. Also, not every single detail from Linear Algebra is introduced in this Math for AI series. Instead, we’ll learn pick up more material along the way.

In today’s post, we’ll explore Numpy. We’ll also discuss some applications in AI including Image Brightness with OpenCV.

NumPy

To create and use an array in NumPy, we must include a numpy package with the following syntax

import numpy as np

NumPy Array

Numpy is a standard Python library commonly used for numerical computing. The core object in NumPy is ndarray which is a multi-dimensional array data structure. NumPy can interact with various libraries and frameworks. We’ll refer to a NumPy array as an array from now on unless stated otherwise.

Lists can work with multi-dimensional data structures. What else do we need for?

A list is ideal for storing one-dimensional data, but it is rather confusing when representing and handling multi-dimensional data due to its flexibility and slow performance. NumPy was created to address these issues. With NumPy, programmers need not write custom functions for common operations like squaring elements—built-in computational methods handle them efficiently. Basically, NumPy is a streamlined library.

• Execution view: The operation is automatically applied to every selected element in the array.
• Implementation view: We write the code as if it is performed on a single element.

The fact that Python lists can be inconsistent in how they are nested makes it difficult to define a clear sense of dimension or shape like we can with arrays. For an array, we can extract such information using built-in methods shape or ndim.

# 1D array
arr_1D = np.array([1, 2])

arr_1D
>>> [1 2]

arr_1D.shape
>>> (2, )

arr_1D.ndim
>>> 1

# 2D array
arr_2D = np.reshape((2,1))

arr_2D
>>> [[1]
     [2]]

arr_2D.shape
>>> (2, 1)

arr_2D.ndim
>>> 2

It is imperative to note that a column matrix is different from a vector in NumPy even though they, in this example, basically are \(\begin{bmatrix} 1 \\ 2 \end{bmatrix}\), which is totally different from in Mathematics.

a = np.array([[1, 2, 7, 3], [3, 4, 8, 5], [5, 6, 9, 4]])

# 2D nested array 
a.ndim
>>> 2

# 3-by-4 matrix
a.shape
>>> (3, 4)

Note that the term dimension and axis are interchangeable with arrays. For example, a one-dimensional array (1D) has one axis, a two-dimensional array (2D) has two axes, a three-dimensional array (3D) has three axes, and so on. The number of elements along each axis determines the shape of an array. Each axis has its own default direction, for instance,

• Axis 0: from top to bottom (rows)
• Axis 1: from left to right (columns)
• Axis 2: from front to back (depth).

Some Common Methods

When learning a concept in coding, it’s imperative we know how to create it. There are a few ways to make an array either from scratch or from a pre-exiting array.

Create a NumPy Array (from scratch)
We can

• convert a list into an array (demonstrated above)
• make an array of a desired shape with zeros/ones via np.zeros() or np.ones()
• produce an equally-spaced array using np.arange().

Note that NumPy arrays are displayed without commas in the output.

import numpy as np

# an_np_array = np.array(python_list)
list1D = [1, 2, 3]
array1D = np.array(list1D)
print(f'array1D from list1D is {array1D}')

# np.zeros(tuple)
zeros_array = np.zeros((2,3))
print(f'zeros_array is {zeros_array}')

# np.ones(tuple)
ones_array = np.ones((2,3))
print(f'ones_array is {ones_array}')

# np.arange(start, end, step)
range_array_1 = np.arange(5)
range_array_2 = np.arange(0, 5, 0.5)
print(f'range of 5 is {range_array_1}')
print(f'range with step is {range_array_2}')

'''
array1D from list1D is [1 2 3]
zeros_array is [[0. 0. 0.]
                [0. 0. 0.]]
ones_array is [[1. 1. 1.]
                [1. 1. 1.]]
range of 5 is [0 1 2 3 4]
range with step is [0.  0.5 1.  1.5 2.  2.5 3.  3.5 4.  4.5]
'''

We can certainly make an array of any number by multiplying a ones array by that number. This is one simple example of execution-implementation views previously mentioned. This idea will be introduced in the Broadcasting section.

Create a NumPy Array (from an pre-existing array)

• Each array has its own shape as we saw earlier, but such a shape isn’t fixed but can rather be reasonably reshaped using np.reshape(), reshape(), or flatten(). These methods operate from left to right and from top to bottom. While flatten() transforms any array into a 1D or vector, the other two are more flexible. In any case, we can only turn an array into a shape in which the number of elements stays unchanged.

For future references, flatten() is utilized in CNNs and Transformers.

import numpy as np

# np.reshape(an_array, new_shape_tuple)
data = np.array([[1, 2, 3, 4], [4, 5, 6, 7], [7, 8, 9, 0]])
data_rs_1 = np.reshape(data, (2,6))
data_rs_1
>>> [[1 2 3 4 4 5]
     [6 7 7 8 9 0]]

# an_array.reshape(new_shape_tuple)
data_rs_2 = data.reshape((4,-1))
data_rs_2
>>> [[1 2 3]
     [4 4 5]
     [6 7 7]
     [8 9 0]]

# an_array.flatten()
data_rs_3 = data.flatten()
data_rs_3
>>> [1 2 3 4 4 5 6 7 7 8 9 0]

From the example above, notice that one dimension can be stated with some negative number as long as all others are specified when reshaping an array.

• Another commonly used method to initialize an array in NumPy is np.repeat().
  • It creates a new array by making a specified number of copies of elements on each axis and placing them along the desired axis.
  • The stated axis can be no more than the original array’s dimension.

This method may be thought of as element-wise repeat. If an array is a LEGO model, np.repeat() makes copies of each individual piece and assemble one set of copies before moving on to the next. On the other hand, if we want to make photocopies of the entire model and tile them up like in Python lists, we need to use np.tile() instead.

import numpy as np

data = np.random.randint(1, 9, (2, 3))
print(f'original array is {data}')

# np.repeat(an_array, repeats, axis)
repeated_data_0 = np.repeat(data, 2, axis=0)
print(f'array repeated along axis 0 is {repeated_data_0}')

repeated_data_1 = np.repeat(data, 2, axis=1)
print(f'array repeated along axis 1 is {repeated_data_1}')

'''
original array is [[5 4 6]
                   [2 4 5]]
array repeated along axis 0 is [[5 4 6]
                                [5 4 6]
                                [2 4 5]
                                [2 4 5]]
array repeated along axis 1 is [[5 5 4 4 6 6]
                                [2 2 4 4 5 5]]
'''

Indexing and Broadcasting

Array Indexing

Slicing

arr[for_axis_0, for_axis_1, ...]

Slicing an array is similar to slicing a list. The main difference between the two is that we can slice an array based on our wanted axes. Let’s take a look at a few examples below.

import numpy as np

data = np.arange(1, 7).reshape((3,-1))
print(f'original data is {data}')

print(f'data at 0 on axis=0 and at 1 on axis=1 is {data[0, 1]}')

print(f'data from 1 to 3 on axis=0 is {data[1:3]}')

print(f'data from 0 to 2 on axis=0 and at 0 on axis=1 is {data[0:2, 0]}')

print(f'data from end to start on axis=0 and from start to end on axis=1 is {data[:, :]}')

'''
original data is [[1 2]
                  [3 4]
                  [5 6]]
data at 0 on axis=0 and at 1 on axis=1 is 2
data from 1 to 3 on axis=0 is [[3 4]
                               [5 6]]
data from 0 to 2 on axis=0 and at 0 on axis=1 is [1 3]
data from end to start on axis=0 and from start to end on axis=1 is [[5 6]
                                                                     [3 4]
                                                                     [1 2]]
'''

Using the idea of slicing or indexing, we can also update an array. Note that updating values of a slice of an array changes the original array and vice versa. Visually, slicing an array is like revealing one part of an image rather than making a copy of that part.

import numpy as np

data = np.arange(1, 7).reshape((2,-1))
print(f'original data is {data}')

slice_data = data[:, 1:]
slice_data[0, 0] = 100

print(f'data after updating slice is {data}')

'''
original data is [[1 2 3]
                  [4 5 6]]
data after updating slice is [[1 100 3]
                              [4   5 6]]
'''

arr = np.ones((3,3))
sub_arr = arr[1:, 1:]
print('original:', sub_arr, arr)

# reassign a new array to sub_arr
sub_arr = sub_arr + 1
print('reassign:', sub_arr, arr)

'''
original: [[1. 1.]  [[1. 1. 1.]
           [1. 1.]]  [1. 1. 1.]
                     [1. 1. 1.]]
                   
reassign: [[2. 2.]  [[1. 1. 1.]
           [2. 2.]]  [1. 1. 1.]
                     [1. 1. 1.]]
'''

arr = np.ones((3,3))
sub_arr = arr[1:, 1:]
print('original:', sub_arr, arr)

# updating values of sub_arr
sub_arr += 1
print('update:', sub_arr, arr)

'''
original: [[1. 1.]  [[1. 1. 1.]
           [1. 1.]]  [1. 1. 1.]
                     [1. 1. 1.]]
                    
update: [[2. 2.]  [[1. 1. 1.]
         [2. 2.]]  [1. 2. 2.]
                   [1. 2. 2.]] 
'''

Using Lists as Indices
Lists can only return an equally-spaced subset of its whereas we can extract random elements from an array using a list of its indices and the slicing syntax.

import numpy as np 

data = np.arange(1,7).reshape((2,-1))
print(f'original data is {data}')

sliced_data = data[[1],[0,2]]
print(f'data on the second row of axis=0 and at indices 0 and 2 of axis=1 is {sliced_data}')

'''
original data is [[1 2 3]
                  [4 5 6]]
data on the second row of axis=0 and at indices 0 and 2 of axis=1 is [4 6]
'''

Boolean Indices
Boolean indexing is an efficent way of filtering data, referring to having a list of “True” or “False” as indices. This happens when we compare an array, technically its elements, to some value. NumPy then keeps only elements of index “True” and wrap them all in a 1D array.

import numpy as np 

data = np.arange(1,7).reshape((2,-1))
print(f'original data is {data}')

# test
boolean_idx = data > 2
print(f'boolean indices are {boolean_idx}')

result = data[boolean_idx]
print(f'elements of original data passing boolean indices is {result}')

'''
original data is [[1 2 3]
                  [4 5 6]]
boolean indices are [[False False True]
                     [True  True  True]]
elements of original data passing boolean indices is [3 4 5 6]
'''

We can mass change the values of elements in an array based on some comparision using this method. Check out practice exercises to learn more about it.

Operations Using Built-in Methods

Summation
Instead of using a for loop, we can use the built-in method np.sum() to calculate the summation of the entire array or each axis.

import numpy as np

data = np.arange(1,7).reshape((2,-1))
print(f'original data is {data}')

# np.sum(an_array, axis)
print(f'summation of all elements is {np.sum(data)}')
print(f'summation along axis=0 is {np.sum(data, axis=0)}')
print(f'summation along axis=1 is {np.sum(data, axis=1)}')

'''
original data is [[1 2 3]
                  [4 5 6]]
summation of all elements is 21
summation along axis=0 is [5 7 9]
summation along axis=1 is [6 15]
'''

Max/Min
Similar to summation, we can compute maximum or minimum values for an array as a whole or along specific axes using max() or min().

import numpy as np

data = np.arange(1,7).reshape((2,-1))
print(f'original data is {data}')

# an_array.max(axis)
print(f'max values is {data.max()}')
print(f'max values along axis=0 is {data.max(axis=0)}')
print(f'max values along axis=1 is {data.max(axis=1)}')

'''
original data is [[1 2 3]
                  [4 5 6]]
max values is 6
max values along axis=0 is [4 5 6]
max values along axis=1 is [3 6]
'''

Broadcasting

Operations introduced in a moment do not change the original array.

Matrix and Scalar
In Mathematics, we can multiply and divide a matrix by a scalar. We can do the same thing with an array in Python. Moreover, we can perform addition, subtraction, and exponential expression between a matrix and a scalar. This is a simple example of execution-implementation views mentioned at the beginning of this post.

import numpy as np

data = np.arange(1,7).reshape((2,-1))
print(f'original data is {data}')

# test
print(f'data multiplied by 2 is {data * 2}')
print(f'data adding 2 is {data + 2}')
print(f'data subtracted from 2 is {2 - data}')

'''
original data is [[1 2 3]
                  [4 5 6]]
data multiplied by 2 is [[2  4  6]
                         [8 10 12]]
data adding 2 is [[3 4 5]
                  [6 7 8]]
data subtracted from 2 is [[ 1  0 -1]
                           [-2 -3 -4]]
'''

Matrix and Vector
Instead of a scalar, NumPy allows basic mathematical operations between a matrix and a vector. In this case, a suitable vector is stretched to have the shape of the matrix and operations are carried out element-wise. In order for the extension to work, the shape of the vector must match the trailing dimensions of the matrix.

data = np.arange(24).reshape((2, 3, 4))
print(f'Original data is {data} and its shape is {data.shape}')

# 1D vector: shape (4,)
vector = np.array([10, 20, 30, 40])
print(f'Vector is {vector} and its shape is {vector.shape}')

# Add the vector to the 3D array
result = data + vector
print(f'A matrix added to a vector results in {result} and its shape is {result.shape}')

'''
Original data is [[[ 0  1  2  3]
                   [ 4  5  6  7]
                   [ 8  9 10 11]]
                  [[12 13 14 15]
                   [16 17 18 19]
                   [20 21 22 23]]] and its shape is (2, 3, 4)
Vector is [10 20 30 40] and its shape is (4,)
A matrix added to a vector results in [[[10 21 32 43]
                                        [14 25 36 47]
                                        [18 29 40 51]]
                                       [[22 33 44 55]
                                        [26 37 48 59]
                                        [30 41 52 63]]] and its shape is (2, 3, 4)
'''

Application in AI

AI is heavily applied to image data, especially in fields like Computer Vision and Medical Imaging. One of the basic AI applications is to change the brightness of images. Computers cannot interact with any other data but numerical one, thus we always have to convert images into a matrix of numeric values.

If you know the definition of function in Mathematics or still remember function in Python, conversion mimics such an idea—the domain/input contains 2D images and the range/ouptut contains multi-dimensional matrices depending on the kind of image input. Specifically, an image is an ndarray, but an ndarray is not necessarily represented by an image. We’ll discuss this more later.

An image can be measured by the number of pixels within itself or its height and width. We can compute the resolution of an image using these three parameters: pĩel, height, and width.

• Grayscale images: 2D matrices (Height, Width) of scalar pixels $p$ such that $0 \le p \le 255$ with 0 for black and 255 for white.
• RGB Color images: 3D matrices (Height, Width, channel) of matrix pixels \(p = \begin{bmatrix} r \\ g \\ b \end{bmatrix}\) where $0 \le r, g, b \le 255$. This convention comes straight from the field of art where mixtures of primary color channels including red R, green G, and blue B can represent colors in images.

OpenCV

1. OpenCV is to read, load, and save images. Image reading automatically performs the conversion.
2. matplotlib is to load images read by OpenCV.

Here we only demonstrate using matplotlib to load images. It works on both local and remote servers.

# import opencv and matplotlib
import cv2
import matplotlib.pyplot as plt

# read an image
# 1 for color images, 0 for grayscale images
color_image = cv2.imread('image1.png', 1)
print(type(color_image))
print(color_image.shape)

# save and load an image simultaneously
# argument cmap="gray" for grayscale images
plt.imshow(color_image)

'''
<class 'numpy.ndarray'>
(162, 311, 3)
'''

Do you notice the difference in color between the original image and the one read by OpenCV and loaded by matplotlib? What do you think we can do to fix it?

This is a subtle but critical difference since color accuracy makes a tremendous impact on practical applications of vision and image-processing tasks, specifically in the field of AI.

• Regular color image: channels are in order of R G B.
• OpenCV color: images are read in order of B G R.
• matplotlib color: images are saved and loaded in order of R G B.

Check out the following code and pictures to see this concept clearly when a red image becomes blue.

import cv2
import numpy as np
import matplotlib.pyplot as plt

# create a pure red image (in BGR format for OpenCV)
red_bgr = np.zeros((162, 311, 3), dtype=np.uint8)
red_bgr[:, :] = [0, 0, 255]  # B=0, G=0, R=255 → Red

# save the image using OpenCV (optional)
cv2.imwrite('red_image.png', red_bgr)

# read the red image using OpenCV
img = cv2.imread('red_image.png')
print(img) # R=0, G=0, B=255 -> Blue

# display with matplotlib WITHOUT color conversion
plt.imshow(img)
plt.title('Should Be Red, But Looks Blue')
plt.axis('off')

Recall that an image is an ndarray which, up to this point, we know how to manipulate using NumPy techniques we’ve learned. For this specific array, (Height, Width, channel), we wish to keep the first two dimensions but modify the order of channel from RGB to BGR. Without NumPy, we can use a built-in method in OpenCV before loading the image with matplotlib.

color_image = cv2.cvtColor(color_image, cv2.COLOR_BGR2RGB)

Remember to once again convert the order of channels when save the image using cv2.imwrite('image_output_name', image_matrix).

Brightness Changes

Data Type
Earlier we said an image is an ndarray but an ndarray is not necessarily represented by an image. The reason behind is that their elements’ values are of different data types.

• ndarray’s data type is int64 or float64.
• images’ data type is uint8 representing 8 bits with unsigned/non-negative integers in from 0 to 255, particularly modulo 256—the values start back from 0 after 255.

Just like other data types in Python, we can always cast them to a specified type, which is exactly what we want—making an array an image and vice versa.

a_variable.astype(np.a_data_type)

data = np.array([0.2, 255.6])
print('before changing data type')
print(data)
print(data.dtype)

data = data.astype(np.uint8)
print('after changing data type')
print(data)
print(data.dtype)

'''
before changing data type
[2.000e-01 2.556e+02]
float64
after changing data type
[  0 255]
uint8
'''

Naive Solution to Brightness Change - Issue with uint8
The property of modulo 256 is unit8’s own issue. If we want to brighten an image, a solution is to add some value to each elements of channels. But this causes light-colored regions become dark since modulo operates circularly rather than linearly.

grayscale_image = cv2.imread('M02W01Wed_numpy/4.opencv/image2.png', 0)

# attempt to make an image brighter
grayscale_image = grayscale_image + 50

Practial Solution to Brightness

1. Step 1: To make an image data type linear, we can apply casting to force its data type to be float64 which can handle beyond 255 before addition.

    grayscale_image = grayscale_image.astype(float64)
    grayscale_image = grayscale + 100
   

2. Limit the brightness for out-of-bound values using np.clip() or np.where().

    # np.clip(an_array, min_assigned_to_all_less_than, max_assigned_to_all_less_than)
    lighter_grayscale_image_clip = np.clip(grayscale_image, 0, 255)
    lighter_grayscale_image = lighter_grayscale_image_clip.astype(np.uint8)
   

    # np.where(condition, array_element_for_True, array_element_for_False)
    lighter_grayscale_image_where = np.where(lighter_grayscale_image > 255, 255, lighter_grayscale_image)
    lighter_grayscale_image = lighter_grayscale_image_where.astype(np.uint8)
   

FAQ

Here are some good questions asked during class. The answers to these questions came from either some admin, TA, STA, or members of AIO2025.