Math for AI 105

Acknowledgement: This post is based mainly on Dr. Vinh-Dinh Nguyen’s presentation and slides and partly on on TA, STA, and peers’ responses from the course AIO2025 offered by AI VIETNAM.

Relevance: The audience of this lecture were those who have read Math for AI 104 or been familiar with probability and statistics. Note that only basic probability material is introduced here; we’ll pick up specific concepts when in need.

In today’s post, we’ll explore some descriptive statistics and probability distribution theory. We’ll also discuss some applications in AI including image processing.

Descriptive Statistics refers to tools that summarize and describe features of a dataset, providing us with a clearer picture of the data. There are three main categories: central tendency, variability, and distribution.

Measure of Central Tendency

Mean and Expected Value

Expected value is not considered a measure of central tendency, but this concept and mean are closely related as they both represent some sort of average.

Median

Given a dataset of N items $X = \{X_1, \cdots, X_N\}$, median is the value dividing the sorted dataset in half. To obtain the median, we follow two steps.

1. Sort $X$.
2. Calculate the median $m$:
   • If $N$ is odd, then $m = X_{\frac{N + 1}{2}}$.
   • If $N$ is even, then $m = \frac{1}{2} (X_{\frac{N}{2}} + X_{\frac{N + 1}{2}})$.

Applications in AI

Image Processing: Image Blurring using Mean
The media show pictures or videos and some images are wished to be blurred for privacy or security. We can apply a process called image blurring to accomplish this purpose.

Using a average/mean filter (AF) is a common and simple method to execute this process. The core idea behind it is to smooth out sharp edges and reduce the intensity variations by replacing each pixel’s value with the average of its neighboring pixels including itself. Averaging pixel values is akin to mixing paint colors togther. The more colors we mix, the more the distinctiveness of individual colors is lost.

Correlation ~ Convolution
To implement this idea, we use a technique called correlation ~ convolution. We know image data is an ndarray or a matrix where each appropriate set of values is a pixel.

Take a grayscale image for example. This can be turned into a 2D $m \times n$ matrix. We want to blur the whole image. The code is below.

1. Determine the shape of neighboring pixel areas, say a $j \times k$ matrix.
2. Take a matrix of that shape and call it a kernel.
3. Elements of this matrix are valued $\frac{1}{\text{number of elements}} = \frac{1}{j \cdot k}$ since we want to compute mean.
4. Slide the kernel over each neighboring pixel area from left to right and from top to bottom to obtain those pixels’ mean.
5. Place the mean on the position of the top-left-corner pixel. It’s best practice to store those means in a new array.
6. The new matrix’s shape is $(m - j + 1) \times (n - k + 1)$.

The change in shape makes the new image smaller. To handle this issue, we need a technique calld padding—adding layers of numbers surrounding the original matrix. This will take care of the lost region. Here we can

• add $j - 1$ and $k - 1$ layers of zeros for the first and second dimension, respectively.
• copy edge values to the padding region.

Of course, we can also blur part of an image. This part is called Region of Interest (ROI). Recall that an image is an array part of which we can take out by slicing. Check out Math for AI 101 for more information about updating a slice of an array.

• Static image: If we already know the coordinates of ROI, using slicing with specific numbers is suitable.
• Static and moving image: If we don’t know the exact coordinates of ROI, we need to detect that part first and then apply slicing. For a facial area, we can use the Haar-Cascade Face Detection algorithm, which returns the top-left-corner coordinates, height, and width of ROI. The face detection set up is as follows.

face_cascade = cv2.CascadeClassifier('haarcascade_frontalface_default.xml')

For color images, we can compute mean for each channel and concatenate three channels in the end to rebuild the third dimension.

There are some available packages we can use to apply put this technique into practice.

• NumPy: np.einsum()
• Scipy: spicy.signal.convolve2d()
• OpenCV: cv2.filter2D().

DEMONSTRATION MAYBE VIDEO IF TOO HARD TO WRITE

# load image and blurring

import numpy as np
import cv2

# load image in grayscale mode
image = cv2.imread('mrbean.jpg', 0)

# create kernel
kernel = np.ones((5,5), np.float32) / 25.0

# compute mean for each pixel
dst = cv2.filter2D(image, cv2.CV_8U, kernel)

# show images
cv2.imshow('image', image)
cv2.imshow('dst', dst)

# waiting for any keys pressed and close windows
cv2.waitKey(0)
cv2.destroyAllWindows()

Image Preprocessing: Image Denoising using Median
Pictures are often not taken by professionals and in the studio condition. They that can be blurred, underexposed or of unwanted speckles are usually referred to as grainy or noisy images. Hanlding these problems which is a part of image preprocessing is called image denoising. Using median is one of the simpliest method to address this.

One common type of noise is salt-and-pepper noise, that is, an image is speckly. We’ll go over an example of this type.

This median filter (MF) also uses the correlation ~ convolution technique to obtain the median of neighboring pixels. We can use package cv2.medianBlur() for this purpose.

Using median rather than mean in this case is because getting median can exclude too-large or too-small values which are noises. Using mean only makes those values smaller or larger but still keeps noises. The new image is also blurred after median filter but not as much affected as using mean filter.

# denoise salt and pepper

import numpy as np
import cv2

img1 = cv2.imread('mrbean_noise.jpg')

# compute median in each kernel with side=3
img2 = cv2.medianBlur(img1, 3)

# show images
cv2.imshow('img1', img1)
cv2.imshow('img2', img2)

# waiting for any keys pressed and close windows
cv2.waitKey(0)
cv2.destroyAllWindows()

Measure of Variability

Variance and Standard Deviation

Application in AI

Image Processing: Image Texture using Standard Deviation

When identifying an object, computers may just need to know its shape. That is, we focus on the boundary of objects in an image rather than their color. This is known as image texture—local patterns created by differences in relative color arrangement in an image. We will discuss this concept in detail later in this AI series.

Since we’re only interested in the edge of objects in an image, it suffices to just work with a grayscale version of an image for the sake of simplicity. This standard deviation filter also uses the correlation ~ convolution technique to obtain the standard deviation of neighboring pixels.

• The standard deviation remains small when the kernel moves over regions with little color variation or uniform intensity.
• The standard deviation increases sharply when the kernel moves across a boundary from a consistent-color region to a region with contrasting colors.

# extract image texture

import numpy as np
import cv2
import math
from scipy.ndimage.filters import generic_filter

# convert a color image to grayscale
img  = cv2.imread('img.jpg')
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
cv2.imwrite('edge_s1.jpg', gray)

# compute standard deviation in each kernel of size=7
x = gray.astype('float')
x_filt = generic_filter(x, np.std, size=7)
cv2.imwrite('edge_s2.jpg', x_filt)

# reassign small values (no sudden change in standard deviation)
# blacken insignificant areas or edges
x_filt[x_filt < 20] = 0
cv2.imwrite('edge_s3.jpg', x_filt)

# lighten significant edges
x_filt = x_filt*2.5
cv2.imwrite('edge_s4.jpg', x_filt)

Probability Distribution Function

A probability distribution is a function that describes the likelihood of occurence of possible events, that is, random variables. It is used to model the behavior of random phenomena from a dataset. There are three main types of probability distribution and we’ll discuss them in a moment.

• Probability Mass Function (PMF)
• Probability Density Function (PDF)
• Cumulative Distribution Function (CDF)

Discrete: PMF and CDF

PMF measures the probability of a given single value x occuring in a countable dataset, denoted by $p_X(x) = P(X = x)$. CDF measures the probability of values occuring from the beginning up to x in a countable dataset, denoted by $F_X(x) = P(X \le x)$.

Take rolling a dice for example. We have $p(3) = \frac{1}{6}$ and $F(3) = p(1) + p(2) + p(3) = \frac{3}{6}$.

Continuous: PDF and CDF

For experiments like assessing grades for an examination, there are so many outcomes that the probability to take on any particular value is always 0. We can hardly use PMF to understand the likelihood of a single value occuring in a dataset. This is where PDF comes into play.

PDF measures the relative likelihood of a random variable to take a value near x, denoted by $f(x)$. It is used to specify the probability that a given single value x falls within an interval, computed by integrate PDF $\int_{a}^{b} f(x) \,dx$. CDF measures the probability of values occuring from the beginning up to x in an uncountable dataset, denoted by $F_X(x) = P(X \le x)$.

Gaussian distribution function \(f(x) = \frac{1}{\sigma \sqrt{2\pi}} \exp^{-\frac{(x-\mu)^2}{2\sigma^2}}\) is a commonly used PDF. The total area under this curve is always 1 and

• the area within 1 𝜎, that is, $[\mu - \sigma, \mu + \sigma]$ is about 62.27%.
• the area within 2 𝜎, that is, $[\mu - 2\sigma, \mu + 2\sigma]$ is about 95.45%.
• the area within 3 𝜎, that is, $[\mu - 3\sigma, \mu + 3\sigma]$ is about 99.73%.

EXAMPLE DEMONSTRATION

Application in AI

Measure of Frequency Distribution: Histogram
Given a grayscale image, we can convert it into an array or matrix. Using such information, we can extract and arrange values in a frequency histogram. Such a histogram represents the frequency distribution of pixel values from the image. Pixel values range from 0 to 255 and we call each value a bin, so we have 256 bins in total. This allows us to imply if a grayscale image is bright or dark without even seeing the image as values a pixel gets lighter and lighter as it goes up.

The output of a frequency histogram depends on the number of pixels in an image as well as each bin. Thus, the output varies among images. We can normalize the output so that it acts as the probability density. This results in a density histogram.

Let $N$ be the total number of pixels of an image and $n_b$ be the number of pixels in bin $b$. The density value of bin b is

So the (probability) cumulative value up to bin b is computed as follows. We can use these values to construct a cumulative histogram—displaying how pixel values accumulate across the intensity range.

In fact, using any of these histograms allows us to determine the brightness of an image.

3 DEMONSTRATION from slide 54 describe how and why they are darker or brighter in each image caption

Histogram Equalization
Besides determining how bright or dark an image is, we can even change the brightness of an image by utilizing the probability distribution functions.

The core idea behind contrast adjustment is to modifying the distribution of the density histogram of an image so that the cumulative histogram approximates to a linear relation. That is, we wish uniformly redistribute the values of the frequency or density histogram. This helps the resulting image has a uniform intensity distribution. This process is known as histogram equalization.

Essentially, the ideal scenario is when all the bins have the same number of values. This gives a straight line when accumulating those values. It’s like a dice roll experiment.

DEMONSTRATION

We do not use density histogram here because it only describes individual bins—there is no relation between intensity values. Instead, to achieve this purpose, we use the cumulative histogram, which tells us how pixel intensities build up across the image. The formula that we’ll explore soon maps the original pixel intensity $b$ to a new value proportional to $H[b]$.

• A very dark pixel like 5 may be pushed to a brighter intensity.
• The accumulation levels out the brightness by spreading values uniformly.
  • If over two thirds of pixels are valued in [0, 50], they’ll be mapped to high intensity values like [0, 200].
  • And only a quarter of the rest falls in [50, 100], they’ll be mapped to just slightly higher intensity values or a narrow range like [200, 225].

  This way, no bin would get way too much addition than others.

Let $L$ be the number of gray/intensity levels. It corresponds to the number of possible values a pixel can take within an image. Each pixel in this case ranges from 0 to 255, so there can be 256 different gray levels.

Note that the cumulative values are in [0, 1], but the gray values are in [0, 255]. This is the reason to employ the multiplication and floor function.

We can also apply a similar method for a color image.

Example with a 1D Array for Simplicity

Problem statement: Given the following matrix. Demonstrate histograms before and after the histogram equalization process. $$\begin{bmatrix} 1 & 2 & 7 & 5 & 6 \\ 7 & 2 & 3 & 4 & 5 \\ 0 & 1 & 5 & 7 & 3 \\ 1 & 2 & 5 & 6 & 7 \\ 6 & 1 & 0 & 3 & 4 \end{bmatrix}$$ Breakdown:

• A table contains all the calculations above.
• N = 25 since it is a 5-by-5 matrix.
• L = 8 since the values run from 0 to 7.
• Show original and equalized histograms to see how new values change the graphs.

Idea: We apply the histogram equalization process.

$b$	Frequency	PMF ~ $h[b]$	CDF ~ $H[b]$	$(L - 1) \cdot H[b]$	$b_{new}$
0	2	0.08	0.08	0.56	1
1	4	0.16	0.24	1.68	2
2	3	0.12	0.36	2.52	3
3	3	0.12	0.48	3.36	3
4	2	0.08	0.56	3.92	4
5	4	0.16	0.72	5.04	5
6	3	0.12	0.84	5.88	6
7	4	0.16	1	7.0	7

FAQ

Here are some good questions asked during class. The answers to these questions came from either some admin, TA, STA, or members of AIO2025.