The information required to generate a heatmap consists of the location of gaze fixation and the duration of the fixation by filtering out the gaze fixations associated with the study.
Basic underlying concept
In order to create a heat map the gaze fixations have to be mapped onto the object/stimulus of interest. It is done by going through all the selected recordings and incrementing the count whenever a gaze fixation has the same (x, y) coordinates as another. This process leads to a histogram of gaze fixations.
After going through all the gaze fixations and incrementing their count accordingly, color values have to be assigned to them. Colors towards the red end of the spectrum represent higher count and so on.
The above figure shows the count value of gaze fixations at certain (x, y) pixel locations and the color assigned to them accordingly, but this doesn’t provide a clear visual understanding. In order to get a smooth heat map the distribution of counts around the gaze fixation point is also considered and approximated according to the Gaussian distribution (commonly known as the bell curve). Gradient color distribution is used to represent the gaze fixations and its neighbors.
Setting up the color gradient of heatmap
1. To define "Key intensity color", 5 parameters are used: Index, R, G, B, Alpha
- Index - A gaussian blurred histogram count of gaze fixations (in pixel location) is used to show where people looked more. Index is a number normalized in range [0, 255], which describes the level where a particular color would be assigned, with 255 being the highest value (Most gaze) at a pixel location (Calculating the index is discussed further in Section III below).
- R – red color intensity (0 – 255)
- G – green color intensity (0 – 255)
- B – blue color intensity (0 – 255)
- Alpha – This is the parameter used to control the transparency. If it is 0, then the underlying
image will show through fully. If it is 255, only the heatmap will show. For example, if it is 200, then it will mostly show the heatmap but also the image will show through.
2. Example of color spectrum with their index and respective RGB values
The below figure shows the Index values and their respective RGB and alpha values which are used to interpolate and produce the entire color map.
So, if the index value is higher meaning that the gaze fixation is more at that pixel location. The index value distribution of the neighbors around the fixation point is accomplished by the Gaussian function in order to smooth the heatmap. Therefore, the color values of the neighboring points gradually change as the pixel location moves away from the gaze fixation point (pixel location). The warmest colors (towards the red end of the spectrum) represent higher index value.
3. Example image showing a subset of the interpolated gradient color map
Figure 3 represents a subset of the colors available for the gradient heatmap obtained by interpolating the values from Figure 2. For example, index value of 120 was assigned with yellow color and index value 200 is assigned red. The colors assigned for index values between 120 and 200 can be seen from the 4th row till the 10th row in Figure 3. All index values greater than 200 are assigned with red color. Similarly a color value is assigned to all the index value from 0 to 255. This color gradient helps in providing a smoother visually aiding heat map.
Calculating index value for each gaze fixation point
- Gaussian kernels are also known as smoothing linear filters.
- Gaussian kernel is a 2-D kernel.
- 2-D Kernel window = ((kernel_size x min (width, height), (kernel_size x min (width, height)).
[width, height of the AOI]
- No action is done for edge cases while inserting the kernel.
1. For each gaze fixation point throughout the session, we insert the kernel onto the gaze point by applying the Gaussian kernel on it. The center point of the kernel is superimposed on the gaze point and the neighboring pixels are assigned values according to the Gaussian distribution.
This is done to all the gaze fixations points during the session/media resulting in a histogram of index values.
2. The histogram of gaze point fixations is normalized in the range [0, 255] to be on similar scale.
3. This generated normalized histogram heatmap is then assigned colors accordingly using the color spectrum produced by interpolation as shown above by matching the index value of gaze fixation. For example, index values above 200 are assigned with red color, index values around 100 is assigned shades of green, and so on.
4. Transparency of the heatmap overlayed on the image/media is adjusted according to the alpha value set while obtaining the color gradient through interpolation.
5. As seen in Figure 4, the kernel window of the gaussian curve is set to be (kernel_size times the minimum value between the Area of Interest (AOI’s) width and height). The orange line indicates a gaze fixation point in pixel location and its index value. A color gradient arrow represents how the color will be assigned to the neighbors depending on their index value.
6. Gaussian kernel is applied on each gaze fixation point, so its neighboring pixel’s index values are updated continuously. So, a histogram of index values of gaze fixation (in terms of pixel location) is obtained.
7. From Figure 5 it can be seen that the gaze fixation has an index value of 220 and it will be updated with red color on the heat map. Also, there is a gradient color arrow that represents how the color is assigned to the neighbors depending on the index value of the neighbors that fall within the gaussian curve as shown in the figure below.
The lower the kernel size, the lower the smoothing effect, as fewer neighboring pixels around the gaze fixation point are considered when applying the Gaussian kernel. If the kernel is very big then more unnecessary neighboring points are considered around a gaze fixation point. So there is a tradeoff between the kernel size and the smoothing effect.