BARCODE 2.0 Metrics

The BARCODE platform is divided into 3 independent branches: the image binarization (IB) branch, intensity distribution (ID) branch, and optical flow (OF) branch. The 25 specific parameters computed across all branches in BARCODE 2.0, described below, are displayed in a color-scaled array to allow for rapid inspection and categorization. The numerical values that encode the colorized array, as well as information-rich reduced data structures (RDS) from which the parameters are determined, are also saved for further quantitative physics-driven and data-driven analyses.

  • Image Binarization (IB) Branch: Connectivity, Maximum Island Area, Maximum Void Area, Maximum Island Area Change, Maximum Void Area Change, Initial Maximum Island Area, Initial 2nd Maximum Island Area, Mean Island Anisotropy, Mean Island Area, Total Island Area, Mean Island Separation, Structural Correlation Length
  • Intensity Distribution (ID) Branch: Maximum Kurtosis, Maximum Median Skewness, Maximum Mode Skewness, Kurtosis Change, Median Skewness Change, Mode Skewness Change
  • Optical Flow (OF) Branch: Mean Speed, Speed Change, Flow Direction, Directional Spread, Velocity Correlation Length, Divergence, Curl

BARCODE assumes that the data inputs are stacks of 2D images (frames) acquired over time (i.e., videos, tiff files) using optical microscopy. Input video files can have multiple channels, corresponding to specific components in the materials, which are analyzed as separate videos. Each frame of a video can be described as an \( m \times n \) matrix of pixels that reports the intensity value at each pixel position \( B(x,y) \) where \( 0 \le x \le m \) and \( 0 \le y \le n \). Each frame \( i^{th} \) is then expressed as a matrix of \( B(x,y,i) \) values. To increase processing speed and reduce the data size, it is possible to downsample in time, by choosing to analyze every \( k^{th} \) frame. Additionally, in the OF branch, it is possible to downsample in space, by averaging together \( p \times p \) windows of pixels to result in a stack of \( T/k \) images of \( m/p \times n/p \) pixels. In BARCODE 2.0, the IB branch also includes a user-selected spatial binning ratio, denoted \( p_b \), applied before binarization. By default, \( k=10 \), \( p=8 \) for OF downsampling, and \( p_b=2 \) for IB binning.

Warnings and Flags

BARCODE may sometimes have difficulty with images that are considered low contrast or which show saturation. We define a low contrast or 'dim' video (or specific channel of a video) as one in which the the mean \( \overline{B(i)} \) and minimum \( B_{min}(i) \) pixel intensity of the first image of the video (or channel) satisfy the criterion $$\frac{2}{e} \cdot \overline{B(i)} \le B_{min}(i)$$

A video (or channel) meeting the 'dim' criterion is assigned \( \mathrm{Flag}=1 \). Users can choose to analyze or skip 'dim' videos.

We define a saturated video as one in which the maximum pixel intensity is equal to the mode pixel intensity value for all frames \( i \): $$B_{max}(i)=B_i^{mode}$$ A video (or channel) meeting the saturation criterion is assigned \( \mathrm{Flag}=2 \). Users can choose to analyze or skip saturated videos.

Videos that include frames in which the structural autocorrelation function (see Section 1.11) or velocity autocorrelation function (see Section 3.5) does not decay enough to determine the corresponding correlation length \( \xi_I \) or \( \xi_v \) are assigned \( \mathrm{Flag}=3 \) or \( \mathrm{Flag}=4 \), respectively. This flag indicates that the correlation length extends beyond the length of the field of view. Videos in which all frames meet this criteria are assigned \( NaN \).

1. Image Binarization Branch

We binarize each frame of the video using a threshold value of \( \overline{B(i)} \times (1 + \text{offset}) \) where the offset is user-inputted with a default value of 0.1 and \( \overline{B(i)} \) is the mean intensity of the \( i^{th} \) original image, averaged across all pixel positions:

$$\overline{B(i)}=\frac{1}{N}\sum_{x,y}B(x,y,i) \quad (1)$$

Here, N is the total number of pixels in the image of size \( m \times n \). The intensity of each pixel in the \( i^{th} \) frame of the binarized image is given by:

$$b(x, y, i) = \begin{cases} 1, & \text{if } B(x, y, i) \ge \overline{B(i)} \times (1 + \text{offset}) \\ 0, & \text{if } B(x, y, i) < \overline{B(i)} \times (1 + \text{offset}) \end{cases} \quad (2)$$

In BARCODE 2.0, the IB branch first applies spatial binning to each frame using the user-selected binning ratio \( p_b \). The binned intensity image is:

$$\tilde{B}(x',y',i)=\frac{1}{p_b^2}\sum_{x=(x'-1)p_b+1}^{x'p_b}\sum_{y=(y'-1)p_b+1}^{y'p_b}B(x,y,i) \quad (3)$$

The threshold is still computed from the original-frame mean \( \overline{B(i)} \), but the threshold is applied to the binned image:

$$b(x, y, i) = \begin{cases} 1, & \text{if } \tilde{B}(x', y', i) \ge \overline{B(i)} \times (1 + \text{offset}) \\ 0, & \text{if } \tilde{B}(x', y', i) < \overline{B(i)} \times (1 + \text{offset}) \end{cases} \quad (4)$$

After thresholding, BARCODE 2.0 removes small connected objects and fills small holes using the user-selected minimum island size parameter. If invert binarization is selected, the binary image is replaced by \( 1-b(x',y',i) \). Each parameter computed in this branch, described below, is derived from analysis of the stack of binarized images, which are saved as RDS.

1.1 Connectivity \( C \)

Connectivity \( C \) evaluates the persistence of percolated pathways across a material by determining whether a continuous path of \( b=1 \) pixels exists across the x or y axis in each frame. A frame with or without a percolated path is assigned a connectivity of \( C_i=1 \) or 0. We determine the number of frames with \( C_i=1 \) and divide by the total number of frames \( N_f \) to determine a fractional measure of connectivity C for each video, which we report in the barcode. To compute \( C_i \), we label the connected \( b=1 \) areas in the binarized images using scipy.ndimage.label:

$$L(x,y,i)=\text{label}(b(x,y,i)) \quad (5)$$

As a result, each distinct connected region in the binary image \( b(x,y,i) \) is assigned a unique label number. We then record the labels of the connected areas on the boundaries of the image of size \( m \times n \): \( L_{y=0} \), \( L_{y=n-1} \), \( L_{x=0} \), and \( L_{x=m-1} \). Next, we determine the connectivity by determining whether a single label persists from one external boundary to the other in the x or y direction.

$$C_i = \begin{cases} 1, & \text{if } L_{y=n-1} \cap L_{y=0} \neq \emptyset \text{ or } L_{x=0} \cap L_{x=m-1} \neq \emptyset \\ 0, & \text{otherwise} \end{cases} \quad (6)$$

To determine the connectivity \( C \) for a given video, we compute the average \( C_i \) value across all frames:

$$C=\frac{1}{N_f}\sum_{i}C_i \quad (7)$$

1.2 Maximum island area \( I \)

The maximum island area \( I \) is defined as the area of the largest contiguous region of white (\( b=1 \)) pixels over the entire binarized video. Let \( I_i^k \) represent the area (in pixels) of the \( k^{th} \) connected region in frame i, where \( k=1 \) represents the largest connected white region. We normalize by the total number of pixels. In BARCODE 2.0, this normalization is performed using the binned image size \( N_b=(m/p_b)(n/p_b) \). To produce a more statistically robust measure of I, we compute and report the mean of the highest 10% of \( I_i^1 \) values in the videos:

$$I=\frac{1}{0.1N_f}\sum_{I_i^1 \in \text{highest } 10\%}\frac{I_i^1}{N_b} \quad (8)$$

1.3 Maximum void area \( V \)

The maximum void area \( V \) is the complement to the maximum island area that identifies and computes the areas of the largest contiguous regions of \( b=0 \) (black) pixels in each frame of the binarized video. The computation is identical to that described above but performed on \( b=0 \).

1.4 Maximum island area change \( \Delta I \)

The maximum island area change \( \Delta I \) quantifies the relative change in the area of the maximum island at the end compared to the beginning of the video. X is a user-defined parameter to specify the fraction (default \( X=0.05 \)).

$$\Delta I=\frac{\langle I_i^1([(1-X)N_f]:N_f)\rangle}{\langle I_i^1(0:XN_f)\rangle} \quad (9)$$
$$\langle I_i^1(i_a:i_b)\rangle=\frac{1}{b-a}\sum_{i_a}^{i_b}I_i^1 \quad (10)$$

1.5 Maximum void area change \( \Delta V \)

The complement to the island area change performed on \( b=0 \) rather than \( b=1 \) pixels.

1.6 Initial maximum and 2nd maximum island \( I_{0,1}, I_{0,2} \)

The initial maximum island, \( I_{0,1} \), is defined as the average of the largest island areas \( I_i^1 \) in a user-defined fraction of initial frames. The initial 2nd maximum island, \( I_{0,2} \), is defined as the average of the 2nd largest island areas \( I_i^2 \).

$$I_{0,1}=\frac{1}{XN_f}\sum_{i \in \text{first } 100X\%}I_i^1 \quad (11)$$

$$I_{0,2}=\frac{1}{XN_f}\sum_{i \in \text{first } 100X\%}I_i^2 \quad (12)$$

1.7 Mean island anisotropy \( A_I \)

For each connected island in frame i, BARCODE 2.0 estimates the major and minor axis lengths, denoted \( a_i^k \) and \( c_i^k \). The anisotropy of island k in frame i is defined as:

$$A_i^k=\frac{a_i^k}{c_i^k} \quad (13)$$

The reported mean island anisotropy is the average of this ratio over all islands and analyzed frames:

$$A_I=\langle A_i^k\rangle_{i,k} \quad (14)$$

1.8 Mean island area \( \overline{I} \)

The mean island area is calculated by averaging the areas of all connected \( b=1 \) islands within each analyzed frame, then averaging this frame-level mean over the video:

$$\overline{I}=\frac{1}{N_f}\sum_{i}\left(\frac{1}{M_i}\sum_{k=1}^{M_i}I_i^k\right)\frac{1}{N_b} \quad (15)$$

1.9 Total island area \( I_{tot} \)

The total area occupied by all \( b=1 \) islands in each analyzed frame, averaged over the video:

$$I_{tot}=\frac{1}{N_f}\sum_{i}\left(\sum_{k=1}^{M_i}I_i^k\right)\frac{1}{N_b} \quad (16)$$

1.10 Mean island separation \( D_I \)

The pairwise centroid distance between islands k and l in frame i is:

$$d_i^{kl}=||r_i^k-r_i^l|| \quad (17)$$

If \( \mathcal{N}_q(k) \) denotes the nearest-neighbor set for island k, the reported mean island separation is:

$$D_I=\langle d_i^{kl}\rangle_{i,k,l \in \mathcal{N}_q(k)} \cdot s \quad (18)$$

1.11 Structural correlation length \( \xi_I \)

A spatial image autocorrelation (SIA) function is computed from each image:

$$B'(x,y,i)=\frac{B(x,y,i)-\overline{B(i)}}{\sigma_B(i)} \quad (19)$$

The 2D autocorrelation image is calculated using the Fourier transform:

$$C_I(\Delta x,\Delta y,i)=\frac{1}{N}\mathcal{F}^{-1}[\mathcal{F}(B')\mathcal{F}(B')^*] \quad (20)$$

The autocorrelation image is then radially averaged to obtain \( C_I(r,i) \).

The structural correlation length \( \xi_I \)for frame \( i \) is determined by identifying the first pair of neighboring radial distances whose correlation values lie on opposite sides of the threshold \( 1/e \):

$$C_I(r_i,i) > e^{-1}, \quad C_I(r_{i+1},i) \le e^{-1} \quad (21)$$

$$\xi_I(i) = \begin{cases} r_i, & \text{if } C_I(r_i, i) - e^{-1} < e^{-1} - C_I(r_{i+1}, i) \\ r_{i+1}, & \text{otherwise} \end{cases} \quad (22)$$

If no consecutive pair satisfying

\[ C_I(r_i,i) > e^{-1}, \quad C_I(r_{i+1},i) \le e^{-1} \] exists then \( \xi_I(i) \) is reported as \( NaN \).

This indicates that the structural correlation remained above the threshold value of \( 1/e \) throughout the measured spatial range. The summary csv file will report Flag = 3 in this case.

2. Intensity Distribution Branch

In this module, we analyze the properties of the distribution of pixel intensities within a frame, as well as how this distribution changes over time. The intensity distribution curve \( f(B) \) is calculated as:

$$f(B)=\frac{1}{N}\sum_{x,y}\delta(B(x,y)-B) \quad (23)$$

We produce a normalized probability distribution \( P(B_{n_j})=\frac{C(B_{n_j})}{\sum_{n_j}C(B_{n_j})} \). We then calculate the moments of the probability distribution:

  • Mean: \( \mu_i=\mathbb{E}[B]=\sum_{n_j}B_{n_j}P(B_{n_j}) \)
  • Centralized Moment: \( \mu_{i,m}=\mathbb{E}[(B-\mu_i)^m]=\sum_{n_j}(B_{n_j}-\mu)^mP(B_{n_j}) \)
  • Variance: \( \sigma_i^2=\mu_{i,2} \)

2.1 Maximum kurtosis \( K \)

A high value of kurtosis indicates that the distribution contains heavier tails compared to a normal Gaussian curve. We calculate kurtosis for a given frame i as \( K_i=\frac{\mu_{i,4}}{\sigma_i^4}-3 \). Maximum kurtosis is defined as the average of the highest 10% kurtosis values:

$$K=\frac{1}{0.1N_f}\sum_{K(i) \in \text{top } 10\%}K(i) \quad (24)$$

2.2 Maximum Median skewness \( S_1 \)

The median skewness for the \( i^{th} \) frame is defined as:

$$S_1(i)=3 \times \frac{\mu_i-B_i^{\text{median}}}{\sigma_i} \quad (25)$$

The maximum median skewness is calculated by taking the average of the highest 10% median skewness values:

$$S_1=\frac{1}{0.1N_f}\sum_{S_1(i) \in \text{top } 10\%}S_1(i) \quad (26)$$

2.3 Maximum Mode skewness \( S_2 \)

The mode skewness is defined as:

$$S_2(i)=\frac{\mu_i-B_i^{\text{mode}}}{\sigma_i} \quad (27)$$

Maximum mode skewness calculation:

$$S_2=\frac{1}{0.1N_f}\sum_{S_2(i) \in \text{top } 10\%}S_2(i) \quad (28)$$

2.4 Kurtosis change \( \Delta K \)

$$\Delta K=\langle K((1-X)N_f:N_f)\rangle-\langle K(0:XN_f)\rangle \quad (29)$$

2.5 Median skewness change \( \Delta S_1 \)

$$\Delta S_1=\langle S_1((1-X)N_f:N_f)\rangle-\langle S_1(0:XN_f)\rangle \quad (30)$$

2.6 Mode skewness change \( \Delta S_2 \)

$$\Delta S_2=\langle S_2((1-X)N_f:N_f)\rangle-\langle S_2(0:XN_f)\rangle \quad (31)$$

3. Optical Flow Branch

To measure the dynamics of active materials, we implemented the Farneback optical flow algorithm. The output is a dense optical flow field represented as a 2-dimensional array. For each frame pair, the format is:

$$\text{flow}[x,y]=\begin{bmatrix} u_{xy} \\ v_{xy} \end{bmatrix} \quad (32)$$

3.1 Mean speed \( v \)

For a given frame pair \( (i,i+\Delta i) \), the flow field is calculated as:

$$\text{flow}_i[x,y]=\begin{bmatrix} u_{xy}^i \\ v_{xy}^i \end{bmatrix} \quad (33)$$

Then the mean speed \( v \) is calculated as the average magnitude of the displacement vectors:

$$V_{xy}^i=\frac{\sqrt{(u_{xy}^i)^2+(v_{xy}^i)^2}}{\Delta i} \quad (34)$$

$$v(i)=\frac{1}{N_v}\sum_{x=1}^{m/p}\sum_{y=1}^{n/p}V_{xy}^i \quad (35)$$

$$v=\frac{1}{N_f}\sum_{i=1}^{N_f}v(i) \quad (36)$$

$$v=\frac{1}{N_v*N_f}\sum_{i=1}^{N_f}\sum_{x=1}^{m/p}\sum_{y=1}^{n/p}V_{xy}^i \quad (37)$$

3.2 Speed change \( \Delta v \)

Defined as the change in the mean speed between the last X% of frames and the first X% of frames:

$$\Delta v=\langle v[(1-X)N_f:N_f]\rangle-\langle v[0:XN_f]\rangle \quad (38)$$

3.3 Flow direction \( \theta \)

Each angle is first represented as a unit vector on the unit circle:

$$\tilde{V}_{ixy} = (\cos\theta_{ixy}, \sin\theta_{ixy}) = \frac{(v_{ixy}, u_{ixy})}{\sqrt{v_{ixy}^2 + u_{ixy}^2}} \quad (39)$$

Then, the angle of flow direction is calculated using the two-argument arctan2 function:

$$\theta = \arctan2\left( \frac{1}{N_v N_f}\sum_{i=1}^{N_f}\sum_{x=1}^{m/p}\sum_{y=1}^{n/p}\sin\theta_{ixy}, \frac{1}{N_v N_f}\sum_{i=1}^{N_f}\sum_{x=1}^{m/p}\sum_{y=1}^{n/p}\cos\theta_{ixy} \right) \quad (40)$$

3.4 Directional spread \( \sigma_\theta \)

The mean vector is defined as:

$$R_i = \frac{1}{N_v}\sum_{x=1}^{m/p}\sum_{y=1}^{n/p}\tilde{V}_{ixy} = (R_{dx}, R_{dy}) \quad (41)$$

The length of \( R_i \), also called the mean resultant length, is given by:

$$L_i = ||R_i|| = \sqrt{R_{dx}^2 + R_{dy}^2} \quad (42)$$

Finally, the angular spread can be computed from L and averaged over all frames:

$$\sigma_\theta = \frac{1}{N_f}\sum_{i=1}^{N_f}\sqrt{-2\ln L_i} \quad (43)$$

3.5 Velocity correlation length \( \xi_v \)

BARCODE 2.0 computes a spatial velocity correlation function from the optical flow field:

$$C_v^i(\Delta x, \Delta y) = \frac{\langle V_i(x, y) \cdot V_i(x + \Delta x, y + \Delta y)\rangle_{x,y}}{\langle ||V_i(x, y)||^2 \rangle_{x,y}} \quad (44)$$

The velocity correlation length for frame i is determined by identifying the first pair of neighboring radial distances whose correlation values lie on opposite sides of the threshold of \( 0.5 \):

$$C_v^i(r_i,i) > 0.5, \quad C_v^i(r_{i+1},i) \le 0.5 \quad (45)$$

$$\xi_v(i) = \begin{cases} r_i, & \text{if } C_v^i(r_i, i) - 0.5 < 0.5 - C_v^i(r_{i+1}, i) \\ r_{i+1}, & \text{otherwise} \end{cases} \quad (46)$$

If no consecutive pair satisfying

\[ C_v^i(r_i,i)>0.5, \qquad C_v^i(r_{i+1},i)\le0.5 \] exists then \( \xi_v(i) \) is reported as \( NaN \).

This indicates that the velocity correlation remained above the threshold value of \( 0.5 \) throughout the measured spatial range. The summary csv file will report Flag = 4 in this case.

3.6 Divergence \( \nabla \)

The velocity field is first normalized to a unit vector field:

$$\hat{V}_i(x, y) = \frac{V_i(x, y)}{||V_i(x, y)||} \quad (47)$$

The cumulative normalized field through frame pair j is:

$$S_j(x, y) = \sum_{i=1}^{j}\hat{V}_i(x, y) \quad (48)$$

The divergence field is then computed as:

$$\nabla \cdot S_j = \frac{\partial S_{jx}}{\partial x} + \frac{\partial S_{jy}}{\partial y} \quad (49)$$

3.7 Curl \( \Omega \)

BARCODE 2.0 also calculates the out-of-plane curl of the normalized optical flow field for each frame pair:

$$\Omega_i(x, y) = (\nabla \times \hat{V}_i)_z = \frac{\partial \hat{V}_{iy}}{\partial x} - \frac{\partial \hat{V}_{ix}}{\partial y} \quad (50)$$

The reported curl metric is the average of the spatial mean curl values across analyzed frame pairs:

$$\Omega = \frac{1}{N_f}\sum_i \langle \Omega_i(x, y) \rangle_{x,y} \quad (51)$$

References

  1. Scipy. NDImage Documentation. Available: https://docs.scipy.org/doc/scipy/reference/ndimage.html
  2. Zwillinger, D. and Kokoska, S. (2000). CRC Standard Probability and Statistics Tables and Formulae. Chapman & Hall: New York. 2000.
  3. OpenCV. calcOpticalFlowFarneback Documentation. Available: https://docs.opencv.org/3.4/dc/d6b/group__video__track.html. Accessed Aug. 10, 2024