BARCODE Tutorial
Installation
The BARCODE software can be installed as an executable application on both Mac and Windows. To install the software, following the link here, and download the zip file corresponding to your operating system of choice. The source code can also be directly downloaded on any system running Python 3.12 or later. To do so, download the source code from the repository here.
BARCODE requires the following packages:
ImageIO: Version ≥ 2.34.1
Matplotlib: Version ≥ 3.8.4
ND2: Version ≥ 0.10.1
Numpy: Version ≥ 2.0.1
OpenCV: Version ≥ 4.10.0.84
PIMS: Version ≥ 0.6.1
PyYAML: Version ≥ 6.0.2
Scipy: Version ≥ 1.14.0
Scikit-Image: Version ≥ 0.24.0
BARCODE was developed on the package versions listed above, and may not work on previous versions of the listed software packages. The executable version of the app includes these packages. These packages can be installed individually or using the included source code file requirements.txt. To use the source file installation, use the command pip install -r requirements.txt to install the required packages, then navigate to the source file in the terminal and run the software using the command python main.py.
General Walk-through
User Interface of BARCODE program
When first using the BARCODE application, it is recommended to start with a single channel of a single file. To do so, use the File Selection setting to select a file. Clicking Browse will allow you to search your file system for .tif and .nd2 files, while selecting the Folder Selection setting and Browse will allow you to select entire folders. Then, choose either the Parse All Channels setting to scan every video channel, or Choose Channel to scan a specific channel. Selecting a channel less than 0 will result in reverse indexing of channels (i.e, selecting -1 will run the program on the last channel of every video scanned, rather than the first).
Then in the general settings, it is recommended that the following features should be turned on: Binarization, Optical Flow, Intensity Distribution, Include Dim Files, Include Dim Channels, Verbose, Save Graphs, Dataset Barcode.
The Binarization, Intensity Distribution, and Optical Flow selections control which branches of the program are run on the data. BARCODE uses all three modules to provide 17 unique metrics to quantify the structure and dynamics of the system. Disabling any of these settings will result in a faster runtime, but fewer metrics will be calculated for each video.
The Include Dim Files and Include Dim Channels settings ensure the data is processed even if the videos are classified as dim, which is defined as videos where the minimum intensity is greater than 2/e times the mean intensity. Turning these settings on will enable the program to process these videos, but will flag them in the CSV output file as having potentially unreliable data outputs due to the low contrast through the Flag parameter as described below.
The Verbose parameter provides additional information while the data is being processed, including time step updates, total processing time, and if the input video was classified as dim.
Lastly, the Save Graphs, Save Reduced Data Structures, and Dataset Barcode settings are additional outputs besides the CSV file of the output metrics that provide more information on each of the processes performed on the data and generate a visual ID for the dataset. The Save Reduced Data Structures (RDS) setting is used to save the binarized images, optical flow fields, and intensity distributions used for the BARCODE analysis in the form of CSV files. Saving these RDS allows for further data analysis and verification.
After running the application on a file or directory, a YAML file will be created in the file directory that saves the settings selected. This file can be selected in the Configuration YAML file parameters to allow use of the same settings for future datasets.
Adjusting Parameters
This Section of the Tutorial discusses how changing parameters effects the output BARCODE and why you may or may not want to adjust these parameters.
Downsampling and Averaging
Frame Step
To increase processing speed and reduce the data size, videos can be downsampled in time by choosing to analyze every kth frame. This Frame Step parameter (k) allows the customization of the speed and precision of each branch. We have found that k = 10 provides sufficient resolution and accuracy for all branches while maintaining rapid processing times.
Percentage of Frames for analyzing changes in metrics
The frame start and end percentage sets the percentage of frames X at the beginning and end of the video used to compute the change in the various metrics. By default, these metrics compare the change in values averaged over the first 5% and last 5% of frames in the video. Increasing X can reduce noise by increasing the number of frames that contribute to the quantity, but will also reduces temporal resolution.
Animated gif of sample file used for data in this tutorial.
Binarization Settings
Binarization Threshold
The Binarization Threshold is an important parameter that determines what pixels within the image represent material, and should be set to 1, and which are background or noise, and should be set to zero. The Binarization Threshold is calculated for each frame as ((1 + Offset) * Mean Pixel Intensity). In practice, the offset should be decreased if the binarized image data is largely empty and should be increased if the binarized image contains significant noise. The effect of the threshold offset selection is demonstrated below by transforming the raw image into the following binary datasets based on various threshold values. As the threshold decreases more noise can be seen while as it increases more features are lost. The best offset value for the following examples would be 0.1 or 0.25. The offset is user-defined. By default, BARCODE uses an offset of 0.1.
Intensity Distribution Settings
Number of Distribution Bins
The Intensity Distribution branch calculates the skewness and kurtosis metrics by binning the pixel intensity values for the selected frames, and then calculating the metrics on these distributions. By default, the number of bins is set to 300. Adjusting this parameter will adjust the number/width of the bins used to create the distribution. Increasing the number of bins may more precisely capture with pixel intensity values, but may result in noisier data, while decreasing the number of bins will result in less precision in the estimation of pixel intensity counts.
Changing the Distribution Noise Threshold
The distribution noise threshold determines the minimum pixel intensity probability allowed within a single bin for that bin to be evaluated. All bins with a normalized probability below this threshold will be removed from the calculation, and the distribution renormalized to ensure that the integrated probability = 1. By default, this distribution noise threshold is set to 0.0005. Decreasing this value can result in noisier data and potential inaccuracies in the calculation of the kurtosis and skewness metrics, while increasing this value can result in a fewer pixels being counted towards the Intensity Distribution branch metrics.
(Above) Original image from a .tif file before binarization.
(Below) A series of binarized images of the original, showing increasing binarization thresholds from -0.25 to 0.25.
Optical Flow Settings
Window Size
The Window Size parameter defines the size of the region around each pixel that is used to compute the local motion within the optical flow algorithm. Larger window sizes result in less noise being captured by the algorithm, but will result in less resolved motion fields, while smaller window sizes are more capable of detecting smaller movement within the material, but will be more susceptible to random noise.
Downsampling
Depending on the scale of movement within the data, it is possible to spatially downsample the image array, which can drastically increase the throughput of the Optical Flow measurements. The extent of downsampling is set by averaging together p×p regions of pixels, which reduces the original m×n field of view to a downsampled array of m/p×n/p pixels.. Increasing p suppresses noise and improves detection of bulk movement while also reducing spatial resolution. Care should be taken in downsampling videos that capture the movement of many smaller areas of material, as these small scale motions may be lost in the averaging. This spatial downsampling also increases processing speed and reduce the RDS size. We have found p= 2-8 to provide sufficient resolution and accuracy while maintaining rapid processing times. By default, the program chooses p=8. The effect of downsampling on the flow field is shown below.
Calibration parameters
The two remaining parameters in the Optical Flow settings are the Micron-to-Pixel ratio and the Exposure Time, which are determined by the image acquisition settings. Allowable values of Micron-to-Pixel ratio range from 0.001 to 1000 microns/pixel. Allowable values of exposure Time range from 0.001 to 3600 seconds. For .nd2 files, these parameters are automatically read using the file metadata.
Optical flow field visualizations at different downsampling rates, averaging 1, 8, and 16 pixels per vector.
Combining CSV files and Generating barcodes
After the creation of the summary CSV files through the BARCODE application, these files can be manipulated and utilized without recalculating each of the metrics. To do so, in Execution Settings, select the Combine CSV Files/Generate Barcodes setting. Then, in the BARCODE Generator and CSV Aggregator, select multiple summary CSV files and combine them into a larger file generated in the specified folder. In addition, if necessary, create a new barcode for any of the input CSV files. This can be used to generate an aggregate BARCODE file from multiple datasets for comparison, or to generate a visual representation of the BARCODE CSV data. Additionally, the Metric Sort parameter selects a BARCODE metric to sort the data on, organizing the data from least to greatest along one of the 17 BARCODE metrics. If not selected, the files will be organized in the order of selection, typically by alphabetical order within the selected folder(s).
After selecting these features, start the BARCODE program. Once finished, look in the directory of the selected data where a summary file, a BARCODE Matrix PNG and a folder containing the generated graphs and data structures should be found.
Reading barcodes
After generating the barcode for a dataset, how should it be interpreted? The rows of the barcode matrix represent the different files and channels analyzed, listed in the order of execution. For a single-channel analysis, only one row will be present. The order of the output metrics is shown in the diagram below. The color of each metric is determined by its scaled value relative to its respective limits and is then plotted using the "plasma" colormap. The exact values for each metric can be found in the summary.csv file. The summary CSV file also stores a Flags parameter for any warning the code produces for a given dataset. For videos classified as “dim” (defined as videos where the first and last frames have the minimum pixel intensity value greater than or equal to 2/e times the mean pixel intensity value for the frame), the video will have a Flag value of 1, while for saturated videos (defined as the case where the mode pixel intensity value is equal to the maximum pixel intensity value for all frames), the video will have a Flag value of 2.
Two examples of BARCODE output metrics with names and variables, and limits of parameters.
FAQs
Can I analyze files that aren’t .tif or .nd2?
No, files must be converted to .tif or .nd2 to analyze with the BARCODE application. Follow Section 2 to convert your files to the correct format.
Can I analyze a directory with both .tif and .nd2 files?
Yes, the program has no issue when executing folders with multiple acceptable file types.
How can I speed up the BARCODE execution?
The easiest way to speed up the BARCODE execution is to increase the Frame Step or decrease the Percentage of Frames.
Links and Resources
Windows and Mac Download
Windows and Mac: Link
README and Source Code
README and Source Code: Link
ImageJ and FIJI Download
ImageJ:Link
FIJI:Link