Web Microanalysis of Big Image Data

Table of Contents. 1



Preface







1 Introduction. 1



1.1 What is image processing pipeline?. 1



1.2 What does web image processing pipeline consist of?. 3



1.3 What are big data microscopy experiments?. 4



1.4 Why are scientists interested in big data microscopy experiments?. 6



1.5 What is the range of applications leveraging image processing pipelines?. 9



1.6 Challenges of big data microscopy experiments. 10



1.7 Tradeoffs before and after digital images are acquired. 12



1.8 Enabling reproducible science from big data microscopy experiments. 14







2 Using Web Image Processing Pipeline for Big Data Microscopy Experiments. 1



2.1 Deploying and Testing the Web Image Processing Pipeline. 2



2.1.1 Types of deployment 4



2.1.2 Deployment of Docker Containers. 6



2.1.3 Deployment recommendations. 7



2.1.4 Test data and computational benchmarks. 8



2.2 Web Image Processing. 10



2.2.1 WIP processing functionality. 10



2.2.2 Examples of WIP usage. 12



2.3 Web Feature Extraction. 15



2.3.1 WFE processing functionality. 17



2.3.2 WFE usage. 19



2.4 Web Statistical Modeling. 21



2.4.1 WSM processing functionality. 23



2.4.2 WSM use case. 24



2.5 Summary. 25







3 Example Use Cases 1


3.1 Cell count and single cell detection. 1


3.1.1 Image processing pipeline. 2


3.1.2 Create a new image collection. 3


3.1.3 Stitching of image tiles. 4


3.1.4 Intensity scaling and pyramid building. 5


3.1.5 Image assembling. 6


3.1.6 Segmentation. 7


3.1.7 Binary image labeling. 8


3.1.8 Feature extraction and single cell detection. 8


3.1.9 Discussion. 9


3.2 Stem cell colony growth computation. 10


3.2.1 Image processing pipeline. 11


3.2.2 Colony tracking and feature extraction<. 12


3.2.3 Discussion. 13


3.3 Summary. 15





4 Building Web Image Processing Pipeline for Big Images. 1



4.1 Mapping functionality to information technologies. 1



4.2 The role of each technology in the client-server architecture. 5



4.3 Basics of web servers. 7



4.4 Communication protocols in client-server architectures. 8



4.4.1 Client-server communication using Hypertext Transfer Protocol 9



4.4.2 Client-server communication using Secure Hypertext Transfer Protocol 11



4.4.3 Web server side Transmission Control Protocol 12



4.4.4 Web server side Message Passing Interface. 12



4.4.5 Web server side Network File System.. 14



4.5 Designing interactive user interfaces in web browsers. 14



4.5.1 Design pattern for code running in web browsers. 14



4.5.2 Dynamic web applications. 15



4.6 Large image visualization and processing in web browsers. 18



4.6.1 Representation of large images. 18 <

4.6.2 Large image visualization in web browsers. 21



4.6.3 Image processing in web browsers. 22



4.7 Managing images, pyramids and metadata on a web server 24



4.7.1 Relational databases. 25



4.7.2 Non-relational database. 27



4.7.3 Web application frameworks. 30



4.8 Meeting computational requirements on a web server 33



4.8.1 Pegasus workflow management system.. 33

4.8.2 HTCondor workload management system.. 36



4.8.3 XML file representation for encoding computational jobs. 36



4.9 Delivering traceable computations. 37



4.9.1 Components for delivering traceable computations. 38



4.9.2 Traceable computations for publications. 39



4.9.3 From traceable to reproducible computations. 41



4.10 Summary. 41







5 Image Processing Algorithms 1


5.1 Image processing. 2


5.1.1 Textbooks about image processing. 2


5.1.2 Usage-based classification of image processing implementations. 3


5.1.3 Classification of open source image processing software. 5


5.1.4 Loading images using OME Bio-Formats library. 7


5.1.5 Basic image processing using ImageJ/Fiji 9


5.2 Overview of algorithms in WIPP. 11


5.3 Image correction algorithms. 13


5.3.1 Dark current correction. 14


5.3.2 Flat field correction. 14<


5.3.3 Background correction. 15


5.3.4 Noise filtering. 19


5.4 Algorithms for stitching and mosaicking many images. 22


5.4.1 Image stitching. 23


5.4.2 Image mosaicking. 27


5.4.3 Practical Remarks. 28


5.5 Object segmentation, tracking and feature extraction algorithms. 29


5.5.1 Object segmentation. 30


5.5.2 Object tracking over time. 39


5.5.3 Image and object feature extractions. 42


5.6 Image intensity scaling and pyramid building algorithms. 44


5.6.1 Image intensity scaling. 44


5.6.2 Image pyramid building. 46


5.6.3 Reprojection of a pyramid set 48


5.7 Summary. 51









6 Interoperability Between Software and Hardware. 1



6.1 Hardware options for accelerating computations. 2



6.2 Implications of big data attributes. 4



6.3 Execution times of computation over big image data. 6



6.3.1 Meeting execution time requirements. 7



6.3.2 Estimating and measuring execution time. 9



6.4 From commercial big data analytics to research big image analyses. 10



6.5 Human interfaces for big image data analytics. 12



6.5.1 Focus on client-side graphical user interfaces. 13



6.5.2 Example of GUI design for web statistical modeling tool 14



6.5.3 Summary. 16



6.6 Storage and data structure for big images. 16



6.6.1 Storage for big images. 17



6.6.2 Data structures for big images. 22



6.6.3 Summary. 23



6.7 Parallel computations over big image data. 23



6.7.1 Data parallel model 24



6.7.2 Master-agent model 26



6.7.3 Task graph model 28



6.7.4 Task pool model 29



6.7.5 Consumer-producer model 30



6.7.6 Hybrid model 32



6.7.7 Summary. 32







7 Supplementary Information. 1



7.1 Software and documentation. 1



7.2 Data for testing software installation. 2



7.3 Deployed demonstrations on the web. 2



8 Abbreviations. 3



9 Terminology. 4



10 Acknowledgements. 5



11 Disclaimer. 6



12 Summary of References. 6

Este título pertence ao(s) assunto(s) indicados(s). Para ver outros títulos clique no assunto desejado.

• Matemática aplicada
• Sistemas de imagem e tecnologia
• Reconhecimento de padrões

Microscopy imaging;Image analyses;Image stitching;Image segmentation;Image tracking;Image feature extraction;Statistical modeling;Big data;Web image processing;Computational acceleration;Traceability and scientific reproducibility;Microscopy in cell biology;Microscopy in materials science