A multi-stage weakly supervised design for spheroid segmentation to explore mesenchymal stem cell differentiation dynamics

Abstract

There is a growing interest in utilizing 3D culture models for stem cell and cancer cell research due to their closer resemblance to in vivo environments. In this study, human mesenchymal stem cells (MSCs) were cultured using adipocytes and osteocytes as differentiative mediums on varying concentrations of chitosan substrate. Light microscopy was employed to capture cell images from the first day to the 21st day of differentiation. Accurate image segmentation is crucial for analyzing the morphological features of the spheroids during the experimental period and for understanding MSC differentiation dynamics for therapeutic applications. Therefore, we developed an innovative, weakly supervised model, aided by convolutional neural networks, to perform label-free spheroid segmentation. Since obtaining pixel-level ground truth labels through manual annotation is labor-intensive, our approach improves the overall quality of the ground-truth map by incorporating a multi-stage process within a weakly supervised learning framework. Additionally, we developed a robust learning scheme for spheroid detection, providing a reliable foundation to study MSC differentiation dynamics. The proposed framework was systematically evaluated using low-resolution microscopic data and challenging, noisy backgrounds. The experimental results demonstrate the effectiveness of our segmentation approach in accurately separating the spheroid from the background. Furthermore, it achieves performance comparable to fully supervised state-of-the-art approaches. To quantitatively evaluate our algorithm, extensive experiments were conducted using available annotated data, confirming the reliability and robustness of our method. Our computationally extracted features can confirm the experimental results regarding alterations in MSC viability, attachment, and differentiation dynamics among the substrates with three concentrations of chitosan used. We observed the formation of more compact spheroids with higher solidity and convex area, resulting improved cell attachment and viability on the 2% chitosan substrate. Additionally, this substrate exhibited a higher propensity for differentiation into osteocytes, as evidenced by the formation of smaller and more ellipsoid spheroids.

Highlights

• "Chitosan biofilms mimic in vivo environments for stem cell culture, advancing therapeutic and fundamental applications.”

• "Innovative weakly supervised model enables label-free spheroid segmentation in stem cell differentiation studies.”

• "Robust learning scheme achieves accurate spheroid separation, comparable to state-of-the-art approaches.”

Introduction

Improving in vitro models to study cell biology is a significant advancement applicable across various fields, including biotechnology, cancer research, drug discovery, and the emerging fields of tissue engineering and regenerative medicine. Traditional two-dimensional (2D) plastic as support for mammalian cell culture has several limitations. Recent studies have shown that cells cultured in a three-dimensional (3D) environment more closely represent natural cellular functions due to increased cell–cell interactions as well as extracellular matrix (ECM) interactions, effectively by mimicking the in vivo architecture of natural organs and tissues [1].

The lack of cell–cell interaction due to the insufficient mutual contact space in 2D cultures has led to the belief that the future supporting framework for cell culturing should more accurately mimic the in vivo environment [2]. Generally, 3D cell culture is defined as culturing cells on 3D scaffolds made from natural or synthetic materials, or as the organization of cells organized into cellular spheroids. With recent advancements in 3D cell culture have led to a consensus that plastic 2D culture plates are inadequate as a reliable structures for behavioral or physiological scientific research [3, 4]. The efficiency and toxicity of potential substances and drugs can be more accurately predicted using spheroid cell cultures. These spheroids have been extensively tested in various applications, including drug screening, tissue engineering, regenerative medicine, and other medical studies [5, 6].

Spheroids create a 3D in vivo microenvironment by facilitating advanced and complex interactions between cells their extracellular matrix (ECM). These interactions spontaneously deliver essential signals, including mechanical forces and biochemical signals, which influence cell shape, proliferation, differentiation, and gene expression [4, 7, 8]. The physical properties of scaffolds, such as stiffness and porosity, play a crucial role in 3D stem cell differentiation while also providing the necessary space for cells to proliferate and differentiate [9, 10].

The dynamics of differentiation in multipotent stem cells, especially in mesenchymal stem cells, represent a complex, multi-scale process. The expression levels of genes associated with cell adhesion and immune response change when the culture environment shifts from 2 to 3D [11]. Numerous studies have demonstrated that cells cultured in 3D exhibit enhanced differentiation, greater self-renewal potential, and increased secretion of paracrine factor [6, 12,13,14]. Stem cell transplantation in the form of spheroids is considered one of the most promising strategies in regenerative medicine to achieve the functional recovery of damaged tissues or organs. It is safe to assert that 3D spheroid cultures serve as practical and convenient platforms for elucidating cell–cell and cell-scaffold interactions, as well as providing valuable models for cancer research [15, 16].

Chitosan (CS) is a natural, semi-crystalline, cationic polysaccharide known for its biocompatibility and structural similarity to glycosaminoglycans. 3D culture or spheroid formation platforms can be divided into two groups: suspension (scaffold-free) and scaffold-based platforms [17]. Numerous studies have used chitosan as a scaffold for differentiation and cell culture, aiming to mimic the environment of the tissue due to its favorable properties, including durability, cost-effectiveness, biodegradability, and antibacterial properties. The availability of chitosan, combined with its exceptional qualities for cultivating stem cells, established it as a prominent biopolymer in the field of regenerative medicine [5, 8, 18,19,20]. The concentrations of a biopolymer is directly correlated with its surface roughness and stiffness. An increase in surface roughness, accompanied by a decrease in its stiffness, may enhance cell migration, suggesting that the substrate interface significantly influences cell proliferation and differentiation [21].

The chitosan membrane promotes and triggers the self-assembly of cultured cells into multicellular spheroids, providing a 3D environment for lineage-specific differentiation. Chitosan-based substrates are distinctive in scaffold-based spheroid formation techniques due to their transient attachment of cells to the surface, their ability to form spheroids within the first day of culture, and their ability to be later floated easily [3, 20, 22]. Differentiated cells, such as fibroblasts, vascular smooth muscle cells (VSMCs), chondrocytes, neurons, and mesenchymal stem cells, exhibit micron-sized focal adhesions interconnected by actin fibers when cultured on rigid substrates. However, these adhesion structures gradually diminish when cells are grown on softer matrices, which can be achieved by modifying the gel's crosslinking density [21, 23,24,25,26].

The standard method for spheroid analysis involves utilizing confocal laser scanning microscopy, which necessitates costly laboratory setups and is often a time-consuming process. Conventional fluorescent microscopy imaging techniques are also employed; however, they provide less detailed images due to a significant amount of unfocused background noise. In studies focusing on the dynamics of differentiation, time-lapsed photos may not be a feasible method [23, 24, 27]. Alternatively, light microscopy imaging might be used. Nevertheless, despite the lower image resolution, computational image processing techniques could compensate for this limitation. Consequently, the analysis of the topographic shapes and parameters of the spheroids promises to yield valuable insights. This information will help us to calibrate and adjust various parameters during the proliferation and differentiation processes. Importantly, this methodology circumvents the need for costly tools, resulting in considerable time and energy savings.

Segmenting the spheroids presents a significant challenge in processing these images, as it involves the task of separating the objects in the image from the background. While this process may seem straightforward, it is nonetheless critical. Various techniques, including machine learning methods, have been developed for image segmentation, especially to process biomedical images [25, 26, 28]. Although recent machine learning techniques like convolutional neural networks (CNNs) have achieved successful segmentation results when applied to large, labeled benchmark datasets, their performance adaptation to biomedical data has been constrained by the limited availability of ground truth labels.

This paper presents a novel multi-stage, weakly supervised segmentation method that eliminates the need for human annotation. Our approach requires only raw laboratory images, and minimal human eye inspection to perform segmentation on a large dataset. The key insight involves generating weak but automated annotation masks for a small portion of the data using traditional unsupervised segmentation models. These initial masks are subsequently enhanced in two steps: first by a shallow, then a deeper U-net like CNN [26]. The improvements achieved in these stages pertain to both the quantity and quality of the masks.

In this work, we present a multi-stage computational procedure designed to extract morphological information from daily images of cells undergoing differentiation. For this purpose, we cultured bone marrow-derived mesenchymal stem cells on three different concentrations of medium-sized chitosan-based membranes and subsequently differentiated them into adipocytes and osteocytes as self-developed 3D spheroids. This approach allowed us to investigate the effect of the polymer concentration on the cells’ fate. To develop a practical and cost-effective approach to study the dynamics of differentiation, light microscopy images of the spheroids daily throughout the differentiation process. These images were then processed using a self-developed artificial intelligence (AI)-based technique. Our in-house developed algorithm extracted a wide range of information based on change in size, shape, and various properties of spheroids formed on our biopolymer films (Fig. 1).

Fig. 1

Graphical Abstract. Overview of the project framework, including the procedures for spheroid generation, differentiation, and image acquisition in the wet lab (A), as well as segmentation and image processing in the AI lab (B). Schematic representation of results from both biological and computational investigations (C). These results are presented separately in the article or supplementary file

To the best of our knowledge, this is the first study on MSC differentiation utilizing on light microscopic images of spheroids aided by AI techniques. This research employs advanced multi-stage learning techniques for detecting and segmenting of spheroids from low-resolution microscopic images without any manual annotation. Using such an accurate segmentation technique, the feature analysis of spheroids will be more reliable results. Overall, our contributions can be summarized as follows:

• 1. Our investigation of stem cell dynamics during differentiation utilized light microscopy. The affordability and accessibility of light microscopic imaging confer a notable advantage for this research.

• 2. It has been proven that various concentrations of chitosan affect the quality of cell attachment, spheroid formation, and differentiation by changing the structure of the chitosan film and its mechanical properties. However, the effects of parameter changes, specifically variations in chitosan concentration over time, has not been thoroughly investigated. Therefore, three concentrations of medium-sized chitosan 0.5%, 1%, and 2% were employed to investigate spheroid dynamics during differentiation.

• 3. We propose a high-performance and robust segmentation framework that does not rely on human annotation. This framework integrates the advantages of classical unsupervised segmentation approaches (i.e., high speed and convenience) with the advantages of weakly supervised deep learning models (i.e., performance-boosting).

• 4. We created a unique database of annotated light microscopic images of spheroids, which can be used to develop deep learning-based segmentation models and study MSCD dynamics.

• 5. Through extensive experiments conducted on both our collected data and a benchmark annotated dataset, we demonstrated that our proposed segmentation algorithm can reliably detect and segment spheroids, achieving performance comparable performance to fully supervised state-of-the-art approaches.

Methodology

Cell culture

Human bone marrow-derived mesenchymal stem cells (hMSCs) were purchased from the Royan Stem Cell Institute. This cell belongs to a 4-year-old boy (passage 3). Cells were grown and passaged twice in DMEM-f12 (Gibco) containing 10% fetal bovine serum (FBS, Gibco), 10 ng /mL FGF-2 (Miltenyi Biotec), 1 mM l-glutamine (Gibco), 1% nonessential amino acids (Gibco), and 100 U/mL penicillin/streptomycin (Gibco) with daily medium exchange. Cells were seeded at 25,000 cells/cm² in 24-well plates (SPL, Korea Republic) and treated with the same DMEM/F12 for 48 h for proliferation and stress reduction. The same medium (toward adipocytes and osteocytes) was used throughout the experiment to maintain constant differentiation conditions.

Differentiation approach

The differentiation of hMSCs into adipocytes and osteocytes was conducted within 21 days (including the first 2 days of proliferation). The adipogenic medium used in the study was prepared using DMEM/F12 complete medium (DMEM/F12 + 10% FBS + 2 mM L-glutamine) enriched and supplemented with 1 μM dexamethasone (Sigma, D4902), 0.5 mM isobutyl methylxanthine (IBMX Serva, 26,455.02) 60 μM of indomethacin (Sigma, 17,378) and 0.5 μM hydrocortisone. The osteogenic medium was the same DMEM/F12 (+ 10% FBS + 2 mM L-glutamine) enriched and supplemented with 10 nM dexamethasone (Sigma, D4902), 50 µM ascorbate-2-phosphate, and 10 mM ß-glycerophosphate [22, 29].

Relative gene expression by real-time PCR

To assess the quality of differentiation in our 3D cell spheroids, total RNA was extracted from the cells at three different time points: days 7, 14, and 21. We collected RNA samples from cells cultured with varying concentrations of chitosan (0.5%, 1%, and 2%) for both adipocyte and osteocyte differentiation pathways. RNA was extracted from the spheroids in 2D culture using milli-columns and an extraction kit, following the manufacturer’s protocol (Novin-Zist Co., Tehran, Iran). The concentration, quantity, and quality of the RNA were determined using a NanoDrop 2000c spectrophotometer.

The extracted mRNA was used to synthesize cDNA by adding the appropriate amount of RT-master mix and nuclease-free water, following the manufacturer's protocol (SMOBIO Technology, Inc., Hsinchu, Taiwan). Primers were designed using the online NCBI Primer-BLAST tool, as detailed in Table S1. Real-time PCR was subsequently performed using the synthesized cDNA, working solutions of relative primers (10 μM) (Sina Clone, Tehran, Iran), a SYBR Green master mix, and the required amount of RNase-free water from the real-time PCR kit, following the manufacturer's instructions (SMOBIO Technology, Inc., Hsinchu, Taiwan). The thermal cycling program for the real-time PCR included an initial denaturation step at 95 °C for 5 min, followed by 40 cycles of denaturation at 95 °C for 30 s, annealing at 60 °C for 30 s, and extension at 72 °C for 30 s, with a final extension at 72 °C for 5 min.

The ΔCt (Delta Ct) values for each gene from all the samples were calculated based on the expression of Beta-2-microglubine (β2M) as our housekeeping gene. The relative mRNA gene expression was then determined using the 2-ΔΔCt method for each gene at days 7, 14, and 21 of differentiation, considering the different concentrations of the chitosan scaffold [30]. To quantify adipose differentiation, we measured the relative expression of PPARγ and lipoprotein lipase (LPL) mRNAs, which are well-established indicators of adipocyte maturity factors [31,32,33]. Additionally, for the osteocyte differentiation pathway, we measured the relative expression of alkaline phosphatase (ALP), the most recognized factor involved in bone differentiation studies [34, 35].

Preparation of chitosan films

Three concentrations of chitosan films were prepared according to the method previously described by Lin [36] and our earlier work [15]. Briefly, to obtain chitosan solutions at concentrations of 0.5, 1, and 2 wt% chitosan powder (Sigma-Aldrich) of a specific molecular weight was dissolved in a 2 wt.% v/v aqueous solution of acetic acid (Sigma-Aldrich) by subjecting the sealed mixture to magnetic stirring for 48 h at room temperature. Floating undissolved microparticles removed by centrifugation for 20 min at 5000 rpm and 4 °C, followed by filtration of the supernatant using a sintered glass funnel with a G0 degree of porosity. Subsequently, the filtered chitosan solution was poured into culture plates and incubated overnight at 45 °C for evaporation of the solvent and chitosan film formation. The residual acetic acid on the chitosan films was neutralized using a 0.5 M NaOH solution of for 2 h, followed by multiple washings of the films with deionized distilled water. The plates were then sterilized for 30 min under direct UV-C light (6W, ZW6S15Y instrument, Foshan Bolien Co., China) [15].

Cell viability and cell attachment comparison

The biocompatibility of chitosan scaffolds, as a function of cell viability, was determined using an MTT cell viability assay after 24, 48, and 72 h of cell culture. For this experiment, cells were cultured in three 48-well plates, and the standard MTT assay protocol was followed [17]. The absorbance of the samples was measured at 570 nm using a Cytation3™ microplate reader (BioTek, Vermont, USA). Moreover, to compare the attachment of hMSC cells to various concentrations of chitosan, a Hoechst 33,342 live cell staining assay was performed. Cells were cultured in two 48-well plates and incubated for 4 and 24 h. After 20 min of incubation, the cells were washed twice with PBS, and the excitation and emission of the plate were measured using the Cytation 3 microplate reader [37].

Data acquisition

To collect morphological data from our generated spheroid on three different concentrations of chitosan scaffolds, for each differentiation pathway, the cells were seeded on 48-well plates already coated with the chitosan biofilm using the procedure explained above. The first 12 wells of the plate were used as the control (without any chitosan coating). The second 12-well group was coated with 0.5% chitosan, and the next 12 well groups were coated with 1% or 2% chitosan scaffolds. After seeding, 48 h of proliferation was provided to avoid splitting stress on the cells. The following day, we acquired getting microscopic images every 24 h and continued to do so until the day 21. The first half of each plate was treated with a fat differentiation medium, and the other half was treated with osteogenesis differentiation medium.

Similarly, in each differentiation pathway for each chitosan concentration (0.5, 1, and 2%), at least 6 photos were taken daily (6 repeats for each condition). We did so to normalize the images statistically and increase their credibility. All microscopic images in this study were taken with the same objective (10X magnification) using the same setup of cell sense dimension life science imaging software on an Olympus IX81 inverted microscope (Olympus; Hamburg, Germany).

Image processing

Morphological analysis of light microscopic images consists of two main parts: spheroid segmentation and feature extraction from the segmented spheroids. The following sections provide a detailed explanation of these parts. It is important to note that the experiments have been performed using TensorFlow 2.4 and Keras libraries in Python 3.5 running on a desktop workstation equipped with an NVIDIA 4 GB GeForce GT 730 GPU.

Spheroids segmentation

The long-standing challenge of image segmentation primarily focuses on localizing specific objects within an image. The ideal outcome is the generation of pixel-level labeled masks, where each pixel is classified as belonging to specific regions [38,39,40]. This technique has a wide range of applications in various scientific fields, including autonomous vehicles [38], satellite image analysis [41], medical imaging [42] etc. Precise image segmentation is vital in our morphology-based feature analysis of spheroids during the experimental periods. Traditional segmentation approaches, like those used by Deckers et al. 2018 [43] failed to follow the gradual changes happening in our spheroids during the experiment period because several parameters needed to be manually set in these models.

Figure 2 presents a sample of four images from our diverse dataset, marking the targeted objects highlighted by circles. Discrimination between the spheroids, background, and noises is not a straightforward classical image processing problem to be addressed by features like the object’s size, contrast with the background, edges, etc. This raises the need to hire an intelligent framework containing the least hyperparameters to be set manually.

Fig. 2

(a–d) Shows four random examples of raw laboratory data. Circles mark the interesting spheroids to be found

Proposed approach

Regarding of AI-based semantic segmentation approaches, CNNs are specifically designed for complex data analysis and extracting semantic-level features [44,45,46]. However, these models require a substantial annotated data for effective training. Image segmentation could be approached through various combinations of supervised and unsupervised ways. In a study performed by Baby et al. in 2020 [47], a comparative analysis of the performance between supervised and unsupervised image segmentation techniques was explored. For our mission, a multi-stage procedure is preferred to ensure reliable results on our unlabeled data. To train a CNN capable of labling all images effectively, a controlled dataset comprised of well-annotated images is necessary first. Benchmark datasets are typically annotated manually, which is a labor-intensive and resource-draining process. This challenge becomes particularly pronounced when labeling a substantial number of images, often making it seem unfeasible. ​To tackle this issue, we propose a design aimed at creating a large labeled dataset from raw laboratory images.

In this work, as illustrated in Fig. 3, We propose a multi-stage design in which the initial step involves labeling portions of the images using traditional image processing methods. In other words, the usual procedure of manually painting the objects of images is automated. Although this process does not achieve perfect results, it is sufficiently effective to serve as an imperfect source of annotated images, which are then fed into a weakly supervised procedure. The next step uses the chosen masks in a semi-automated process. In other words, pruning the outcome of the first step requires human-eye inspection. To accelerate pruning, we have proposed an anomaly mask detection procedure that filters candidates for corrupted masks for visual inspection. Figure 4 illustrates the process of spheroid labeling at different steps, whereas Fig. 4(b) shows the weakly-labeled example resulting from the first step. After preparing of small but cleaned annotated data (530 images out of ~ 1.8 k input images in our case), an initial shallow CNN is used to generalize and enhance the performance of the first-step outputs, and not for generating the desired dataset for feature extraction. We found that the simple shallow CNN mostly boosted the previous results. Figure 4(c) shows the results obtained in this step. The output of the second step must to be filtered as described above. Then, for the final step, approximately 1 K well-annotated data are ready to train a deep CNN designed for precise semantic segmentation. Detailed descriptions of different steps in the proposed framework are given in the following.

Fig. 3

A summarized schematic of the segmentation workflow. It consists of three main steps; step 1 produces non-perfect masks for a part of raw data. After visual inspection and pruning, the remaining masks are fed to an initial shallow CNN via weakly supervised training in step 2, just to improve the masking performance. In a multi-stage procedure, these steps are repeated once again by a deeper CNN to follow more abstract features from the inputs to produce the final masks

Fig. 4

An example of the segmentation procedure through all three steps. a Laboratory image. The candidate spheroids are marked by circles. The yellow circle shows a lost spheroid in the first step of traditional image processing. b The traditional image processing output. c The shallow CNN result. (In comparison to the first step, one more spheroid is detected.) d The final mask by deeper CNN (in comparison to the second step, the modification on the spheroid edges is seen in some cases)

First step: traditional image processing methods to obtain initial masks

To prepare an initial imperfect mask dataset, a combination of image thresholding analogous to the hired method in [43] and ballistic modification is used. Initially, the Robert gradient was applied, followed by Otsu thresholding on the gray-scaled images. Otsu thresholding is commonly employed to differentiate foreground objects from the background [1]. This method is effective for segmenting individual images by adjusting a few parameters, but it is not suitable for large datasets. The results from this approach on our data primarily include spheroid areas that did not meet the threshold criteria.

To address this limitation, we implemented our ballistic modifier, which identifies sparsely distributed OFF pixels (indicating the background) within densely packed ON pixels (representing the objects) and compensates for them. Additionally, the modifier eliminates noise by turning off sparse pixels in the background. Following these compensation and noise removal steps, the edges of the spheroid areas were detected using the Canny operator [2] and then filled to produce binary masks. This step was applied to less than half of all our raw images (about 900 images) since the corresponding output requires time-consuming inspection. As a result, we ought to be working with a small portion of all the data. To bypass this challenge and enhance the results of this step, we might use a compact model with a limited number of parameters. Therefore, prior to proceeding to the final step, we leverage an intermediate shallow CNN.

Second Step: Shallow CNN to modify annotation performance

The output masks from the first step were not as accurate as expected for the target analysis. This dataset still necessitates certain adjustments to train a sophisticated CNN capable of extracting semantic information. To achieve this, we utilized a small annotated dataset to train a compact CNN. Although deeper neural networks are more adept at handling intricate data, they require a larger volume of training data to ensure effective training. The level of abstract features in our dataset is far less than that of real-life datasets like CFAR10 [48]. Hence, for such data, a relatively simpler CNN can easily perform segmentation tasks for an acceptable portion of images.

As illustrated in Fig. 5, there are two parallel calculation branches in this part of our model. The main branch (lower branch) is a shallow CNN that includes one convolutional layer followed by max-pooling and another convolutional layer followed by an up-sampling layer, which aims at feature extraction. The other branch (upper branch) has only one convolutional block, and its output is concatenated to that of the first branch. This branch aims to remind the spatial information to the output for better configuration regeneration [26]. Based on our preliminary experience, we concluded that this direct branch (skip connection) with minimal convolutions would significantly help localizing objects while it minimizes the computational cost for the model by reducing the need for more convolutional layers.

Fig. 5

A shallow CNN is used in the second workflow step. With a minimal number of layers and parameters, it contains a single pooling level paralleled by a single skip connection. The skip connection is used to remind the spatial features of the input to the output masks

After weekly training this CNN on ~ 25% () of all the data and extrapolating on all the images, including the training set, for the second time, human inspection is needed to remove corrupted masks from the results. It is worth noting that due to the nature of employing a weak training process, the model inherently avoids overfitting. Therefore, in this scenario, utilizing the training set for extrapolation can not be interpreted within the framework of conventional supervised processes. This is elaborated in the following sections.

Third step: deeper CNN to obtain final masks

Following the completion of the preceding two steps, there is now a satisfactory quantity of masked images (approximately 1 k) prepared to be input into the U-net-like CNN, which is designed to accurately annotate the spheroids. Although the performance of the shallow CNN was acceptable on a substantial portion of images, it was still a simple model that could not follow all the gradual changes in images throughout the experimental period. Hence, a deeper CNN was required to successfully generalize our workflow for various kinds of data. In Fig. 6, an illustration of this model is shown, which is a more complex version of the previous CNN but still simpler than the original U-net [26]. The details are the same as discussed in the previous step.

Fig. 6

Schema of deeper CNN used in the third step of the workflow

Notes on the optimization process and weakly supervised learning

To optimize the CNNs, we employ the Adam optimizer with a gradually decreasing learning rate, starting from 10 and ending at 1. We also propose to use the Dice loss function as a successful metric for medical segmentation assessment [49, 50]. Dice loss is a robust and suitable deep learning loss function, especially for unbalanced classes.

All training procedures conducted in this study are performed using weakly supervised learning, which provides a practical solution when acquiring accurate ground-truth labels is difficult due to the high costs associated with data labeling.

In this study, we regulated the weak learning process by implementing rigorous regularization techniques during training. Generally, in deep learning, regularization serves to strike a balance between overfitting and underfitting [51]. We have implemented two distinct strategies to control the regularizations effectively. First, we applied relatively high values for the L2 regularizers, set between 0.05 and 0.5 for different scenarios. Second, we incorporated a 40 percent dropout rate after each convolutional layer. ​These regularization techniques ensure that the model does not converge perfectly during the training process, thereby promoting weaker learning and enhancing generalization. In a weakly supervised process, the model should not fully adapt to the input due to imperfect annotations and high levels of noise present in the training data. Similar to an underfitted model, a weakly supervised model can identify patterns within the training data. Consequently, the weakly trained model cannot simply replicate its training set; however, this characteristic would benefit to surpass the imperfect input data [51]. An illustration of the quality enhancement achieved by our model, facilitated by weakly supervised shallow and deeper CNNs compared to their inputs, is depicted in Fig. 4. Further results are presented in subsequent sections.

Anomaly detection for filtering corrupted images

To speed up and automate the inspection process as much as possible, we defined some characteristics of candidate anomalies and designed a tool to filter corrupted candidate masks. The proposed tool mostly takes advantage of statistical features and distribution of ON/OFF pixels in the masks.

The simplest rule is to find the empty masks by counting “ON or OFF” pixels and comparing it to a threshold, let’s say 500 pixels, considering too small objects as fooled spheroids. Other anomaly candidates are images containing a high number of recognized objects (arising from sample pollution), too large area of recognized objects (arising from burned areas of images), or objects with large perimeters that are far from the normal distribution (arising from imperfect recognized spheroids with redundant edges). According to this list of criteria, for example, in the last step, about 400 candidate corrupted masks were filtered to be removed or kept after being checked by the human eye, instead of inspecting ~ 1.8 K image-mask pairs in total.

Feature extraction

To study MSCD dynamics, the morphological parameters of segmented spheroids should be quantified and analyzed during the experimental periods. The following features are described in detail:

Convex Area: In a picture, an area is defined by the number of pixels, and the convex area of a spheroid refers to the area of the convex hull that encloses the object. The convex area of the spheroid would be measured and used to define and measure deeper and more complex features of the spheroid, such as its solidity. Additionally, the convex perimeter of a spheroid is defined as the perimeter of the convex hull that surrounds the spheroid.

Spheroid Compactness: The compactness of a spheroid is defined as the ratio of the area of the spheroid to the area of a circle with the same perimeter. The rounder the borders of the spheroid that fit within the mentioned circle (the most compact shape), the closer this ratio approaches one. This ratio could be a useful representation of the quality of cell–cell and cell-surface interactions. As the number of pseudo-shape particles within the spheroids increases, the spheroid’s compactness decreases.

Spheroid Solidity: Spheroid solidity quantifies the density of a cellular spheroid, by comparing the area of the spheroid to the area of its convex hull. A solidity value of 1 indicates a solid spheroid, while values less than 1 suggest irregular boundaries or perforations within the structure. A decrease in insolidity corresponds to more dendritic-like boundaries of the spheroid. This metric can elucidate the interaction between the 3D spheroid and its scaffold.

Spheroids Eccentricity: The eccentricity of a spheroid is defined as the ratio of its minor axis length to its major axis length. This value ranges from 0 to 1, where a higher value indicaties a more elongated shape, while a value closer to 0 suggests a more rounded shape. The term "ellipticity" is often used interchangeably with spheroid eccentricity.

Results

MTT assay

The viability of the cultured cells at various concentrations of chitosan scaffolds was measured during the first 72 h of culturing MSCs on the polymers. The data extracted from the viable cells were highly correlated with the data extracted from our in-house developed image processing algorithm. As shown in figure S1, the number of cells that survived on 2% chitosan in the first 24 h was almost two times greater than that on 2D PET plastic culture plates. Analysis of Variance (ANOVA) of data from multiple test repetitions also supported the data extracted from processing the images.

Hoechst assay

The attachment of the cells to different concentrations of biopolymer scaffolds was investigated using the fluorescent Hoechst 33,342 assay. As illustrated in figure S2, the number of cells after the first four hours of culture on the scaffold surface was twofold higher compared to other concentrations. This was highly correlated with the values measured in the MTT assay. ANOVA of repeated experiments also confirmed the superior performance of the 2% scaffold concentration.

Imaging through differentiation

To develop and train our in-house image processing algorithms, we prepared six identical controls for each concentration of chitosan biofilm (0.5, 1, and 2%) for both differentiation pathways (adipogenesis and osteogenesis) following the self-formation of spheroids on chitosan films. The spheroids were segmented from the images using recognition methods, and the presented dataset was extracted from the whole dataset.

In this study, 2D images of the spheroids were captured daily. The spheroids were segmented from the raw images after running six copies of each condition per day. Subsequently, the primary data obtained from the segmentation and various morphological factors of the spheroids were statistically analyzed. The following graphs represent the effects of basal scaffolds on cell aggregate-substrate interactions, the quality of differentiation toward adipocytes and osteocytes, and also visualize the effect of the physical stiffness of different scaffolds on these differentiation pathways.

Our algorithm provides a rapid, morphology-based feature analysis of spheroids. Subsequently, we compare the differentiation and differentiation potential of various concentrations of our 3D scaffolds based on the extracted data. Furthermore, regardless of cell type and environmental situation, our setup can be used for diverse spheroid analysis applications, including drug discovery and fundamental cell biology research. This innovative tool has the potential to enhance cancer drug delivery studies and investigations that focus on the effects of external or internal factors influencing 3D cell cultures or mimicking the physiological environment of solid tumors [52].

AI-based image segmentation

This section provides qualitative evaluations of the proposed segmentation algorithm based on our collected data. Apart from our label-free database, we conduct extensive experiments on an available annotated data to quantitatively assess the segmentation approach and be able to compare the method to other state-of-the-art approaches. Additionally, to quantitatively demonstrate the effectiveness of our algorithm specially in segmenting our low-resolution database when compared to other weakly supervised approaches, we manually annotated several images from our collected dataset. We then compared the results of our algorithm with those produced by existing segmentation tools in this field. ​However, since we wish to maintain the reader's focus on the primary objective of the paper—studying the dynamics of MSCs in spheroids over a specified time period—we have included the technical details of our results in the supplementary materials (figures S4-S8, and tables S2-S3).

Note that the total number of training images for the final step is 1500 (out of 1800 original images), while the total number of images for testing and evaluation in this final step is 21.​ Given the absence of ground truth data to report any quantitative results, retaining a large dataset for testing or validation is unnecessary. The limited amount of data we are using is solely intended to verify that the regularizers are functioning properly.

Figure 7 and Fig. 8 depict a visual comparison between the results obtained from different stages of our proposed segmentation algorithm. The figures demonstrate the progressive improvement in capturing challenging details from (b) to (c), and the enhanced comparability of edges to the raw image from (c) to (d).

Fig. 7

An example of the multi-stage improvement in masking raw input images: (a) laboratory image: (b) traditional image processing steps output as an initial mask: (c) shallow CNN boosted output: (d) final output of deep CNN (It is shown that from b to c, more challenging details are captured and from c to d, edges are more comparative to the raw image)

Fig. 8

An example of the multi-stage improvement in masking raw input images: (a) laboratory image: (b) traditional image processing steps output as an initial mask: (c) shallow CNN boosted output: (d) final output of deep CNN (It is shown that from b to c, more challenging details are captured and from c to d, edges are more comparative to the raw image)

Feature analysis

The extracted data sets from the deep learning algorithm reveal valuable information about the cellular spheroids grown on different concentrations of chitosan scaffolds. The statistical graphs are all scaled between 0 and 1, to reach a clear comparison between the two different graphs regarding a specific path. Every pair graph compares a specific morphological property regarding the osteocyte and adipocyte pathways. separately over 21 days for three different chitosan densities.

Convex area

Based on the convex area graphs displayed in Fig. 9, According to the convex area graphs displayed in Fig. 9, the trends observed for the two distinct differentiation pathways diverge. However, certain similarities can still be discerned in these graphs. In the case of osteogenic differentiation, the total convex area of spheroids in 2% chitosan biofilms is nearly double that of the other concentrations. Conversely, in the adipogenesis pathway, the convex areas for the three chitosan concentrations remained relatively consistent.

Fig. 9

Violin plots with emphasized mean values for comparison between spheroids convex areas regarding the a osteocyte and b adipocyte pathways, during a 21-days period for three different densities of chitosan. red: 0.5, blue: 1, and green: 2

The trend for the two different differentiation directions (adipocytes and osteocytes) throughout the first half of the differentiation period (the first 10–12 days) was not the same. MSCD into osteocytes shows even a decrease in the number of cells, which could mean that cells are already turning into pre-osteocytes and have stopped proliferating. On the other hand, in the adipogenesis pathway, the trend is increasing until a similar time point and then decreases dramatically.

Comparing the distribution graphs as shown in Fig. 9 reveals that in the adipogenesis pathway, the spheroids’ maximum convex areas are mostly larger than the other spheroids. Generally, we can conclude that the adipocyte spheroids are bigger. The mean convex area graphs in the osteogenesis pathway show that for density 2, giant spheroids are constructed. But between days 4 and 12, the mean area will be reduced reduced, which means large spheroids will start to break down. This breaking down could be a result of sedimentations that happen during osteo-differentiation [20]. For both densities equal to 1 and 2, the mean size remains the same with some fluctuations, and for 1 the spheroids are almost smaller. In the adipogenesis pathway, for all densities, at the beginning spheroids grow every day, but they start to break down at day 7 for density 2, day 4 for density 1, and day 5 for density 0.5. For densities 1 and 0.5, the mean sizes are almost the same, and for density 2, it is always larger.

Spheroid compactness

The analysis of spheroids' compactness in both differentiation pathways is illustrated in Fig. 10. It demonstrates more compact spheroids in the first two weeks, which is the result of a tighter cell–cell and cell-substrate connection. Also, this compactness is observed to be higher in the density of 2% and is more in the osteo-differentiation pathway. These observations show the relationship between substrate stiffness and cell attachment quality.

Fig. 10

Normalized violin plots with emphasized mean values for comparison between spheroids compactness regarding (a) osteocyte and (b) adipocyte pathways during a 21-day period, for three different densities of chitosan: red: 0.5, blue: 1, and green: 2

Spheroid’s solidity

Solidity is also an indicator of the cell's attachment to the substrate. As depicted in Fig. 11, the extracted data exhibit a pattern similar to spheroid compactness results. The solidification of spheroids is higher on the sample with a chitosan density of 2% than the other. This shows a higher cell attachment quality for this substrate.

Fig. 11

Normalized violin plots with emphasized mean values for comparison between spheroids solidity regarding (a) osteocyte and (b) adipocyte pathways during a 21-day period for three different densities of chitosan. red: 0.5, blue: 1, and green: 2

Spheroids eccentricity

Results illustrated in Fig. 12 reveal increased eccentricity in cells during the adipocyte pathway in the latter two weeks, whereas osteocytes exhibit sustained lower eccentricity. This finding confirms the stronger cell–cell attachment observed in osteocyte spheroids.

Fig. 12

Normalized violin plots with emphasized mean values for comparison between spheroids eccentricity regarding (a) osteocyte and (b) adipocyte pathways during a 21-day period for three different densities of chitosan. red: 0.5, blue:1 and green:2

Discussion and conclusion

Developing and suggesting alternative and innovative strategies for studying the dynamics of cellular shifts and changes based on phenotypic screening has been a hot topic recently. This trend can be attributed to the increasing shift toward 3D cell cultures in different fields of biology, from regenerative medicine and cancer studies to drug development and the screaming process. These approaches are still mostly limited to 2D culture models and are calibrated to study morphological changes in single cells and to measure cell viability.

Despite the availability of commercialized software like ImageJ or CellProfiler, tools that perfom high level analysis capable of spheroid segmentation from low-resolution images with noisy backgrounds are not enough accurate to meet our requirements.

In our research, we developed a machine learning-based pipeline able to segment cells without using fluorescent staining. We measured and compared the shape metrics of various 3D cell spheroids formed on chitosan films, analyzing their changes during differentiation toward adipocytes and osteocytes.

The affordability and accessibility of light microscopy make it advantageous for this research. Accurate image segmentation is crucial for analyzing the morphological characteristics of spheroids during the experimental phase and for understanding the dynamics of MSCD for therapeutic purposes. To address the labor-intensive nature of manual annotation for obtaining pixel-level ground truth labels, we devised an innovative, weakly supervised multi-stage model that leverages CNNs for label-free spheroid segmentation. Our approach improves the overall quality of ground truth maps by incorporating a multi-stage process within a weakly supervised learning framework. We systematically evaluated the proposed framework using our collected low-resolution microscopic data with noisy backgrounds. The experimental results demonstrate the effectiveness of our segmentation approach in accurately isolating spheroids from the background. To quantitatively assess our algorithm, we also conducted extensive experiments using available annotated data, confirming the reliability and robustness of our method (refer to supplementary materials). Through costly experiments on both our collected data and a benchmark annotated dataset, we demonstrated that our proposed segmentation algorithm can reliably detect and segment spheroids, achieving performance comparable to fully supervised state-of-the-art approaches.

Indeed, our segmentation algorithm effectively identifies spheroids, particularly when a large labeled dataset is not available.​ However, since a portion of the algorithm necessitates human expert review of potentially corrupted candidates, the inspection process may become time-consuming when dealing with larger datasets, such as those containing millions of images. In such cases, labeling a small subset of images and using them as training sets through a supervised learning approach may be a more effective strategy. Another limitation of our approach is its ability to identify roughly circular objects. As part of our anomaly detection framework, one of the rules is to exclude highly elongated objects, which represent artifacts or noise resembling hair-like structures. Those wishing to segment such shapes must modify this segment of the algorithm and establish alternative rules within their automatic anomaly detection system. Additionally, we have implemented a rule to eliminate noise based on the size of objects by setting a defined threshold. If the goal is to distinguish between noise and objects of approximately the same size, this may pose a challenge to our segmentation design. ​Certainly, this limitation may impact the performance of our proposed algorithm when addressing the spatial overlap of organoids.​ When multiple organoids overlap, various scenarios can arise that may result in the removal of certain cases. These scenarios may include the application of a size filter or situations involving high elongation, both of which are automatically eliminated during the “Anomaly detection” step, as detailed in Sect. 3.1.

To conclude, in this research, the MSCs were cultured on chitosan substrates with concentrations of 0.5%, 1%, and 2%. These cells were then differentiated into “adipocytes” and “osteocytes” using adipogenic and osteogenic media, respectively, as described in the material and methods section. Cell viability within the spheroids was assessed using the using MTT assay and a cell-surface attachment test (using Hoechst dye) for each chitosan substrate concentration. To confirm proper differentiation, a real-time PCR analysis was conducted (see Supplementary Table S1 and Figure S3). The results indicated successful differentiation into adipocytes, as evidenced by the expression of PPARγ and lipoprotein lipase (LPL) genes, or into osteocytes, indicated by the expression of alkaline phosphatase (ALP) genes. Beta-2 microglobulin was utilized as the housekeeping gene. Our results are consistent with those of Zanoni et al., who showed a correlation between the cell viability within the spheroids and various morphological characteristics of the spheroids [53]. Among the three concentrations of chitosan used in the substrate, the 2% chitosan concentration demonstrated superior cell attachment and viability, as well as the formation of more compact spheroids with higher solidity and convex area, making it the most suitable and more pronging to differentiate osteocytes. To study the dynamics of differentiation, we aimed to use the morphological properties of the 3D spheroids and develop a pipeline to analyze and quantify these properties based on daily 2D images captured from the 3D cellular spheroids. For this purpose, from the day that the differentiation medium was added, every day for 21 days, the images of the spheroids were taken with an optical microscope (Olympus IX81) to extract the morphological features of the spheroids with artificial intelligence algorithms. The morphological properties obtained from spheroids via machine learning approaches include sphericity, roundness, total surface area, convex area, compactness, solidity, and eccentricity. Most of these studies used advanced microscopy techniques or fluorescent staining to acquire more detailed data within 3D cellular spheroids. These techniques require time-consuming sample preparation and imaging time, are not easily feasible for screening purposes, and require a large number of samples. Our current research aims to compare the differentiation potential of three different concentrations of our in-house chitosan biofilm as a basal scaffold for differentiating stem cells into adipocytes and osteocytes. Wu et al. used multiplexed immunofluorescence imaging along with deep learning for the characterization of tumor microenvironments [54]. This demonstrates that deep learning approaches are now widely used to analyze the morphology, microenvironment, and even mechanical properties of 3D spheroids. Vaidyanathan et al. used a machine learning pipeline to investigate the responses of vascular smooth muscle cell spheroid morphology to drug perturbations [55]. In our work, the morphological properties of 3D cell spheroids have also been obtained by an innovative deep learning pipeline to analyze time-lapse images from these cell culture methods, which can facilitate more efficient, cost-effective, and accurate analyses.