X‐ray microscopy and automatic detection of defects in through silicon vias in three‐dimensional integrated circuits

Through silicon vias (TSVs) are a key enabling technology for interconnection and realization of complex three-dimensional integrated circuit (3D-IC) components. In order to perform failure analysis without the need of destructive sample preparation, x-ray microscopy (XRM) is a rising method of analyzing the internal structure of samples. However, there is still a lack of evaluated scan recipes or best practices regarding XRM parameter settings for the study of TSVs in the current state of literature. There is also an increased interest in automated machine learning and deep learning approaches for qualitative and quantitative inspection processes in recent years. Especially deep learning based object detection is a well-known methodology for fast detection and classification capable of working with large volumetric XRM datasets. Therefore, a combined XRM and deep learning object detection workflow for automatic micrometer accurate defect location on liner-TSVs was developed throughout this work. Two measurement setups including detailed information about the used parameters for either full IC device scan or detailed TSV scan were introduced. Both are able to depict delamination defects and finer structures in TSVs with either a low or high resolution. The combination of a 0.4 objective with a beam voltage of 40 kV proved to be a good combination for achieving optimal imaging contrast for the full-device scan. However, detailed TSV scans have demonstrated that the use of a 20 objective along with a beam voltage of 140 kV significantly improves image quality. A database with 30,000 objects was created for automated data analysis, so that a well-established object recognition method for automated defect analysis could be integrated into the process analysis. This RetinaNet-based object detection method achieves a very strong average precision of 0.94. It supports the detection of erroneous TSVs in both top view and side view, so that defects can be detected at different depths. Consequently, the proposed workflow can be used for failure analysis, quality control or process optimization in R&D environments

Author Correction: Novel diagnostic and therapeutic techniques reveal changed metabolic profiles in recurrent focal segmental glomerulosclerosis

Idiopathic forms of Focal Segmental Glomerulosclerosis (FSGS) are caused by circulating permeability factors, which can lead to early recurrence of FSGS and kidney failure after kidney transplantation. In the past three decades, many research endeavors were undertaken to identify these unknown factors. Even though some potential candidates have been recently discussed in the literature, “the” actual factor remains elusive. Therefore, there is an increased demand in FSGS research for the use of novel technologies that allow us to study FSGS from a yet unexplored angle. Here, we report the successful treatment of recurrent FSGS in a patient after living-related kidney transplantation by removal of circulating factors with CytoSorb apheresis. Interestingly, the classical published circulating factors were all in normal range in this patient but early disease recurrence in the transplant kidney and immediate response to CytoSorb apheresis were still suggestive for pathogenic circulating factors. To proof the functional effects of the patient’s serum on podocytes and the glomerular filtration barrier we used a podocyte cell culture model and a proteinuria model in zebrafish to detect pathogenic effects on the podocytes actin cytoskeleton inducing a functional phenotype and podocyte effacement. We then performed Raman spectroscopy in the < 50 kDa serum fraction, on cultured podocytes treated with the FSGS serum and in kidney biopsies of the same patient at the time of transplantation and at the time of disease recurrence. The analysis revealed changes in podocyte metabolome induced by the FSGS serum as well as in focal glomerular and parietal epithelial cell regions in the FSGS biopsy. Several altered Raman spectra were identified in the fractionated serum and metabolome analysis by mass spectrometry detected lipid profiles in the FSGS serum, which were supported by disturbances in the Raman spectra. Our novel innovative analysis reveals changed lipid metabolome profiles associated with idiopathic FSGS that might reflect a new subtype of the disease

Novel diagnostic and therapeutic techniques reveal changed metabolic profiles in recurrent focal segmental glomerulosclerosis

Idiopathic forms of Focal Segmental Glomerulosclerosis (FSGS) are caused by circulating permeability factors, which can lead to early recurrence of FSGS and kidney failure after kidney transplantation. In the past three decades, many research endeavors were undertaken to identify these unknown factors. Even though some potential candidates have been recently discussed in the literature, “the” actual factor remains elusive. Therefore, there is an increased demand in FSGS research for the use of novel technologies that allow us to study FSGS from a yet unexplored angle. Here, we report the successful treatment of recurrent FSGS in a patient after living-related kidney transplantation by removal of circulating factors with CytoSorb apheresis. Interestingly, the classical published circulating factors were all in normal range in this patient but early disease recurrence in the transplant kidney and immediate response to CytoSorb apheresis were still suggestive for pathogenic circulating factors. To proof the functional effects of the patient’s serum on podocytes and the glomerular filtration barrier we used a podocyte cell culture model and a proteinuria model in zebrafish to detect pathogenic effects on the podocytes actin cytoskeleton inducing a functional phenotype and podocyte effacement. We then performed Raman spectroscopy in the < 50 kDa serum fraction, on cultured podocytes treated with the FSGS serum and in kidney biopsies of the same patient at the time of transplantation and at the time of disease recurrence. The analysis revealed changes in podocyte metabolome induced by the FSGS serum as well as in focal glomerular and parietal epithelial cell regions in the FSGS biopsy. Several altered Raman spectra were identified in the fractionated serum and metabolome analysis by mass spectrometry detected lipid profiles in the FSGS serum, which were supported by disturbances in the Raman spectra. Our novel innovative analysis reveals changed lipid metabolome profiles associated with idiopathic FSGS that might reflect a new subtype of the disease

Determination of Forming Limits in Sheet Metal Forming Using Deep Learning

The forming limit curve (FLC) is used to model the onset of sheet metal instability during forming processes e.g., in the area of finite element analysis, and is usually determined by evaluation of strain distributions, derived with optical measurement systems during Nakajima tests. Current methods comprise of the standardized DIN EN ISO 12004-2 or time-dependent approaches that heuristically limit the evaluation area to a fraction of the available information and show weaknesses in the context of brittle materials without a pronounced necking phase. To address these limitations, supervised and unsupervised pattern recognition methods were introduced recently. However, these approaches are still dependent on prior knowledge, time, and localization information. This study overcomes these limitations by adopting a Siamese convolutional neural network (CNN), as a feature extractor. Suitable features are automatically learned using the extreme cases of the homogeneous and inhomogeneous forming phase in a supervised setup. Using robust Student’s t mixture models, the learned features are clustered into three distributions in an unsupervised manner that cover the complete forming process. Due to the location and time independency of the method, the knowledge learned from formed specimen up until fracture can be transferred on to other forming processes that were prematurely stopped and assessed using metallographic examinations, enabling probabilistic cluster membership assignments for each frame of the forming sequence. The generalization of the method to unseen materials is evaluated in multiple experiments, and additionally tested on an aluminum alloy AA5182, which is characterized by Portevin-LE Chatlier effects

Bestimmung und Erweiterung der Grenzformänderungskurve für Blechwerkstoffe mittels Maschinellem Lernen

Future legal standards for European automobiles will require a considerable reduction in CO2 emissions by 2021. In order to meet these requirements, an optimization of the automobiles is required, comprising technological improvements of the engine and aerodynamics, or even more important, weight reductions by using light-weight components. The properties of light-weight materials differ considerably from those of conventional materials and therefore, it is essential to correctly define the formability of high-strength steel or aluminum alloys. In sheet metal forming, the forming capacity is determined by means of the forming limit curve that specifies the maximum forming limits for a material. However, current methods are based on heuristics and have the disadvantage that only a very limited portion of the evaluation area is considered. Moreover, the methodology of the industry standard is user-dependent with simultaneously varying reproducibility of the results. Consequently, a large safety margin from the experimentally determined forming limit curves is required in process design. This thesis introduces pattern recognition methods for the determination of the forming limit curve. The focus of this work is the development of a methodology that circumvents the previous disadvantages of location-, time-, user- and material dependencies. The dependency on the required a priori knowledge is successively reduced by incrementally improving the proposed methods. The initial concept proposes a supervised classification approach based on established textural features in combination with a classifier and addresses a four-class problem consisting of the homogeneous forming, the diffuse and local necking, as well as the crack class. In particular for the relevant class of local necking, a sensitivity of up to 92% is obtained for high-strength materials. Since a supervised procedure would require expert annotations for each new material, an unsupervised classification method to determine the local necking is preferred, so that anomaly detection is feasible by means of predefined features. A probabilistic forming limit curve can thus be defined in combination with Gaussian distributions and consideration of the forming progression. In order to further reduce the necessary prior knowledge, data-driven features are learned based on unsupervised deep learning methods. These features are adapted specifically to the respective forming sequences of the individual materials and are potentially more robust and characteristic in comparison to the predefined features. However, it was discovered that the feature space is not well-regularized and thus not suitable for unsupervised clustering procedures. Consequently, the last methodology introduces a weakly supervised deep learning approach. For this purpose, several images of the beginning and end of the forming sequences are used to learn optimal features in a supervised setup while regularizing the feature space. Through unsupervised clustering, this facilitates the class membership determination for individual frames of the forming sequences and the definition of the probabilistic forming limit curve. Moreover, this approach enables a visual examination and interpretation of the actual necking area.Zukünftige gesetzliche Standards erfordern für europäische Automobile eine erhebliche Reduktion der CO2 Emissionen bis 2021. Um diese Auflagen zu erfüllen, bedarf es einer Optimierung der Fahrzeuge, die sich aus technologischen Verbesserungen des Motors und der Aerodynamik zusammensetzt, oder viel wichtiger einer signifikanten Gewichtsreduktion durch den Einsatz von Leichtbau. Die Eigenschaften von Leichtbaumaterialien weichen erheblich von denen herkömmlicher Werkstoffe ab, weshalb für hochfesten Stahl oder Aluminiumlegierungen eine korrekte Definition ihrer Umformfähigkeit erforderlich ist. In der Blechumformung wird die Umformfähigkeit mittels Grenzformänderungskurve bestimmt, welche die maximalen Umformungsgrenzen für ein Material definiert. Alle derzeitigen Methoden basieren auf Heuristiken und haben den Nachteil, dass nur ein sehr geringer Anteil des Auswertungsbereiches berücksichtigt wird. Darüber hinaus ist die Methodik des Industriestandards benutzerabhängig bei gleichzeitig schwankender Reproduzierbarkeit der Ergebnisse. Als Konsequenz kommt in der Prozessauslegung ein großer Sicherheitsabstand von den experimentell bestimmten Grenzformänderungskurven zur Anwendung. Mit dieser Arbeit werden erstmalig Methoden der Mustererkennung zur Bestimmung der Grenzformänderungskurve eingesetzt. Mittelpunkt der Arbeit ist die Entwicklung einer Methodik, welche die bisherigen Nachteile der Orts-, Zeit-, Benutzer- und Materialabhängigkeiten umgeht. In einer inkrementellen Herangehensweise wird die Abhängigkeit vom benötigten a priori Wissen sukzessive verringert. Ausgangspunkt ist ein überwachter Klassifikationsansatz basierend auf etablierten Texturmerkmalen im Zusammenspiel mit einem Klassifikator. Dieser löst ein Vierklassenproblem bestehend aus der homogenen Umformung, der diffusen- und lokalen Einschnürung sowie der Riss Klasse. Insbesondere für die relevante Klasse der lokalen Einschnürung, wird für hochfeste Materialien eine Sensitivität von bis zu 92% erreicht. Da eine überwachte Vorgehensweise für jedes neue Material Expertenannotationen benötigen wurde, wird eine unüberwachte Klassifikationsmethode zur Bestimmung der lokalen Einschnürung bevorzugt, sodass mit Hilfe von vordefinierten Merkmalen eine Anomalie-Erkennung möglich ist. Hierbei wird die Abweichung von der homogenen Umformungsphase in den Umformsequenzen festgestellt. In Verbindung mit Verteilungsfunktionen und unter Zuhilfenahme des Umformungsfortschrittes kann somit eine probabilistische Grenzformänderungskurve definiert werden. Um das notwendige Vorwissen weiter zu reduzieren, werden mittels unüberwachten Deep Learning Verfahren datengetrieben Merkmale gelernt. Diese sind charakteristisch an die jeweiligen Umformsequenzen der einzelnen Materialien angepasst und potentiell robuster als die vordefinierten Merkmale. In diesem Zusammenhang hat sich herausgestellt, dass der resultierende Merkmalsraum für unüberwachte Clusterverfahren nicht geeignet ist. Als Konsequenz wird in der letzten Methodik ein schwach-überwachtes Deep Learning Verfahren eingeführt. Dieses verwendet nur einen Bruchteil der zur Verfügung stehenden Bilder des Beginns und des Endes der Umformungssequenz, um einen optimalen Merkmalsraum zu lernen. Dies ermöglicht mittels unüberwachtem Clusterverfahren neben der Bestimmung von Versagensklassen für einzelne Frames der Umformsequenzen und der probabilistischen Grenzformänderungskurve ebenso eine Abschätzung des tatsächlichen Einschnürungsbereiches

Analysis of Forming Limits in Sheet Metal Forming with Pattern Recognition Methods. Part 2: Unsupervised Methodology and Application

The forming limit curve (FLC) is used in finite element analysis (FEA) for the modeling of onset of sheet metal instability during forming. The FLC is usually evaluated by achieving forming measurements with optical measurement system during Nakajima tests. Current evaluation methods such as the standard method according to DIN EN ISO 12004-2 and time-dependent methods limit the evaluation range to a fraction of the available information and show weaknesses in the context of brittle materials that do not have a pronounced constriction phase. In order to meet these challenges, a supervised pattern recognition method was proposed, whose results depend on the quality of the expert annotations. In order to alleviate this dependence on experts, this study proposes an unsupervised classification approach that does not require expert annotations and allows a probabilistic evaluation of the onset of localized necking. For this purpose, the results of the Nakajima tests are examined with an optical measuring system and evaluated using an unsupervised classification method. In order to assess the quality of the results, a comparison is made with the time-dependent method proposed by Volk and Hora, as well as expert annotations, while validated with metallographic investigations. Two evaluation methods are presented, the deterministic FLC, which provides a lower and upper limit for the onset of necking, and a probabilistic FLC, which allows definition of failure quantiles. Both methods provide a necking range that shows good correlation with the expert opinion as well as the results of the time-dependent method and metallographic examinations

Temporal and Spatial Detection of the Onset of Local Necking and Assessment of its Growth Behavior

This study proposes a method for the temporal and spatial determination of the onset of local necking determined by means of a Nakajima test set-up for a DC04 deep drawing and a DP800 dual-phase steel, as well as an AA6014 aluminum alloy. Furthermore, the focus lies on the observation of the progress of the necking area and its transformation throughout the remainder of the forming process. The strain behavior is learned by a machine learning approach on the basis of the images when the process is close to material failure. These learned failure characteristics are transferred to new forming sequences, so that critical areas indicating material failure can be identified at an early stage, and consequently enable the determination of the beginning of necking and the analysis of the necking area. This improves understanding of the necking behavior and facilitates the determination of the evaluation area for strain paths. The growth behavior and traceability of the necking area is objectified by the proposed weakly supervised machine learning approach, thereby rendering a heuristic-based determination unnecessary. Furthermore, a simultaneous evaluation on image and pixel scale is provided that enables a distinct selection of the failure quantile of the probabilistic forming limit curve

Analysis of Forming Limits in Sheet Metal Forming with Pattern Recognition Methods. Part 2: Unsupervised Methodology and Application

The forming limit curve (FLC) is used in finite element analysis (FEA) for the modeling of onset of sheet metal instability during forming. The FLC is usually evaluated by achieving forming measurements with optical measurement system during Nakajima tests. Current evaluation methods such as the standard method according to DIN EN ISO 12004-2 and time-dependent methods limit the evaluation range to a fraction of the available information and show weaknesses in the context of brittle materials that do not have a pronounced constriction phase. In order to meet these challenges, a supervised pattern recognition method was proposed, whose results depend on the quality of the expert annotations. In order to alleviate this dependence on experts, this study proposes an unsupervised classification approach that does not require expert annotations and allows a probabilistic evaluation of the onset of localized necking. For this purpose, the results of the Nakajima tests are examined with an optical measuring system and evaluated using an unsupervised classification method. In order to assess the quality of the results, a comparison is made with the time-dependent method proposed by Volk and Hora, as well as expert annotations, while validated with metallographic investigations. Two evaluation methods are presented, the deterministic FLC, which provides a lower and upper limit for the onset of necking, and a probabilistic FLC, which allows definition of failure quantiles. Both methods provide a necking range that shows good correlation with the expert opinion as well as the results of the time-dependent method and metallographic examinations