The study of open charm meson production provides an efficient tool for detailed investigations of the properties of hot and dense matter formed in relativistic nucleus-nucleus collisions. In particular, charm mesons are of vivid interest in the context of the phase-transition between confined hadronic matter and the quark-gluon plasma as well as for interpretation of data on J/ψ production measured by the NA38/NA50 and NA60 experiments. Also, such study gives a unique opportunity to test the validity of theoretical models based on perturbative Quantum Chromodynamics and Statistical model approaches for nucleus collisions at the top SPS energy. Such models provide predictions for charm yields, that differ by up to two orders of magnitude. Direct measurements of open charm mesons are challenging since the yields of D mesons are very low and their lifetimes are short, which means that such measurements require precise determination of particle tracks and high primary and secondary vertex resolution. To meet these challenges a new high resolution Small Acceptance Vertex Detector (SAVD) based on silicon pixel sensors MIMOSA-26AHR was constructed and installed in the NA61/SHINE experiment which was motivated by the importance and the possibility of the first direct measurements of open charm mesons in heavy ion collisions at SPS energies. The SAVD was installed in 2016 to perform a test for Pb+Pb reactions with the beam momentum of 150A GeV/c. Later, in 2017 and 2018 large statistic data sets have been taken for Xe+La and Pb+Pb collisions at 150A GeV/c. Within this PhD project reconstruction and analysis of the collected data of Xe+La and Pb+Pb collisions in the NA61/SHINE experiment were performed in order to determine the yield of D0+D0. The obtained D0+D0 signal was the first, direct observation of open charm in nucleus-nucleus collisions at the SPS energies. These results were compared to the theoretical model predictions as well as the estimation made by the NA50/NA60 experiments. Furthermore, the NA61/SHINE experiment plans a systematic measurements of open charm production in Pb+Pb collisions in the period 2021-2024 after the major detector upgrade conducted during the Long Shutdown 2. The performed simulation study of the upgraded Vertex Detector showed, that these future data will allow for the more detailed research of charm particles