Multi-modal Embedding Fusion-based Recommender

Recommendation systems have lately been popularized globally, with primary use cases in online interaction systems, with significant focus on e-commerce platforms. We have developed a machine learning-based recommendation platform, which can be easily applied to almost any items and/or actions domain. Contrary to existing recommendation systems, our platform supports multiple types of interaction data with multiple modalities of metadata natively. This is achieved through multi-modal fusion of various data representations. We deployed the platform into multiple e-commerce stores of different kinds, e.g. food and beverages, shoes, fashion items, telecom operators. Here, we present our system, its flexibility and performance. We also show benchmark results on open datasets, that significantly outperform state-of-the-art prior work.Comment: 7 pages, 8 figure

A New Way for Face Sketch Construction and Detection Using Deep CNN

Traditional hand-drawn face sketches have encountered speed and accuracy issues in the field of forensic science when used in conjunction with contemporary criminal identification technologies. To close this gap, we provide a ground-breaking research article that is built on a stand-alone program that aims to revolutionize the production and identification of composite face sketches. This ground-breaking approach does away with the requirement for forensic artists by enabling users to easily create composite sketches using a drag-and-drop interface. Utilizing the power of deep learning and cloud infrastructure, these generated sketches are seamlessly cross-referenced against an enormous police database to identify suspects quickly and precisely. Our research study offers a dual-pronged approach to combating the rise in criminal activity while using the quick breakthroughs in artificial intelligence. First, we demonstrate how a specific Deep Convolutional Neural Network model transforms sketches of faces into photorealistic photographs. Second, we employ transfer learning for precise suspect identification using the pre-trained VGG-Face model. Utilizing Convolutional Neural Networks, which are famous for their data processing powers and hierarchical feature extraction, is a key component of our strategy. This approach exceeds current methods and boasts an extraordinary average accuracy of 0.98 in identifying people from sketches, providing a crucial tool for strengthening and speeding up forensic investigations. A unique Convolutional Neural Network framework that demonstrates significant improvements over state-of-the-art techniques is also revealed as we dive into the challenging task of matching composite sketches with corresponding digital photos. Our thorough analysis shows the framework to be remarkably accurate, constituting a substantial advance in the field of forensic face sketch production and recognition

Deep Learning for Free-Hand Sketch: A Survey

Free-hand sketches are highly illustrative, and have been widely used by humans to depict objects or stories from ancient times to the present. The recent prevalence of touchscreen devices has made sketch creation a much easier task than ever and consequently made sketch-oriented applications increasingly popular. The progress of deep learning has immensely benefited free-hand sketch research and applications. This paper presents a comprehensive survey of the deep learning techniques oriented at free-hand sketch data, and the applications that they enable. The main contents of this survey include: (i) A discussion of the intrinsic traits and unique challenges of free-hand sketch, to highlight the essential differences between sketch data and other data modalities, e.g., natural photos. (ii) A review of the developments of free-hand sketch research in the deep learning era, by surveying existing datasets, research topics, and the state-of-the-art methods through a detailed taxonomy and experimental evaluation. (iii) Promotion of future work via a discussion of bottlenecks, open problems, and potential research directions for the community.Comment: This paper is accepted by IEEE TPAM

cGAN-based Manga Colorization Using a Single Training Image

The Japanese comic format known as Manga is popular all over the world. It is traditionally produced in black and white, and colorization is time consuming and costly. Automatic colorization methods generally rely on greyscale values, which are not present in manga. Furthermore, due to copyright protection, colorized manga available for training is scarce. We propose a manga colorization method based on conditional Generative Adversarial Networks (cGAN). Unlike previous cGAN approaches that use many hundreds or thousands of training images, our method requires only a single colorized reference image for training, avoiding the need of a large dataset. Colorizing manga using cGANs can produce blurry results with artifacts, and the resolution is limited. We therefore also propose a method of segmentation and color-correction to mitigate these issues. The final results are sharp, clear, and in high resolution, and stay true to the character's original color scheme.Comment: 8 pages, 13 figure