Simple Heuristics for Scheduling Apache Airflow: A Case Study at PT. X

Abstract. In the big data era, there will be a lot of data that needs to be handled properly for making well-informed business decisions. As a result, the data needs to get updated regularly for relevant analysis. Apache Airflow is commonly utilized to schedule the data update by running the query through sequences of DAG tasks. Unfortunately, scheduling DAG tasks is categorized as NP-Hard problem which is difficult to obtain the global optimum solution. This paper shows a simple heuristic algorithm to solve the subset of DAG tasks at PT.X using 2 computers (virtual machines). This is a special case of P_m|prec|C_max problem where there are 2 computers are being used. Before constructing the algorithm, CPM (Critical Path Method) is used as the baseline to get the most optimal solution. A Knapsack problem is then used to add additional tasks for virtual machine #1, and a heuristic for virtual machine #2 as a compromise to balance the cost and time of completion. The characteristics of this simple algorithm is discussed with numerical examples/experiments. Keywords: Heuristics, Apache Airflow, Directed Acyclic Graphs, Critical Path Method

Predicting IBE Webinar Registrants Using Linearization of Some Sigmoidal Functions

We present a novel approach to obtain parameters of some Sigmoidal functions such as: Stevens, Gompertz, Logistics, Weibull, Brody, von Bertalaffy, & Ontogenetic. This approach extends the approach relies on iterative process to find “maximum” R2 and the traditional Ordinary Least Square (OLS) that can be implemented very easily. We demonstrate the approach in predicting IBE Webinar registrants. We also provide a new modification for the stopping rule based on Taylor approximation that can make the algorithm more robust as well as stop faster with little impact to the prediction

Does it Pay to be Perfect? A Lesson Learnt from CV XYZ

E-commerce provides an excellent opportunity to increase the company revenues, especially during the COVID-19. Many features that appear in the E-commerce platform can influence the customer purchase intention. Therefore, most companies strive to achieve the highest maximum Star-rating that serves as the electronic version of word of mouth. CV XYZ expects to increase the 5-Star-rating count by creating the after-sales service program. However, does it pay to be perfect? By collecting Star-rating data from six online stores across three different online marketplaces organized by CV XYZ, we develop simple Statistics to a more sophisticated Markov Chain and Regression to analyze the data. The result shows that the distribution movement of Star rating before and after implement the program is not significantly different. However, the after-sales service program greatly improved the average daily Sales around IDR 31M (p-value for the T-test = 0.0039). By simply using Pearson Correlation analysis, we found a negative correlation between Sales and Star-Rating counts that encounter us in the �too good to be true� phenomenon. Therefore, it is more critical for the company to figure out an optimal number of 5-Star ratings achieved to maximize Sales than only focus to achieve the highest 5-Star ratings

Two-Stage Memory Allocation using AHP & Knapsack at PT Berca Hardayaperkasa

We propose to manage a (MicroStrategy) Business Intelligence Server in terms of RAM allocation for its Intelligent Cubes as a two-stage resource allocation problem in which the first stage is formulated as an multi-criteria problem that can be solved using Analytic Hierarchy Process (AHP) and the second stage is multiple (several) 0-1 classic Knapsack problems with the constraints that are obtained using the result from the first stage. This Approach happens to have an advantage in terms of computational complexity as well, it reduces from O(nM) to O(max{nj}max{Mj}) when calculated in parallel. We illustrate our proposal with a numerical example based on our experience

A Proposal on How to Use Software Reliability Growth Model to Build Confidence in Dashboard Testing

Software testing as an integral part of software development leads to the question of when the software (or application/dashboard) can be released and how confident that most defects/bugs/faults have been discovered. This paper discussed a relatively new but simple and practical proposal that can be used to build confidence for releasing software (or applications/dashboards). Instead of contrasting various software reliability growth models (SRGM) and choosing which one is the best, we use them to collaborate to help make decisions. We demonstrate our proposal with 18 real-life datasets that are publicly available in the literature. We use three widely used SRGMs, namely: Bass, Gompertz, and Logistic in our proposal to identify when we can stop testing. It turns out that when the testing has found most defects, most (if not all) of the SRGMs will converge to similar value for the maximum potential defects in the system

On the Weights for Characteristics and Comparables for Property Valuation using Quality Rating Valuation Estimation

This study considers the problem of finding weights for building characteristics and compares buildings in property valuation to provide a more rigorous analytical foundation for a simple yet practical valuation technique knowns as Quality Rating Valuation Estimation (QRVE). Mathematically, we prove that the “best” characteristic weights can be obtained from Multiple Linear Regression Analysis (MRA) coefficients. Furthermore, by applying the Gower Similarity index and the Partition Around Medoid (PAM) clustering technique, the proposed algorithm provides an appropriate similarity of the weighing of compared buildings. The case studies illustrate a way to select a subset of characteristics when there are many of them with two numerical examples, as well as a complete modification of QRVE in conjunction with the grid adjustment technique. The modified QRVE proposal results in a very reasonable and high valuation performance of the building value estimate

Marketing a Webinar: Lesson Learnt from a Real Experiment

Implementing an email campaign can powerfully and positively influence customers� attitudes about the content of the email used as promotional media. However, email marketing efforts may not reach their full potential without evaluating their marketing strategy. This paper discusses the real-world experience of the email marketing strategy of the webinar that we managed by applying email marketing strategy evaluation combined with mathematical diffusion models to increase the number of registrations. After starting with a digital external campaign for our webinar event, we monitor and predict using Bass, von Bertalanffy, Gompertz, and Logistic. While the mathematical diffusion models show that our webinar reached a plateau very early, it could become an alarm for the management team to react and make another internal digital campaign to boost our webinar participants. Interestingly, we also encountered a procrastination phenomenon since this was a free webinar. Finally, using Zoom registration, MySQL database, and R, we analyzed how effective the digital email campaign was in getting registrants daily

Capturing Social Value using Ordinal Data, Gower Distance & Quality Point

Social value refers to an actionable concept and something that will positively impact stakeholders and broader society for better living. Capturing the social value of a building in a sales or rental price is very important but challenging because its definition could be interpreted widely due to the subjectivity and nature of social value for each case building, which produces ambiguity for the different auditors and the stakeholders. To give a reasonable estimate for the value (either for sales and/or rental price), we propose and illustrate how to use ordinal (and nominal) rating data to capture social value together with other common building characteristics. We propose a novel, simple, and practical approach using Gower Distance, i.e., Gower (Dis)similarity index(es) among comparable buildings & clustering technique as the basis. Hence, it is more flexible in selecting comparable buildings & can be easily adapted to measure Social Value. Together with the classic grid adjustment method (Quality Point/Quality Rating adjustment), they can be used to provide a fair market estimate for the rental (or sale) price of buildings capturing the social value. Calculation using numerical examples from a client in Jakarta is provided to illustrate how the proposal works

Does it Pay to be Perfect? A Lesson Learnt from CV XYZ

E-commerce provides an excellent opportunity to increase the company revenues, especially during the COVID-19. Many features that appear in the E-commerce platform can influence the customer purchase intention. Therefore, most companies strive to achieve the highest maximum Star-rating that serves as the electronic version of word of mouth. CV XYZ expects to increase the 5-Star-rating count by creating the after-sales service program. However, does it pay to be perfect? By collecting Star-rating data from six online stores across three different online marketplaces organized by CV XYZ, we develop simple Statistics to a more sophisticated Markov Chain and Regression to analyze the data. The result shows that the distribution movement of Star rating before and after implement the program is not significantly different. However, the after-sales service program greatly improved the average daily Sales around IDR 31M (p-value for the T-test = 0.0039). By simply using Pearson Correlation analysis, we found a negative correlation between Sales and Star-Rating counts that encounter us in the �too good to be true� phenomenon. Therefore, it is more critical for the company to figure out an optimal number of 5-Star ratings achieved to maximize Sales than only focus to achieve the highest 5-Star ratings