214 research outputs found

    FLuID: Mitigating Stragglers in Federated Learning using Invariant Dropout

    Full text link
    Federated Learning (FL) allows machine learning models to train locally on individual mobile devices, synchronizing model updates via a shared server. This approach safeguards user privacy; however, it also generates a heterogeneous training environment due to the varying performance capabilities across devices. As a result, straggler devices with lower performance often dictate the overall training time in FL. In this work, we aim to alleviate this performance bottleneck due to stragglers by dynamically balancing the training load across the system. We introduce Invariant Dropout, a method that extracts a sub-model based on the weight update threshold, thereby minimizing potential impacts on accuracy. Building on this dropout technique, we develop an adaptive training framework, Federated Learning using Invariant Dropout (FLuID). FLuID offers a lightweight sub-model extraction to regulate computational intensity, thereby reducing the load on straggler devices without affecting model quality. Our method leverages neuron updates from non-straggler devices to construct a tailored sub-model for each straggler based on client performance profiling. Furthermore, FLuID can dynamically adapt to changes in stragglers as runtime conditions shift. We evaluate FLuID using five real-world mobile clients. The evaluations show that Invariant Dropout maintains baseline model efficiency while alleviating the performance bottleneck of stragglers through a dynamic, runtime approach

    Hadoop Distributed file system, Hive and Its Applications: A Survey

    Get PDF
    Business intelligence is growing area across the industry and data getting collected and analyzed in rapid way due to which legacy warehousing tools has become very costly. Hadoop is framework which is open source and stores data and runs applications on cluster of normal i.e commodity hardware. Hadoop provides large amount of processing power and storage for various kinds of data. It is able to handle concurrent tasks or jobs. HDFS (Hadoop Distributed File System) is a distributed file system which can provide high performance data access across Hadoop cluster of servers. Due to Managing pools of big data and supporting big data analytics application HDFS has become a strong tool. Developer has to write custom programs in map reduce programming model which are difficult to maintain and reuse. Hive is open source solution built on top of hadoop which is used as data ware house. Hive supports HiveQL which is SQL-like language, which are compiled into mapreduce jobs to be executed on Hadoop

    Equity Reports Optimization Using Hive, HDFS.

    Get PDF
    Now a day there is much competition in stock market world. Every brokerage firm Want to lesser down the TURN AROUND TIME for their applications so that business users/analyzers could be able provide solutions to their clients within a time frame. Equity clients prefer to do a business with firms which provide the very fast solutions for their queries. To solve clients query users has to fetch the reports from the software system. As stock market is time critical business. Every corporate has minimum time frame to take their business decisions. For taking business decisions in fast way, we should have usable data available within a time frame. With the help of reporting and research applications, it becomes easy to make accurate business data available in fast way

    Novel technique of removal of broken intra-medullary nail from femur with secondary DCS plating

    Get PDF
    Femoral shaft fractures are one of the commonest fractures of the lower limb which are frequently operated with intramedullary nailing which enables immediate post-operative mobilization of the patient. There could be various causes of nail breakage – some of the notable being weight bearing over the non-union of the femur shaft, or a re-trauma over the operated limb causing both the implant and the nail to be broken. There are various methods of removal of the broken implant the commonest being the use of T-reamer technique. However not always can this be used due to varied intra-operative obstacles in different cases as described in this case below. We have a 35 year old male patient who was brought to us 2 hours after an alleged history of slip and fall following which he had sustained right sided subtrochanteric femur fracture with a broken implant – intramedullary interlock nail. The patient is a previously operated case of right sided femur shaft fracture with interlocking nailing done 15 years back. The patient was operated with – broken implant removal on the right side along with a secondary DCS plating with bone grafting for the subtrochanteric femur fracture. Intra operative period was met with a certain number of challenges and difficulties in view of a 15 year old implant for removal which was successfully with removed with DCS plating done. As is obvious with the above case, it would be quite imperative to say that older the implant, more difficult it becomes for its removal.
    • …
    corecore