Efficient Real Time Recurrent Learning through combined activity and parameter sparsity

Backpropagation through time (BPTT) is the standard algorithm for training recurrent neural networks (RNNs), which requires separate simulation phases for the forward and backward passes for inference and learning, respectively. Moreover, BPTT requires storing the complete history of network states between phases, with memory consumption growing proportional to the input sequence length. This makes BPTT unsuited for online learning and presents a challenge for implementation on low-resource real-time systems. Real-Time Recurrent Learning (RTRL) allows online learning, and the growth of required memory is independent of sequence length. However, RTRL suffers from exceptionally high computational costs that grow proportional to the fourth power of the state size, making RTRL computationally intractable for all but the smallest of networks. In this work, we show that recurrent networks exhibiting high activity sparsity can reduce the computational cost of RTRL. Moreover, combining activity and parameter sparsity can lead to significant enough savings in computational and memory costs to make RTRL practical. Unlike previous work, this improvement in the efficiency of RTRL can be achieved without using any approximations for the learning process.Comment: Published as a workshop paper at ICLR 2023 Workshop on Sparsity in Neural Network

Balancing the interests of employer and employee in dismissal for misconduct

South Africa emerged from a history dogged by an oppressive system in which race was used as a medium of oppression. Workers and in particular African workers’ rights were severely curtailed. However, following the advent of the Constitution, several employees’ rights and freedoms are now entrenched key amongst them in the right to fair labour practices is enshrined in section 23 (1) of the Constitution. Post 1994, South Africa adopted various new forms of labour legislation, including the Labour Relations Act. This marked the watershed in changing the balance of power away from the employer. The LRA gives form and content to the rights enshrined in the Constitution by establishing substantive and procedural requirements prior to dismissal. Equally important is the guidelines contained in schedule 8 to the LRA which depict an attempt by the legislature to ensure that employees are protected against unfair dismissal. The historical background of the employment relationship stems from the Master and Servant Act. The common law evolved in South Africa from Roman-Dutch and English practices. The common law was shaped against the backdrop of Apartheid modified to some extent through the Wiehahn Commission4 and more recently politically through union and National Economic Development and Labour Council (NEDLAC) involvement regulating labour practices through legislation. In South Africa, the employment relationship is regulated by three main sources of law. These include the Constitution, labour legislation and the law of contract. Besides these sources, South Africa is a member state of the International Labour Organisation

Opportunities presented by information and communication technologies for women led SMMEs in KwaZulu-Natal.

MBA University of KwaZulu-Natal, Durban 2012.In South Africa, and especially in KwaZulu-Natal, small, medium, and microenterprises (SMMEs) contribute significantly to the economy. However, these businesses operate in a highly competitive environment and have to contend with many larger enterprises. Within the SMME sector in South Africa many women are operating their small businesses, thus contributing to the country’s gross domestic product (GDP). These women, however, are subjected to many challenges, and face many barriers when operating their small businesses. Access to and the use of information and communication technologies (ICTs) for business development and growth cannot be overemphasised. It is for this reason that this study aims to determine the opportunities ICTs can present to women-led SMMEs; the barriers and challenges to accessing and using ICTs in business; as well as to make recommendations that enable and support the use of ICTs for women-led SMMEs in KwaZulu-Natal and the rest of South Africa. In order to gain a better perspective of the opportunities that ICT plays in women-led SMMEs, a sample of 32 women in KZN were chosen on whom to conduct a study. A mixed-methods research methodology was undertaken which included both quantitative and qualitative data. A non-probability sample of 32 women-led SMMEs were chosen from a population of 80 from the greater Durban area of KwaZulu-Natal. Data was collected by using a questionnaire designed by the researcher which comprised of close-ended questions. An additional structured interview schedule was also used which included open-ended questions in order to gain a deeper understanding of the problem. Statistical analysis included statistical testing using hypothesis testing, Cronbach’s alpha, factor analysis and correlations. Statistical analysis revealed that women-led SMMEs face many barriers and challenges in accessing and using ICTs in their businesses; these include time, family responsibilities, finance, as well as lack of skills and infrastructure. This study can benefit governments as well as women SMME owners, and other role players involved in women empowerment, in identifying the barriers and challenges these women face when accessing and using ICTs. It also makes recommendations for the use of ICTs in improving their business performance. This study recommends the elimination of bias, access to finance, education, skills training, access and awareness of ICT services, as well as government and business support for women-led SMMEs, in order for them to utilise ICTs to develop and grow their businesses

Evaluating modular neuroevolution in robotic keepaway soccer

textKeepaway is a simpler subtask of robot soccer where three `keepers' attempt to keep possession of the ball while a `taker' tries to steal it from them. This is a less complex task than full robot soccer, and lends itself well as a testbed for multi-agent systems. This thesis does a comprehensive evaluation of various learning methods using neuroevolution with Enforced Sub-Populations (ESP) with the robocup soccer simulator. Both single and multi-component ESP are evaluated using various learning methods on homogeneous and heterogeneous teams of agents. In particular, the effectiveness of modularity and task decomposition for evolving keepaway teams is evaluated. It is shown that in the robocup soccer simulator, homogeneous agents controlled by monolithic networks perform the best. More complex learning approaches like layered learning, concurrent layered learning and co-evolution decrease the performance as does making the agents heterogeneous. The results are also compared with previous results in the keepaway domain.Computer Science

The privatisation of public space: case study of the Durban inner-city coastal area.

Masters in Town and Regional Planning. University of KwaZulu-Natal, Howard College 2016.This research investigates the extent to which public space is privatised in the Durban Inner-city coastal area. Public spaces at Vetch's Beach and the Point Development area are being privatized due to mechanisms of security. Privatization of public space has also occurred through the encroachment of private development projects. Concepts of Postmodern Urbanism refers to this process as the commodification of public spaces. Through the use of a case study methodology, public space users' perceptions, municipal officials of eThekwini municipality and developers of the Point Development Precinct have provided an insight into the privatization of public space. People are excluded from, or have limited access to public spaces showing that people have few rights to protect public space in the case study areas

The role of the school management team in improving the matric results in a national strategy for learner attainment (NSLA) school.

Master of Education in Education, Development, Leadership and Management. University of KwaZulu-Natal, Durban 2016.This single case study looked at the experiences of the School Management Team (SMT) within one National Strategy for Learner Attainment (NSLA) School in the Umlazi District in KwaZulu-Natal. The school in this study attained the NSLA status due to poor performance in the National Senior Certificate (NSC) examination, hence the study focused specifically at the experiences of SMT members in trying to improve the NSC results. My study makes a significant contribution towards the improvement and development of NSLA Schools in South Africa who are struggling to meet the demands of the Department of Basic Education and trying to achieve the minimum standards in the NSC examinations. A qualitative research design was employed for this study which was located within the interpretivist paradigm. One NSLA secondary school was chosen with its school management members, comprising of the principal, and four heads of departments, through purposeful sampling. A thorough review of both local and international literature was conducted on the School Management Teams within underperforming schools, and specifically, NSLA Schools in South Africa. Kurt Lewin’s (1953) change theory and the leadership for learning theory formed the theoretical frameworks for this study. Based on the data that was analysed, these were the key findings of this study: The roles and responsibilities of the SMT within the NSLA School differs from other schools based on an overload of management duties and administration related directly to the grade 12 learners and the improvement of the NSC results. The SMT in the NSLA School engage in in-depth analysis and problem solving strategies to improve the NSC results. Most of the management duties of the SMT revolve around the grade 12 learners and their curriculum which has impacted on their instruction time and resulted in stressful working conditions. The SMT also manage serious discipline problems on a daily basis and are subjected to violent learner behaviour and those learners who have drug related problems. In trying to overcome some of the challenges that this NSLA School is currently facing, the SMT plans regular grade 12 intervention classes and host parent meetings. The SMT provide learners with support in the form of mentorship and nutrition for those learners that are affected by the socio-economic ills of the community which they come from. The SMT deals with a host of negative factors that seem to have contributed to the poor NSC pass rates and also low morale and frustration. The SMT of the school hold the progression policy accountable for the mass of learners who get to grade 12, but fail because they are not ready. The Department of Basic Education, besides monitoring and evaluating the SMT, provide very little support to improve the NSC results of the NSLA School. The SMT have also not received specialised training from the Department of Basic Education to equip them for the challenges they currently face in the NSLA School. Based on the research findings, the study makes two recommendations. One recommendation is proposed to the SMT of the NSLA School while the other is directed towards the Department of Basic Education, both with the aim of improving learner academic performances in the NSC examinations

Interdepartmental collaboration between Internal medicine and General surgery to develop and deliver a curriculum in Patient safety and quality to first year surgical residents

Presented as a Poster Presentation at 2020 IUSM Education Day