Attention Scheme Inspired Softmax Regression

Large language models (LLMs) have made transformed changes for human society. One of the key computation in LLMs is the softmax unit. This operation is important in LLMs because it allows the model to generate a distribution over possible next words or phrases, given a sequence of input words. This distribution is then used to select the most likely next word or phrase, based on the probabilities assigned by the model. The softmax unit plays a crucial role in training LLMs, as it allows the model to learn from the data by adjusting the weights and biases of the neural network. In the area of convex optimization such as using central path method to solve linear programming. The softmax function has been used a crucial tool for controlling the progress and stability of potential function [Cohen, Lee and Song STOC 2019, Brand SODA 2020]. In this work, inspired the softmax unit, we define a softmax regression problem. Formally speaking, given a matrix $A \in \mathbb{R}^{n \times d}$ and a vector $b \in \mathbb{R}^n$, the goal is to use greedy type algorithm to solve \begin{align*} \min_{x} \| \langle \exp(Ax), {\bf 1}_n \rangle^{-1} \exp(Ax) - b \|_2^2. \end{align*} In certain sense, our provable convergence result provides theoretical support for why we can use greedy algorithm to train softmax function in practice

GRENADE: Graph-Centric Language Model for Self-Supervised Representation Learning on Text-Attributed Graphs

Self-supervised representation learning on text-attributed graphs, which aims to create expressive and generalizable representations for various downstream tasks, has received increasing research attention lately. However, existing methods either struggle to capture the full extent of structural context information or rely on task-specific training labels, which largely hampers their effectiveness and generalizability in practice. To solve the problem of self-supervised representation learning on text-attributed graphs, we develop a novel Graph-Centric Language model -- GRENADE. Specifically, GRENADE exploits the synergistic effect of both pre-trained language model and graph neural network by optimizing with two specialized self-supervised learning algorithms: graph-centric contrastive learning and graph-centric knowledge alignment. The proposed graph-centric self-supervised learning algorithms effectively help GRENADE to capture informative textual semantics as well as structural context information on text-attributed graphs. Through extensive experiments, GRENADE shows its superiority over state-of-the-art methods. Implementation is available at \url{https://github.com/bigheiniu/GRENADE}.Comment: Findings of EMNLP 202

Shaping a Smart Transportation System for Sustainable Value Co-Creation

The smart transportation system (STS) leverages ubiquitous and networked computing to improve the efficiency of urban mobility. Whilst existing IS work has explored various factors influencing STS development, there is a lack of consideration of how value can be created for building a more sustainable STS. Drawing upon the value co-creation theory and stakeholder theory, we seek to understand the socio-technical shaping of the STS ecosystem and how government, firms and citizens collaboratively create sustainable value for designing and implementing STS initiatives. To reach this aim, we carry out a longitudinal case study over 2016–2018 in Shijiazhuang, China. We offer both theoretical and practical explanations on (i) key value facets with regard to sustainable STS design and implementation; and (ii) a holistic view of iterative value co-creation process pushed by key stakeholders. This study makes particular contributions to the IS, marketing and transportation literature by offering a critical understanding of the social dynamics for shaping a big data-driven STS ecosystem.</p

Filmic exploration in an urban cinema complex design

There is an inextricable link between the creation of films and the development of our built environment. At their most basic level, architecture and cinema have natural inbuilt affinities. Path, construction; script, and production;--architects and filmmakers proceed down the parallel routes to create their works. Meanwhile, film and architecture are simultaneous in character. The succession of scenes in a montage creates a mental space, while architecture provides physical space. Both architecture and film make use of the sequential movements through space. The purpose of the thesis is to explore the parallels and reciprocal relationship between film and architecture, which will help to bridge these two visual and spatial arts, thus enable us to examine our architectural practice from a different perspective. The thesis is composed of literature research and an urban cinema complex design, both of which were integrated as the exploration of the intimate relationship between film and architecture

Decentralized model predictive control of urban drainage systems

This thesis applies a modeling method proposed by previous literature to a specific example and develops control techniques based on this model. The inherent nonlinear behaviors of the drainage systems were accommodated by introducing binary variables and linear inequalities to merge different modes of operation into a single expression. The objective function is constituted by 3 cost functions considering several priorities. Except the normal objective of minimizing overflows, we present two methods of reducing operation costs by harvesting rain power and energy from rainfall collecting locales. Pressure forebay regulates water with large potential energy and generates electricity as the water is directed through hydraulic pumps. Surface aqueduct collects water with high kinetic energy and pushes water through spiral case to generate electricity. Locations with these devices installed induce lower operation cost and thus have higher priorities to be utilized. Once we formulate the water management problem as an optimization problem with specified constraints, we can apply Model Predictive Control (MPC) to compensate for modeling errors and prediction inaccuracies. As we regularly update system states and disturbances information, we achieve our goal of applying real time control to drainage systems. As the system size grows, the system is partitioned into several subsections and each one of them forms a subsystem which makes local decisions based on partial information. The performance of different partition schemes was compared against centralized MPC and open loop controllers. It was shown that even decentralized controllers may suffer performance loss, the computation time was significantly reduced compared with centralized controllers. In rain scenarios with large intensity, the performance loss of decentralized controllers is insignificant compared with the advantage gained by computation time reduced