Assessing Resource-Performance Trade-off of Natural Language Models using Data Envelopment Analysis

Natural language models are often summarized through a high-dimensional set of descriptive metrics including training corpus size, training time, the number of trainable parameters, inference times, and evaluation statistics that assess performance across tasks. The high dimensional nature of these metrics yields challenges with regard to objectively comparing models; in particular it is challenging to assess the trade-off models make between performance and resources (compute time, memory, etc.). We apply Data Envelopment Analysis (DEA) to this problem of assessing the resource-performance trade-off. DEA is a nonparametric method that measures productive efficiency of abstract units that consume one or more inputs and yield at least one output. We recast natural language models as units suitable for DEA, and we show that DEA can be used to create an effective framework for quantifying model performance and efficiency. A central feature of DEA is that it identifies a subset of models that live on an efficient frontier of performance. DEA is also scalable, having been applied to problems with thousands of units. We report empirical results of DEA applied to 14 different language models that have a variety of architectures, and we show that DEA can be used to identify a subset of models that effectively balance resource demands against performance.Comment: 9 pages, 1 figure, Eval4NLP worksho

SOF-MAGTF Capabilities Integrations Analysis and Operational Modeling (Continuation)

NPS NRP Project PosterThis research, working in conjunction with ongoing NPS and MCWL efforts examining Expeditionary Advance Base Operations (EABO), will examine SOF-MAGTF Capabilities Integrations in a future Distributed Maritime Operations (DMO). This effort will focus on exploring, assessing and identify mechanisms that will allow MARSOC to better serve as a bridge for capabilities integration with SOF and deployed MAGTFs to fully maximize the complementary capabilities of each formation. This research will be conducted with multiple Joint Campaign Analysis and Wargaming Applications courses.Marine Corps Forces Special Operations Command (MARSOC)Marine Corps Forces Command (COMMARFORCOM)This research is supported by funding from the Naval Postgraduate School, Naval Research Program (PE 0605853N/2098). https://nps.edu/nrpChief of Naval Operations (CNO)Approved for public release. Distribution is unlimited.

SOF-MAGTF Capabilities Integrations Analysis and Operational Modeling (Continuation)

NPS NRP Technical ReportThis research, working in conjunction with ongoing NPS and MCWL efforts examining Expeditionary Advance Base Operations (EABO), will examine SOF-MAGTF Capabilities Integrations in a future Distributed Maritime Operations (DMO). This effort will focus on exploring, assessing and identify mechanisms that will allow MARSOC to better serve as a bridge for capabilities integration with SOF and deployed MAGTFs to fully maximize the complementary capabilities of each formation. This research will be conducted with multiple Joint Campaign Analysis and Wargaming Applications courses.Marine Corps Forces Special Operations Command (MARSOC)Marine Corps Forces Command (COMMARFORCOM)This research is supported by funding from the Naval Postgraduate School, Naval Research Program (PE 0605853N/2098). https://nps.edu/nrpChief of Naval Operations (CNO)Approved for public release. Distribution is unlimited.

SOF-MAGTF Capabilities Integrations Analysis and Operational Modeling (Continuation)

NPS NRP Executive SummaryThis research, working in conjunction with ongoing NPS and MCWL efforts examining Expeditionary Advance Base Operations (EABO), will examine SOF-MAGTF Capabilities Integrations in a future Distributed Maritime Operations (DMO). This effort will focus on exploring, assessing and identify mechanisms that will allow MARSOC to better serve as a bridge for capabilities integration with SOF and deployed MAGTFs to fully maximize the complementary capabilities of each formation. This research will be conducted with multiple Joint Campaign Analysis and Wargaming Applications courses.Marine Corps Forces Special Operations Command (MARSOC)Marine Corps Forces Command (COMMARFORCOM)This research is supported by funding from the Naval Postgraduate School, Naval Research Program (PE 0605853N/2098). https://nps.edu/nrpChief of Naval Operations (CNO)Approved for public release. Distribution is unlimited.

Assess Intermediate Force Capabilities (IFC) and concept of operations for application during the Competition Phase in an environment of GPC

NPS NRP Executive SummaryIntermediate force capabilities represent a strategic risk mitigation investment that are designed to provide warfighters tools to compete below the level of major armed conflict without losing credibility in the information. This research examines the effects of a set of intermediate force capabilities to assess the strategic impact on a near peer adversary during the competition phase in a 'gray zone' scenario. The effort will seek to gain insights and identify challenges to the employment of IFCs, through several venues, including leveraging defense analysis and operations research department faculty and students and utilizing the NPS warfighting continuum of Joint Campaign Analysis and Wargaming Applications courses.HQMC Plans, Policies & Operations (PP&O)This research is supported by funding from the Naval Postgraduate School, Naval Research Program (PE 0605853N/2098). https://nps.edu/nrpChief of Naval Operations (CNO)Approved for public release. Distribution is unlimited.

Assess Intermediate Force Capabilities (IFC) and concept of operations for application during the Competition Phase in an environment of GPC

NPS NRP Project PosterIntermediate force capabilities represent a strategic risk mitigation investment that are designed to provide warfighters tools to compete below the level of major armed conflict without losing credibility in the information. This research examines the effects of a set of intermediate force capabilities to assess the strategic impact on a near peer adversary during the competition phase in a 'gray zone' scenario. The effort will seek to gain insights and identify challenges to the employment of IFCs, through several venues, including leveraging defense analysis and operations research department faculty and students and utilizing the NPS warfighting continuum of Joint Campaign Analysis and Wargaming Applications courses.HQMC Plans, Policies & Operations (PP&O)This research is supported by funding from the Naval Postgraduate School, Naval Research Program (PE 0605853N/2098). https://nps.edu/nrpChief of Naval Operations (CNO)Approved for public release. Distribution is unlimited.

Assess Intermediate Force Capabilities (IFC) and concept of operations for application during the Competition Phase in an environment of GPC

NPS NRP Technical ReportIntermediate force capabilities represent a strategic risk mitigation investment that are designed to provide warfighters tools to compete below the level of major armed conflict without losing credibility in the information. This research examines the effects of a set of intermediate force capabilities to assess the strategic impact on a near peer adversary during the competition phase in a 'gray zone' scenario. The effort will seek to gain insights and identify challenges to the employment of IFCs, through several venues, including leveraging defense analysis and operations research department faculty and students and utilizing the NPS warfighting continuum of Joint Campaign Analysis and Wargaming Applications courses.HQMC Plans, Policies & Operations (PP&O)This research is supported by funding from the Naval Postgraduate School, Naval Research Program (PE 0605853N/2098). https://nps.edu/nrpChief of Naval Operations (CNO)Approved for public release. Distribution is unlimited.

Anti-allergic Activity of Stem Bark of Myrica esculenta Buch.-Ham. (Myricaceae)

Allergic diseases, such as allergic asthma, are hypersensitivity reactions initiated by immunological mechanisms. Myrica esculenta (M. esculenta) is known traditionally in Ayurveda to possess anti-asthmatic activity. The present investigation was undertaken to evaluate the effect of crude extract of stem bark of M. esculenta (Family Myricaceae, commonly known as Kaiphal) on experimental allergic reactions. Experimental models studied were allergic pleurisy and vascular permeability induced by acetic acid in mice. Pretreatment with M. esculenta (75 mg/kg and 150 mg/kg, p.o.) significantly inhibited the eosinophil accumulation (P < 0.01) respectively in the pleural cavity. M. esculenta (75 and 150 mg/kg, p.o.) significantly inhibited the rise in plasma exudation (57.12% and 59.77%, P < 0.01) induced by acetic acid in mice. These findings demonstrate that the crude extract from the stem bark of M. esculenta possesses antiallergic activity. This activity may be mediated by reducing the release of mediators such as histamine, inhibition of mast cell degranulation, and inhibition of eosinophil accumulation thereby preventing the release of cytokines and chemokines