To understand applications' memory usage details, engineers use instrumented
builds and profiling tools. Both approaches are impractical for use in
production environments or deployed mobile applications. As a result,
developers can gather only high-level memory-related statistics for deployed
software. In our experience, the lack of granular field data makes fixing
performance and reliability-related defects complex and time-consuming. The
software industry needs lightweight solutions to collect detailed data about
applications' memory usage to increase developer productivity. Current research
into memory attribution-related data structures, techniques, and tools is in
the early stages and enables several new research avenues.Comment: 3 pages. To be published in the 45th International Conference on
  Software Engineering (ICSE 2023), May 14 - May 20 2023, Melbourne, Australi

Kudrjavets, Gunnar

Nagappan, Nachiappan

Rastogi, Ayushi

Thomas, Jeffrey

English

arXiv

To understand applications' memory usage details, engineers use instrumented builds and profiling tools. Both approaches are impractical for use in production environments or deployed mobile applications. As a result, developers can gather only high-level memory-related statistics for deployed software. In our experience, the lack of granular field data makes fixing performance and reliability-related defects complex and time-consuming. The software industry needs lightweight solutions to collect detailed data about applications' memory usage to increase developer productivity. Current research into memory attribution-related data structures, techniques, and tools is in the early stages and enables several new research avenues

ARTS repository - University of Groningen

   University of GroningenWho Ate My Memory? Towards Attribution in Memory ManagementKudrjavets, Gunnar; Rastogi, Ayushi; Thomas, Jeffrey; Nagappan, NachiappanPublished in:Proceedings of 45th International Conference on Software Engineering (ICSE 2023)IMPORTANT NOTE: You are advised to consult the publisher's version (publisher's PDF) if you wish to cite fromit. Please check the document version below.Document VersionEarly version, also known as pre-printPublication date:2022Link to publication in University of Groningen/UMCG research databaseCitation for published version (APA):Kudrjavets, G., Rastogi, A., Thomas, J., & Nagappan, N. (2022). Who Ate My Memory? Towards Attributionin Memory Management. Manuscript submitted for publication. In Proceedings of 45th InternationalConference on Software Engineering (ICSE 2023) arXiv. https://arxiv.org/pdf/2212.11866CopyrightOther than for strictly personal use, it is not permitted to download or to forward/distribute the text or part of it without the consent of theauthor(s) and/or copyright holder(s), unless the work is under an open content license (like Creative Commons).The publication may also be distributed here under the terms of Article 25fa of the Dutch Copyright Act, indicated by the “Taverne” license.More information can be found on the University of Groningen website: https://www.rug.nl/library/open-access/self-archiving-pure/taverne-amendment.Take-down policyIf you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediatelyand investigate your claim.Downloaded from the University of Groningen/UMCG research database (Pure): http://www.rug.nl/research/portal. For technical reasons thenumber of authors shown on this cover page is limited to 10 maximum.Download date: 01-11-2023Who Ate My Memory?Towards Attribution in Memory ManagementGunnar Kudrjavets, Ayushi RastogiUniversity of Groningen9712 CP Groningen, Netherlandsg.kudrjavets@rug.nl, a.rastogi@rug.nlJeff Thomas, Nachiappan NagappanMeta Platforms, Inc.Menlo Park, CA 94025, USAjeffdthomas@meta.com, nnachi@meta.comAbstract—To understand applications’ memory usage details,engineers use instrumented builds and profiling tools. Bothapproaches are impractical for use in production environments ordeployed mobile applications. As a result, developers can gatheronly high-level memory-related statistics for deployed software.In our experience, the lack of granular field data makes fixingperformance and reliability-related defects complex and time-consuming. The software industry needs lightweight solutionsto collect detailed data about applications’ memory usage toincrease developer productivity. Current research into memoryattribution-related data structures, techniques, and tools is in theearly stages and enables several new research avenues.Index Terms—Allocator, memory attribution, memory taggingI. BACKGROUND AND MOTIVATIONThis paper is motivated by our industry experience whileworking on problems related to optimizing and trackingmemory usage in both commercial and open-source softwareproducts. We observe the presence of a homogeneous set ofissues related to attribution (“How much memory is used bya particular component?”) and accountability (“What compo-nent is exceeding their memory budget?”). Answering thosequestions correctly and continuously is necessary to ensure theapplication’s performance and reliability.Software performance is associated with user satisfactionand how actively users engage with an application [1]. Onecriterion that impacts an application’s performance is itsmemory usage [2]. Characteristics such as the total size ofallocations, allocation rate, or the allocated memory typeserve as metrics describing how efficiently an applicationuses memory. Memory is one of the most precious systemresources on mobile platforms that do not support paging, suchas iOS [3]. A suggested design pattern for environments wherememory is limited is to make each component responsible foraccounting for its own memory usage [4].A detailed understanding of memory usage is also requiredto increase reliability. A variety of modern kernels and operat-ing systems based on them, such as Android, FreeBSD, iOS,or Linux, use a system component called an out-of-memorykiller [5], [6], [7]. An out-of-memory killer is responsible forterminating processes when their memory usage exceeds acertain quota, or excessive memory usage puts the stabilityof an entire operating system in danger. Engineers need tounderstand the application’s memory footprint in detail toavoid premature termination.In industry, we note that organizations develop their in-house ecosystems to solve company-specific memory attri-bution issues. A similar trend is present with open-sourcesoftware where projects such as Mozilla Firefox have aproduct-specific extensive framework to track and attributememory usage [8]. We observe the shortcomings in memoryattribution capabilities and tools even in popular kernels suchas Linux that has three decades of real-world usage and anactive developer community [9].Most of the research related to memory management fo-cuses on increasing the performance of custom allocators [10],[11], [12], [13], [14] or ensuring their correctness [15], [16].The limited existing research into memory attribution is inthe early stages, involves invasive profiling techniques, and isspecific to the Message Passing Interface [17].II. STATE OF AFFAIRS“. . . how it is that memory can be allocated without thekernel knowing where it went. The problem is that thetracking infrastructure just isn’t there.”— 2022 Linux Storage, Filesystem, MM and BPF SummitModern operating systems enable tracking memory usageat the process level [18], [19], [20], [21]. However, thegranularity at the process level is insufficient for engineers toperform efficient debugging or performance engineering [2].A common task that engineers encounter in practice is deter-mining what portion of the allocated amount of memory isconsumed by a specific component, scenario, or a subsystem.In the context of performance or reliability engineering thatcan mean a dynamic library, feature, function, or thread. Dataabout memory consumption is necessary to determine (a) whatcomponents to optimize, (b) if performance regressions arepresent, and (c) what performance tuning techniques to use.It is also necessary to understand how memory usage changesover time. For example, after a new commit, updates to thetoolset, such as a compiler, change in dependent libraries, orchange in some application’s configuration parameters.The standard approach to acquiring details about an ap-plication’s memory usage is to use a profiler. When anarXiv:2212.11866v1  [cs.SE]  22 Dec 2022application is executed under the profiler, such as Visual StudioProfiler [22] or Xcode [23], then the profiler tracks eachallocation and its source. An engineer can later filter, query,and visualize the resulting dataset. For example, the profilermay record the complete stack information coupled with theallocation size and originating dynamic library. Gathering theprofiler data is time-consuming, can require the application tobe compiled with specific flags (in case an instrumentationframework such as Valgrind [24] is used), and requires asignificant amount of disk storage to store the entire historyof allocations. Above all, this approach is not practical out-side the application’s development environment. The overheadcaused by instrumentation can cause an application to executean order of magnitude slower.A. Existing attribution techniquesTwo main approaches to attribute memory usage in sourcecode exist: annotating each call to allocate memory andannotating each scope. We describe the existing usage patternsfor each technique.In kernel mode, Windows uses pool tagging [19]. Eachdriver can specify a pool tag when it requests to allocatememory [25], [26], [27]. Depending on a size of a driver,it can use one or more pool tags to differentiate betweenvarious subsystems. Pool tags also have another function inWindows—kernel crashes if the driver does not release all theallocations with a specific tag when the driver is unloaded [19].On macOS, a caller can use a function such as OSMallocand associate each allocation with an opaque tag. However,that tag is used only for reference counting [28]. The tagcount is increased by one each time a specific tag is al-located. In user mode, macOS also enables passing customtags generated by VM_MAKE_TAG macro to functions such asvm_allocate [28]. Listing 1 shows a sample usage patternwhen a custom tag is associated with an allocation.Listing 1TAGGED ALLOCATION IN MACOS./* An allocation billed to networking. */err = vm_allocate(..., VM_MEMORY_LIBNETWORK);One of the FreeBSD ports is a basic heap memory account-ing system libpdel [29] that similarly requires each callerto specify a “memory type” in a form of a string.As a result of annotations, the application can duringthe runtime enumerate its virtual memory, and gather thedistribution and size of allocations per a different memory tagor a type.Hierarchically tracking memory usage by annotating sourcecode is another option. Developers need to attribute eachscope with a specific tag. All the allocator activity in thatscope and its children will be attributed to that tag. Listing 2displays how all the allocations in the function bar() and itschildren will be “billed” to the tag foo using the example ofTfMallocTag tagging system [30].Listing 2ALLOCATION TRACKING USING THE TFMALLOCTAG SYSTEM.void bar() {TfAutoMallocTag tag("foo");funcA();funcB();}B. Limitations of current techniquesAll these approaches have constraints because they require(a) annotation of each allocation or scope, (b) complete sourcecode to be available, and (c) hierarchical tracking needs tointercept all the allocations in the current process. However,a standard application has dependencies, such as system orthird-party libraries. Without modifying the dependencies,the ability to track allocations in detail is limited to theapplication’s “own code.”We are unaware of any operating systems, languages, ortools that enable engineers to query and keep track of memoryattribution at a granular level without sacrificing the applica-tion’s performance. The exploration of possible solutions isstill in the early stages. The most recent proposal for memoryallocation tracking in Linux is from August 2022 [31]. Thelack of these facilities negatively impacts each non-trivialsoftware project. Understanding the application’s memoryusage in detail is realistic only in the development environ-ment. However, in our experience, predicting or debugging anapplication’s behavior in the production environment based onthe data from the development environment is ineffective.III. FUTURE RESEARCH DIRECTIONSThe choice of the abstraction layers and a variety in theproblem space enables multiple research avenues. Ideally, thesupport for memory attribution will be integrated throughoutthe operating system, memory allocator, and a runtime librarysuch as the GNU C Library (glibc) [32]. Support for program-ming languages used for systems programming (e.g., C, C++,Rust) is imperative.The desired solution to improve engineers’ ability to at-tribute memory (a) can be enabled and disabled on demand,(b) has a minor performance overhead and will be usable inthe production environment, (c) enables querying the memoryattribution during runtime, and (d) has a well-designed set ofAPIs.One potential practical approach we envision in user modeis the usage of custom memory allocators [14], [33], [34],[13] to assist with the attribution. Custom allocators suchas jemalloc intercept each allocation request made in thecontext of a process. Therefore, the intercept mechanismcan track all the metadata such as allocation size, currentthread, timestamp, or specific flags passed to the function. Theintercept mechanism can use either data from the applicationthat specifies the current attribution scope, classify callersbased on sampling the call stack, or use some other techniques.REFERENCES[1] M. Hort, M. Kechagia, F. Sarro, and M. Harman, “A Survey ofPerformance Optimization for Mobile Applications,” IEEE Transactionson Software Engineering, vol. 48, no. 8, pp. 2879–2904, 2022. [Online].Available: https://doi.org/10.1109/TSE.2021.3071193[2] B. Gregg, Systems Performance: Enterprise and the Cloud, 2nd ed., ser.Addison-Wesley professional computing series. Boston, MA, USA:Addison-Wesley, 2020.[3] J. Levin, *OS internals. Volume 1: User space, 2nd ed. Edison, NJ,USA: Technologeeks.com, 2017.[4] J. Noble and C. Weir, Small Memory Software, ser. Software PatternsSeries. Boston, MA, USA: Addison Wesley, Nov. 2000.[5] Android OS Documentation. (2022, Oct.) Low Memory Killer Daemon.[Online]. Available: https://source.android.com/devices/tech/perf/lmkd[6] D. Xu. (2018, Jul.) Open-sourcing oomd, a new approach tohandling OOMs. Meta Platforms, Inc. [Online]. Available: https://engineering.fb.com/2018/07/19/production-engineering/oomd/[7] Eklektix, Inc. (2022) OOM killer. [Online]. Available: https://lwn.net/Kernel/Index/#OOM_killer[8] Mozilla Foundation. (2022) Performance—Firefox Source Docsdocumentation. [Online]. Available: https://firefox-source-docs.mozilla.org/performance/index.html[9] J. Corbet. (2022, May) Better tools for out-of-memory debugging.[Online]. Available: https://lwn.net/Articles/894546/[10] D. Leijen, B. Zorn, and L. de Moura, “Mimalloc: Free ListSharding in Action,” Microsoft, Tech. Rep. MSR-TR-2019-18, Jun.2019. [Online]. Available: https://www.microsoft.com/en-us/research/publication/mimalloc-free-list-sharding-in-action/[11] M. Jansson. (2017) rpmalloc—General Purpose Memory Allocator.[Online]. Available: https://github.com/mjansson/rpmalloc[12] P. Liétar, T. Butler, S. Clebsch, S. Drossopoulou, J. Franco,M. J. Parkinson, A. Shamis, C. M. Wintersteiger, and D. Chisnall,“Snmalloc: A Message Passing Allocator,” in Proceedings ofthe 2019 ACM SIGPLAN International Symposium on MemoryManagement, ser. ISMM 2019. New York, NY, USA: Associationfor Computing Machinery, 2019, pp. 122–135. [Online]. Available:https://doi.org/10.1145/3315573.3329980[13] S. Lee, T. Johnson, and E. Raman, “Feedback Directed Optimizationof TCMalloc,” in Proceedings of the Workshop on Memory SystemsPerformance and Correctness, ser. MSPC ’14. New York, NY, USA:Association for Computing Machinery, 2014. [Online]. Available:https://doi.org/10.1145/2618128.2618131[14] J. Evans, “A scalable concurrent malloc(3) implementation forFreeBSD,” in Proceedings of the BSDCan Conference, Universityof Ottawa, Ottawa, Canada, 2006. [Online]. Available: https://www.bsdcan.org/2006/papers/jemalloc.pdf[15] T. Ball, S. Chaki, and S. K. Rajamani, “Parameterized Verificationof Multithreaded Software Libraries,” in Proceedings of the 7thInternational Conference on Tools and Algorithms for the Constructionand Analysis of Systems, ser. TACAS 2001. Berlin, Heidelberg:Springer-Verlag, 2001, p. 158–173. [Online]. Available: https://doi.org/10.1007/3-540-45319-9_12[16] A. W. Appel and D. A. Naumann, “Verified Sequential Malloc/Free,”in Proceedings of the 2020 ACM SIGPLAN International Symposiumon Memory Management, ser. ISMM 2020. New York, NY, USA:Association for Computing Machinery, 2020, p. 48–59. [Online].Available: https://doi.org/10.1145/3381898.3397211[17] S. K. Gutiérrez, D. C. Arnold, K. Davis, and P. McCormick,“On the Memory Attribution Problem: A Solution and Case StudyUsing MPI,” Journal on Concurrency and Computation: Practice andExperience, vol. 32, no. 3, p. e5159, Feb. 2019. [Online]. Available:https://doi.org/10.1002/cpe.5159[18] A. S. Tanenbaum, Modern Operating Systems, 2nd ed. Upper SaddleRiver, NJ, USA: Prentice Hall, 2001.[19] M. E. Russinovich, D. A. Solomon, and A. Ionescu, Windows Inter-nals, 6th ed. Redmond, WA, USA: Microsoft Press, 2012, OCLC:ocn753301527.[20] R. Love, Linux Kernel Development, 2nd ed. Indianapolis, IN, USA:Novell Press, 2005.[21] W. Stallings, Operating Systems: Internals and Design Principles,6th ed. Upper Saddle River, NJ, USA: Pearson/Prentice Hall, 2009.[22] Microsoft Corporation. (2022) Measure performance in Visual Studio—Visual Studio (Windows). [Online]. Available: https://docs.microsoft.com/en-us/visualstudio/profiling/[23] Apple Inc. (2022) Xcode—Features. [Online]. Available: https://developer.apple.com/xcode/features/[24] Valgrind™ Developers. (2022) Valgrind Home. [Online]. Available:https://valgrind.org/[25] W. Oney, Programming the Microsoft Windows Driver Model, ser.Microsoft programming series. Redmond, WA, USA: Microsoft Press,Oct. 1999.[26] E. N. Dekker and J. M. Newcomer, Developing Windows NT devicedrivers. Boston, MA, USA: Addison-Wesley Educational, Mar. 1999.[27] P. Orwick and G. Smith, Developing drivers with the Windows DriverFoundation. Redmond, WA, USA: Microsoft Press, Apr. 2007.[28] A. Singh, Mac OS X Internals: a Systems Approach. Boston, MA,USA: Addison-Wesley Professional, 2016, OCLC: 1005337597.[29] A. Cobbs and Packet Design, LLC. (2002, Apr.)typed_mem—heap memory accounting system. [Online]. Avail-able: https://www.freebsd.org/cgi/man.cgi?query=FREE&manpath=FreeBSD+13.1-RELEASE+and+Ports[30] Pixar. (2022, Jul.) Universal Scene Description: The TfMallocTagMemory Tagging System. [Online]. Available: https://graphics.pixar.com/usd/dev/api/page_tf__malloc_tag.html[31] S. Baghdasaryan. (2022, Aug.) Code tagging frameworkand applications. [Online]. Available: https://lore.kernel.org/all/20220830214919.53220-1-surenb@google.com/[32] Free Software Foundation. (2022) The GNU C Library. [Online].Available: https://www.gnu.org/software/libc/[33] J. Evans, “Tick Tock, malloc Needs a Clock,” in Applicative 2015, ser.Applicative 2015. New York, NY, USA: Association for ComputingMachinery, 2015. [Online]. Available: https://doi.org/10.1145/2742580.2742807[34] ——. (2011, Jan.) Scalable memory allocation using jemalloc.[Online]. Available: https://engineering.fb.com/2011/01/03/core-data/scalable-memory-allocation-using-jemalloc/

Who Ate My Memory? Towards Attribution in Memory Management

Proceedings - University of Groningen

   University of GroningenWho Ate My Memory? Towards Attribution in Memory ManagementKudrjavets, Gunnar; Rastogi, Ayushi; Thomas, Jeffrey; Nagappan, NachiappanPublished in:Proceedings of 45th International Conference on Software Engineering (ICSE 2023)IMPORTANT NOTE: You are advised to consult the publisher's version (publisher's PDF) if you wish to cite fromit. Please check the document version below.Document VersionEarly version, also known as pre-printPublication date:2022Link to publication in University of Groningen/UMCG research databaseCitation for published version (APA):Kudrjavets, G., Rastogi, A., Thomas, J., & Nagappan, N. (2022). Who Ate My Memory? Towards Attributionin Memory Management. Manuscript submitted for publication. In Proceedings of 45th InternationalConference on Software Engineering (ICSE 2023) arXiv. https://arxiv.org/pdf/2212.11866CopyrightOther than for strictly personal use, it is not permitted to download or to forward/distribute the text or part of it without the consent of theauthor(s) and/or copyright holder(s), unless the work is under an open content license (like Creative Commons).The publication may also be distributed here under the terms of Article 25fa of the Dutch Copyright Act, indicated by the “Taverne” license.More information can be found on the University of Groningen website: https://www.rug.nl/library/open-access/self-archiving-pure/taverne-amendment.Take-down policyIf you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediatelyand investigate your claim.Downloaded from the University of Groningen/UMCG research database (Pure): http://www.rug.nl/research/portal. For technical reasons thenumber of authors shown on this cover page is limited to 10 maximum.Download date: 12-09-2023Who Ate My Memory?Towards Attribution in Memory ManagementGunnar Kudrjavets, Ayushi RastogiUniversity of Groningen9712 CP Groningen, Netherlandsg.kudrjavets@rug.nl, a.rastogi@rug.nlJeff Thomas, Nachiappan NagappanMeta Platforms, Inc.Menlo Park, CA 94025, USAjeffdthomas@meta.com, nnachi@meta.comAbstract—To understand applications’ memory usage details,engineers use instrumented builds and profiling tools. Bothapproaches are impractical for use in production environments ordeployed mobile applications. As a result, developers can gatheronly high-level memory-related statistics for deployed software.In our experience, the lack of granular field data makes fixingperformance and reliability-related defects complex and time-consuming. The software industry needs lightweight solutionsto collect detailed data about applications’ memory usage toincrease developer productivity. Current research into memoryattribution-related data structures, techniques, and tools is in theearly stages and enables several new research avenues.Index Terms—Allocator, memory attribution, memory taggingI. BACKGROUND AND MOTIVATIONThis paper is motivated by our industry experience whileworking on problems related to optimizing and trackingmemory usage in both commercial and open-source softwareproducts. We observe the presence of a homogeneous set ofissues related to attribution (“How much memory is used bya particular component?”) and accountability (“What compo-nent is exceeding their memory budget?”). Answering thosequestions correctly and continuously is necessary to ensure theapplication’s performance and reliability.Software performance is associated with user satisfactionand how actively users engage with an application [1]. Onecriterion that impacts an application’s performance is itsmemory usage [2]. Characteristics such as the total size ofallocations, allocation rate, or the allocated memory typeserve as metrics describing how efficiently an applicationuses memory. Memory is one of the most precious systemresources on mobile platforms that do not support paging, suchas iOS [3]. A suggested design pattern for environments wherememory is limited is to make each component responsible foraccounting for its own memory usage [4].A detailed understanding of memory usage is also requiredto increase reliability. A variety of modern kernels and operat-ing systems based on them, such as Android, FreeBSD, iOS,or Linux, use a system component called an out-of-memorykiller [5], [6], [7]. An out-of-memory killer is responsible forterminating processes when their memory usage exceeds acertain quota, or excessive memory usage puts the stabilityof an entire operating system in danger. Engineers need tounderstand the application’s memory footprint in detail toavoid premature termination.In industry, we note that organizations develop their in-house ecosystems to solve company-specific memory attri-bution issues. A similar trend is present with open-sourcesoftware where projects such as Mozilla Firefox have aproduct-specific extensive framework to track and attributememory usage [8]. We observe the shortcomings in memoryattribution capabilities and tools even in popular kernels suchas Linux that has three decades of real-world usage and anactive developer community [9].Most of the research related to memory management fo-cuses on increasing the performance of custom allocators [10],[11], [12], [13], [14] or ensuring their correctness [15], [16].The limited existing research into memory attribution is inthe early stages, involves invasive profiling techniques, and isspecific to the Message Passing Interface [17].II. STATE OF AFFAIRS“. . . how it is that memory can be allocated without thekernel knowing where it went. The problem is that thetracking infrastructure just isn’t there.”— 2022 Linux Storage, Filesystem, MM and BPF SummitModern operating systems enable tracking memory usageat the process level [18], [19], [20], [21]. However, thegranularity at the process level is insufficient for engineers toperform efficient debugging or performance engineering [2].A common task that engineers encounter in practice is deter-mining what portion of the allocated amount of memory isconsumed by a specific component, scenario, or a subsystem.In the context of performance or reliability engineering thatcan mean a dynamic library, feature, function, or thread. Dataabout memory consumption is necessary to determine (a) whatcomponents to optimize, (b) if performance regressions arepresent, and (c) what performance tuning techniques to use.It is also necessary to understand how memory usage changesover time. For example, after a new commit, updates to thetoolset, such as a compiler, change in dependent libraries, orchange in some application’s configuration parameters.The standard approach to acquiring details about an ap-plication’s memory usage is to use a profiler. When anarXiv:2212.11866v1  [cs.SE]  22 Dec 2022application is executed under the profiler, such as Visual StudioProfiler [22] or Xcode [23], then the profiler tracks eachallocation and its source. An engineer can later filter, query,and visualize the resulting dataset. For example, the profilermay record the complete stack information coupled with theallocation size and originating dynamic library. Gathering theprofiler data is time-consuming, can require the application tobe compiled with specific flags (in case an instrumentationframework such as Valgrind [24] is used), and requires asignificant amount of disk storage to store the entire historyof allocations. Above all, this approach is not practical out-side the application’s development environment. The overheadcaused by instrumentation can cause an application to executean order of magnitude slower.A. Existing attribution techniquesTwo main approaches to attribute memory usage in sourcecode exist: annotating each call to allocate memory andannotating each scope. We describe the existing usage patternsfor each technique.In kernel mode, Windows uses pool tagging [19]. Eachdriver can specify a pool tag when it requests to allocatememory [25], [26], [27]. Depending on a size of a driver,it can use one or more pool tags to differentiate betweenvarious subsystems. Pool tags also have another function inWindows—kernel crashes if the driver does not release all theallocations with a specific tag when the driver is unloaded [19].On macOS, a caller can use a function such as OSMallocand associate each allocation with an opaque tag. However,that tag is used only for reference counting [28]. The tagcount is increased by one each time a specific tag is al-located. In user mode, macOS also enables passing customtags generated by VM_MAKE_TAG macro to functions such asvm_allocate [28]. Listing 1 shows a sample usage patternwhen a custom tag is associated with an allocation.Listing 1TAGGED ALLOCATION IN MACOS./* An allocation billed to networking. */err = vm_allocate(..., VM_MEMORY_LIBNETWORK);One of the FreeBSD ports is a basic heap memory account-ing system libpdel [29] that similarly requires each callerto specify a “memory type” in a form of a string.As a result of annotations, the application can duringthe runtime enumerate its virtual memory, and gather thedistribution and size of allocations per a different memory tagor a type.Hierarchically tracking memory usage by annotating sourcecode is another option. Developers need to attribute eachscope with a specific tag. All the allocator activity in thatscope and its children will be attributed to that tag. Listing 2displays how all the allocations in the function bar() and itschildren will be “billed” to the tag foo using the example ofTfMallocTag tagging system [30].Listing 2ALLOCATION TRACKING USING THE TFMALLOCTAG SYSTEM.void bar() {TfAutoMallocTag tag("foo");funcA();funcB();}B. Limitations of current techniquesAll these approaches have constraints because they require(a) annotation of each allocation or scope, (b) complete sourcecode to be available, and (c) hierarchical tracking needs tointercept all the allocations in the current process. However,a standard application has dependencies, such as system orthird-party libraries. Without modifying the dependencies,the ability to track allocations in detail is limited to theapplication’s “own code.”We are unaware of any operating systems, languages, ortools that enable engineers to query and keep track of memoryattribution at a granular level without sacrificing the applica-tion’s performance. The exploration of possible solutions isstill in the early stages. The most recent proposal for memoryallocation tracking in Linux is from August 2022 [31]. Thelack of these facilities negatively impacts each non-trivialsoftware project. Understanding the application’s memoryusage in detail is realistic only in the development environ-ment. However, in our experience, predicting or debugging anapplication’s behavior in the production environment based onthe data from the development environment is ineffective.III. FUTURE RESEARCH DIRECTIONSThe choice of the abstraction layers and a variety in theproblem space enables multiple research avenues. Ideally, thesupport for memory attribution will be integrated throughoutthe operating system, memory allocator, and a runtime librarysuch as the GNU C Library (glibc) [32]. Support for program-ming languages used for systems programming (e.g., C, C++,Rust) is imperative.The desired solution to improve engineers’ ability to at-tribute memory (a) can be enabled and disabled on demand,(b) has a minor performance overhead and will be usable inthe production environment, (c) enables querying the memoryattribution during runtime, and (d) has a well-designed set ofAPIs.One potential practical approach we envision in user modeis the usage of custom memory allocators [14], [33], [34],[13] to assist with the attribution. Custom allocators suchas jemalloc intercept each allocation request made in thecontext of a process. Therefore, the intercept mechanismcan track all the metadata such as allocation size, currentthread, timestamp, or specific flags passed to the function. Theintercept mechanism can use either data from the applicationthat specifies the current attribution scope, classify callersbased on sampling the call stack, or use some other techniques.REFERENCES[1] M. Hort, M. Kechagia, F. Sarro, and M. Harman, “A Survey ofPerformance Optimization for Mobile Applications,” IEEE Transactionson Software Engineering, vol. 48, no. 8, pp. 2879–2904, 2022. [Online].Available: https://doi.org/10.1109/TSE.2021.3071193[2] B. Gregg, Systems Performance: Enterprise and the Cloud, 2nd ed., ser.Addison-Wesley professional computing series. Boston, MA, USA:Addison-Wesley, 2020.[3] J. Levin, *OS internals. Volume 1: User space, 2nd ed. Edison, NJ,USA: Technologeeks.com, 2017.[4] J. Noble and C. Weir, Small Memory Software, ser. Software PatternsSeries. Boston, MA, USA: Addison Wesley, Nov. 2000.[5] Android OS Documentation. (2022, Oct.) Low Memory Killer Daemon.[Online]. Available: https://source.android.com/devices/tech/perf/lmkd[6] D. Xu. (2018, Jul.) Open-sourcing oomd, a new approach tohandling OOMs. Meta Platforms, Inc. [Online]. Available: https://engineering.fb.com/2018/07/19/production-engineering/oomd/[7] Eklektix, Inc. (2022) OOM killer. [Online]. Available: https://lwn.net/Kernel/Index/#OOM_killer[8] Mozilla Foundation. (2022) Performance—Firefox Source Docsdocumentation. [Online]. Available: https://firefox-source-docs.mozilla.org/performance/index.html[9] J. Corbet. (2022, May) Better tools for out-of-memory debugging.[Online]. Available: https://lwn.net/Articles/894546/[10] D. Leijen, B. Zorn, and L. de Moura, “Mimalloc: Free ListSharding in Action,” Microsoft, Tech. Rep. MSR-TR-2019-18, Jun.2019. [Online]. Available: https://www.microsoft.com/en-us/research/publication/mimalloc-free-list-sharding-in-action/[11] M. Jansson. (2017) rpmalloc—General Purpose Memory Allocator.[Online]. Available: https://github.com/mjansson/rpmalloc[12] P. Liétar, T. Butler, S. Clebsch, S. Drossopoulou, J. Franco,M. J. Parkinson, A. Shamis, C. M. Wintersteiger, and D. Chisnall,“Snmalloc: A Message Passing Allocator,” in Proceedings ofthe 2019 ACM SIGPLAN International Symposium on MemoryManagement, ser. ISMM 2019. New York, NY, USA: Associationfor Computing Machinery, 2019, pp. 122–135. [Online]. Available:https://doi.org/10.1145/3315573.3329980[13] S. Lee, T. Johnson, and E. Raman, “Feedback Directed Optimizationof TCMalloc,” in Proceedings of the Workshop on Memory SystemsPerformance and Correctness, ser. MSPC ’14. New York, NY, USA:Association for Computing Machinery, 2014. [Online]. Available:https://doi.org/10.1145/2618128.2618131[14] J. Evans, “A scalable concurrent malloc(3) implementation forFreeBSD,” in Proceedings of the BSDCan Conference, Universityof Ottawa, Ottawa, Canada, 2006. [Online]. Available: https://www.bsdcan.org/2006/papers/jemalloc.pdf[15] T. Ball, S. Chaki, and S. K. Rajamani, “Parameterized Verificationof Multithreaded Software Libraries,” in Proceedings of the 7thInternational Conference on Tools and Algorithms for the Constructionand Analysis of Systems, ser. TACAS 2001. Berlin, Heidelberg:Springer-Verlag, 2001, p. 158–173. [Online]. Available: https://doi.org/10.1007/3-540-45319-9_12[16] A. W. Appel and D. A. Naumann, “Verified Sequential Malloc/Free,”in Proceedings of the 2020 ACM SIGPLAN International Symposiumon Memory Management, ser. ISMM 2020. New York, NY, USA:Association for Computing Machinery, 2020, p. 48–59. [Online].Available: https://doi.org/10.1145/3381898.3397211[17] S. K. Gutiérrez, D. C. Arnold, K. Davis, and P. McCormick,“On the Memory Attribution Problem: A Solution and Case StudyUsing MPI,” Journal on Concurrency and Computation: Practice andExperience, vol. 32, no. 3, p. e5159, Feb. 2019. [Online]. Available:https://doi.org/10.1002/cpe.5159[18] A. S. Tanenbaum, Modern Operating Systems, 2nd ed. Upper SaddleRiver, NJ, USA: Prentice Hall, 2001.[19] M. E. Russinovich, D. A. Solomon, and A. Ionescu, Windows Inter-nals, 6th ed. Redmond, WA, USA: Microsoft Press, 2012, OCLC:ocn753301527.[20] R. Love, Linux Kernel Development, 2nd ed. Indianapolis, IN, USA:Novell Press, 2005.[21] W. Stallings, Operating Systems: Internals and Design Principles,6th ed. Upper Saddle River, NJ, USA: Pearson/Prentice Hall, 2009.[22] Microsoft Corporation. (2022) Measure performance in Visual Studio—Visual Studio (Windows). [Online]. Available: https://docs.microsoft.com/en-us/visualstudio/profiling/[23] Apple Inc. (2022) Xcode—Features. [Online]. Available: https://developer.apple.com/xcode/features/[24] Valgrind™ Developers. (2022) Valgrind Home. [Online]. Available:https://valgrind.org/[25] W. Oney, Programming the Microsoft Windows Driver Model, ser.Microsoft programming series. Redmond, WA, USA: Microsoft Press,Oct. 1999.[26] E. N. Dekker and J. M. Newcomer, Developing Windows NT devicedrivers. Boston, MA, USA: Addison-Wesley Educational, Mar. 1999.[27] P. Orwick and G. Smith, Developing drivers with the Windows DriverFoundation. Redmond, WA, USA: Microsoft Press, Apr. 2007.[28] A. Singh, Mac OS X Internals: a Systems Approach. Boston, MA,USA: Addison-Wesley Professional, 2016, OCLC: 1005337597.[29] A. Cobbs and Packet Design, LLC. (2002, Apr.)typed_mem—heap memory accounting system. [Online]. Avail-able: https://www.freebsd.org/cgi/man.cgi?query=FREE&manpath=FreeBSD+13.1-RELEASE+and+Ports[30] Pixar. (2022, Jul.) Universal Scene Description: The TfMallocTagMemory Tagging System. [Online]. Available: https://graphics.pixar.com/usd/dev/api/page_tf__malloc_tag.html[31] S. Baghdasaryan. (2022, Aug.) Code tagging frameworkand applications. [Online]. Available: https://lore.kernel.org/all/20220830214919.53220-1-surenb@google.com/[32] Free Software Foundation. (2022) The GNU C Library. [Online].Available: https://www.gnu.org/software/libc/[33] J. Evans, “Tick Tock, malloc Needs a Clock,” in Applicative 2015, ser.Applicative 2015. New York, NY, USA: Association for ComputingMachinery, 2015. [Online]. Available: https://doi.org/10.1145/2742580.2742807[34] ——. (2011, Jan.) Scalable memory allocation using jemalloc.[Online]. Available: https://engineering.fb.com/2011/01/03/core-data/scalable-memory-allocation-using-jemalloc/

University of Groningen

   University of GroningenWho Ate My Memory? Towards Attribution in Memory ManagementKudrjavets, Gunnar; Rastogi, Ayushi; Thomas, Jeffrey; Nagappan, NachiappanPublished in:Proceedings of 45th International Conference on Software Engineering (ICSE 2023)IMPORTANT NOTE: You are advised to consult the publisher's version (publisher's PDF) if you wish to cite fromit. Please check the document version below.Document VersionEarly version, also known as pre-printPublication date:2022Link to publication in University of Groningen/UMCG research databaseCitation for published version (APA):Kudrjavets, G., Rastogi, A., Thomas, J., & Nagappan, N. (2022). Who Ate My Memory? Towards Attributionin Memory Management. Manuscript submitted for publication. In Proceedings of 45th InternationalConference on Software Engineering (ICSE 2023) arXiv. https://arxiv.org/pdf/2212.11866CopyrightOther than for strictly personal use, it is not permitted to download or to forward/distribute the text or part of it without the consent of theauthor(s) and/or copyright holder(s), unless the work is under an open content license (like Creative Commons).The publication may also be distributed here under the terms of Article 25fa of the Dutch Copyright Act, indicated by the “Taverne” license.More information can be found on the University of Groningen website: https://www.rug.nl/library/open-access/self-archiving-pure/taverne-amendment.Take-down policyIf you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediatelyand investigate your claim.Downloaded from the University of Groningen/UMCG research database (Pure): http://www.rug.nl/research/portal. For technical reasons thenumber of authors shown on this cover page is limited to 10 maximum.Download date: 15-02-2023Who Ate My Memory?Towards Attribution in Memory ManagementGunnar Kudrjavets, Ayushi RastogiUniversity of Groningen9712 CP Groningen, Netherlandsg.kudrjavets@rug.nl, a.rastogi@rug.nlJeff Thomas, Nachiappan NagappanMeta Platforms, Inc.Menlo Park, CA 94025, USAjeffdthomas@meta.com, nnachi@meta.comAbstract—To understand applications’ memory usage details,engineers use instrumented builds and profiling tools. Bothapproaches are impractical for use in production environments ordeployed mobile applications. As a result, developers can gatheronly high-level memory-related statistics for deployed software.In our experience, the lack of granular field data makes fixingperformance and reliability-related defects complex and time-consuming. The software industry needs lightweight solutionsto collect detailed data about applications’ memory usage toincrease developer productivity. Current research into memoryattribution-related data structures, techniques, and tools is in theearly stages and enables several new research avenues.Index Terms—Allocator, memory attribution, memory taggingI. BACKGROUND AND MOTIVATIONThis paper is motivated by our industry experience whileworking on problems related to optimizing and trackingmemory usage in both commercial and open-source softwareproducts. We observe the presence of a homogeneous set ofissues related to attribution (“How much memory is used bya particular component?”) and accountability (“What compo-nent is exceeding their memory budget?”). Answering thosequestions correctly and continuously is necessary to ensure theapplication’s performance and reliability.Software performance is associated with user satisfactionand how actively users engage with an application [1]. Onecriterion that impacts an application’s performance is itsmemory usage [2]. Characteristics such as the total size ofallocations, allocation rate, or the allocated memory typeserve as metrics describing how efficiently an applicationuses memory. Memory is one of the most precious systemresources on mobile platforms that do not support paging, suchas iOS [3]. A suggested design pattern for environments wherememory is limited is to make each component responsible foraccounting for its own memory usage [4].A detailed understanding of memory usage is also requiredto increase reliability. A variety of modern kernels and operat-ing systems based on them, such as Android, FreeBSD, iOS,or Linux, use a system component called an out-of-memorykiller [5], [6], [7]. An out-of-memory killer is responsible forterminating processes when their memory usage exceeds acertain quota, or excessive memory usage puts the stabilityof an entire operating system in danger. Engineers need tounderstand the application’s memory footprint in detail toavoid premature termination.In industry, we note that organizations develop their in-house ecosystems to solve company-specific memory attri-bution issues. A similar trend is present with open-sourcesoftware where projects such as Mozilla Firefox have aproduct-specific extensive framework to track and attributememory usage [8]. We observe the shortcomings in memoryattribution capabilities and tools even in popular kernels suchas Linux that has three decades of real-world usage and anactive developer community [9].Most of the research related to memory management fo-cuses on increasing the performance of custom allocators [10],[11], [12], [13], [14] or ensuring their correctness [15], [16].The limited existing research into memory attribution is inthe early stages, involves invasive profiling techniques, and isspecific to the Message Passing Interface [17].II. STATE OF AFFAIRS“. . . how it is that memory can be allocated without thekernel knowing where it went. The problem is that thetracking infrastructure just isn’t there.”— 2022 Linux Storage, Filesystem, MM and BPF SummitModern operating systems enable tracking memory usageat the process level [18], [19], [20], [21]. However, thegranularity at the process level is insufficient for engineers toperform efficient debugging or performance engineering [2].A common task that engineers encounter in practice is deter-mining what portion of the allocated amount of memory isconsumed by a specific component, scenario, or a subsystem.In the context of performance or reliability engineering thatcan mean a dynamic library, feature, function, or thread. Dataabout memory consumption is necessary to determine (a) whatcomponents to optimize, (b) if performance regressions arepresent, and (c) what performance tuning techniques to use.It is also necessary to understand how memory usage changesover time. For example, after a new commit, updates to thetoolset, such as a compiler, change in dependent libraries, orchange in some application’s configuration parameters.The standard approach to acquiring details about an ap-plication’s memory usage is to use a profiler. When anarXiv:2212.11866v1  [cs.SE]  22 Dec 2022application is executed under the profiler, such as Visual StudioProfiler [22] or Xcode [23], then the profiler tracks eachallocation and its source. An engineer can later filter, query,and visualize the resulting dataset. For example, the profilermay record the complete stack information coupled with theallocation size and originating dynamic library. Gathering theprofiler data is time-consuming, can require the application tobe compiled with specific flags (in case an instrumentationframework such as Valgrind [24] is used), and requires asignificant amount of disk storage to store the entire historyof allocations. Above all, this approach is not practical out-side the application’s development environment. The overheadcaused by instrumentation can cause an application to executean order of magnitude slower.A. Existing attribution techniquesTwo main approaches to attribute memory usage in sourcecode exist: annotating each call to allocate memory andannotating each scope. We describe the existing usage patternsfor each technique.In kernel mode, Windows uses pool tagging [19]. Eachdriver can specify a pool tag when it requests to allocatememory [25], [26], [27]. Depending on a size of a driver,it can use one or more pool tags to differentiate betweenvarious subsystems. Pool tags also have another function inWindows—kernel crashes if the driver does not release all theallocations with a specific tag when the driver is unloaded [19].On macOS, a caller can use a function such as OSMallocand associate each allocation with an opaque tag. However,that tag is used only for reference counting [28]. The tagcount is increased by one each time a specific tag is al-located. In user mode, macOS also enables passing customtags generated by VM_MAKE_TAG macro to functions such asvm_allocate [28]. Listing 1 shows a sample usage patternwhen a custom tag is associated with an allocation.Listing 1TAGGED ALLOCATION IN MACOS./* An allocation billed to networking. */err = vm_allocate(..., VM_MEMORY_LIBNETWORK);One of the FreeBSD ports is a basic heap memory account-ing system libpdel [29] that similarly requires each callerto specify a “memory type” in a form of a string.As a result of annotations, the application can duringthe runtime enumerate its virtual memory, and gather thedistribution and size of allocations per a different memory tagor a type.Hierarchically tracking memory usage by annotating sourcecode is another option. Developers need to attribute eachscope with a specific tag. All the allocator activity in thatscope and its children will be attributed to that tag. Listing 2displays how all the allocations in the function bar() and itschildren will be “billed” to the tag foo using the example ofTfMallocTag tagging system [30].Listing 2ALLOCATION TRACKING USING THE TFMALLOCTAG SYSTEM.void bar() {TfAutoMallocTag tag("foo");funcA();funcB();}B. Limitations of current techniquesAll these approaches have constraints because they require(a) annotation of each allocation or scope, (b) complete sourcecode to be available, and (c) hierarchical tracking needs tointercept all the allocations in the current process. However,a standard application has dependencies, such as system orthird-party libraries. Without modifying the dependencies,the ability to track allocations in detail is limited to theapplication’s “own code.”We are unaware of any operating systems, languages, ortools that enable engineers to query and keep track of memoryattribution at a granular level without sacrificing the applica-tion’s performance. The exploration of possible solutions isstill in the early stages. The most recent proposal for memoryallocation tracking in Linux is from August 2022 [31]. Thelack of these facilities negatively impacts each non-trivialsoftware project. Understanding the application’s memoryusage in detail is realistic only in the development environ-ment. However, in our experience, predicting or debugging anapplication’s behavior in the production environment based onthe data from the development environment is ineffective.III. FUTURE RESEARCH DIRECTIONSThe choice of the abstraction layers and a variety in theproblem space enables multiple research avenues. Ideally, thesupport for memory attribution will be integrated throughoutthe operating system, memory allocator, and a runtime librarysuch as the GNU C Library (glibc) [32]. Support for program-ming languages used for systems programming (e.g., C, C++,Rust) is imperative.The desired solution to improve engineers’ ability to at-tribute memory (a) can be enabled and disabled on demand,(b) has a minor performance overhead and will be usable inthe production environment, (c) enables querying the memoryattribution during runtime, and (d) has a well-designed set ofAPIs.One potential practical approach we envision in user modeis the usage of custom memory allocators [14], [33], [34],[13] to assist with the attribution. Custom allocators suchas jemalloc intercept each allocation request made in thecontext of a process. Therefore, the intercept mechanismcan track all the metadata such as allocation size, currentthread, timestamp, or specific flags passed to the function. Theintercept mechanism can use either data from the applicationthat specifies the current attribution scope, classify callersbased on sampling the call stack, or use some other techniques.REFERENCES[1] M. Hort, M. Kechagia, F. Sarro, and M. Harman, “A Survey ofPerformance Optimization for Mobile Applications,” IEEE Transactionson Software Engineering, vol. 48, no. 8, pp. 2879–2904, 2022. [Online].Available: https://doi.org/10.1109/TSE.2021.3071193[2] B. Gregg, Systems Performance: Enterprise and the Cloud, 2nd ed., ser.Addison-Wesley professional computing series. Boston, MA, USA:Addison-Wesley, 2020.[3] J. Levin, *OS internals. Volume 1: User space, 2nd ed. Edison, NJ,USA: Technologeeks.com, 2017.[4] J. Noble and C. Weir, Small Memory Software, ser. Software PatternsSeries. Boston, MA, USA: Addison Wesley, Nov. 2000.[5] Android OS Documentation. (2022, Oct.) Low Memory Killer Daemon.[Online]. Available: https://source.android.com/devices/tech/perf/lmkd[6] D. Xu. (2018, Jul.) Open-sourcing oomd, a new approach tohandling OOMs. Meta Platforms, Inc. [Online]. Available: https://engineering.fb.com/2018/07/19/production-engineering/oomd/[7] Eklektix, Inc. (2022) OOM killer. [Online]. Available: https://lwn.net/Kernel/Index/#OOM_killer[8] Mozilla Foundation. (2022) Performance—Firefox Source Docsdocumentation. [Online]. Available: https://firefox-source-docs.mozilla.org/performance/index.html[9] J. Corbet. (2022, May) Better tools for out-of-memory debugging.[Online]. Available: https://lwn.net/Articles/894546/[10] D. Leijen, B. Zorn, and L. de Moura, “Mimalloc: Free ListSharding in Action,” Microsoft, Tech. Rep. MSR-TR-2019-18, Jun.2019. [Online]. Available: https://www.microsoft.com/en-us/research/publication/mimalloc-free-list-sharding-in-action/[11] M. Jansson. (2017) rpmalloc—General Purpose Memory Allocator.[Online]. Available: https://github.com/mjansson/rpmalloc[12] P. Liétar, T. Butler, S. Clebsch, S. Drossopoulou, J. Franco,M. J. Parkinson, A. Shamis, C. M. Wintersteiger, and D. Chisnall,“Snmalloc: A Message Passing Allocator,” in Proceedings ofthe 2019 ACM SIGPLAN International Symposium on MemoryManagement, ser. ISMM 2019. New York, NY, USA: Associationfor Computing Machinery, 2019, pp. 122–135. [Online]. Available:https://doi.org/10.1145/3315573.3329980[13] S. Lee, T. Johnson, and E. Raman, “Feedback Directed Optimizationof TCMalloc,” in Proceedings of the Workshop on Memory SystemsPerformance and Correctness, ser. MSPC ’14. New York, NY, USA:Association for Computing Machinery, 2014. [Online]. Available:https://doi.org/10.1145/2618128.2618131[14] J. Evans, “A scalable concurrent malloc(3) implementation forFreeBSD,” in Proceedings of the BSDCan Conference, Universityof Ottawa, Ottawa, Canada, 2006. [Online]. Available: https://www.bsdcan.org/2006/papers/jemalloc.pdf[15] T. Ball, S. Chaki, and S. K. Rajamani, “Parameterized Verificationof Multithreaded Software Libraries,” in Proceedings of the 7thInternational Conference on Tools and Algorithms for the Constructionand Analysis of Systems, ser. TACAS 2001. Berlin, Heidelberg:Springer-Verlag, 2001, p. 158–173. [Online]. Available: https://doi.org/10.1007/3-540-45319-9_12[16] A. W. Appel and D. A. Naumann, “Verified Sequential Malloc/Free,”in Proceedings of the 2020 ACM SIGPLAN International Symposiumon Memory Management, ser. ISMM 2020. New York, NY, USA:Association for Computing Machinery, 2020, p. 48–59. [Online].Available: https://doi.org/10.1145/3381898.3397211[17] S. K. Gutiérrez, D. C. Arnold, K. Davis, and P. McCormick,“On the Memory Attribution Problem: A Solution and Case StudyUsing MPI,” Journal on Concurrency and Computation: Practice andExperience, vol. 32, no. 3, p. e5159, Feb. 2019. [Online]. Available:https://doi.org/10.1002/cpe.5159[18] A. S. Tanenbaum, Modern Operating Systems, 2nd ed. Upper SaddleRiver, NJ, USA: Prentice Hall, 2001.[19] M. E. Russinovich, D. A. Solomon, and A. Ionescu, Windows Inter-nals, 6th ed. Redmond, WA, USA: Microsoft Press, 2012, OCLC:ocn753301527.[20] R. Love, Linux Kernel Development, 2nd ed. Indianapolis, IN, USA:Novell Press, 2005.[21] W. Stallings, Operating Systems: Internals and Design Principles,6th ed. Upper Saddle River, NJ, USA: Pearson/Prentice Hall, 2009.[22] Microsoft Corporation. (2022) Measure performance in Visual Studio—Visual Studio (Windows). [Online]. Available: https://docs.microsoft.com/en-us/visualstudio/profiling/[23] Apple Inc. (2022) Xcode—Features. [Online]. Available: https://developer.apple.com/xcode/features/[24] Valgrind™ Developers. (2022) Valgrind Home. [Online]. Available:https://valgrind.org/[25] W. Oney, Programming the Microsoft Windows Driver Model, ser.Microsoft programming series. Redmond, WA, USA: Microsoft Press,Oct. 1999.[26] E. N. Dekker and J. M. Newcomer, Developing Windows NT devicedrivers. Boston, MA, USA: Addison-Wesley Educational, Mar. 1999.[27] P. Orwick and G. Smith, Developing drivers with the Windows DriverFoundation. Redmond, WA, USA: Microsoft Press, Apr. 2007.[28] A. Singh, Mac OS X Internals: a Systems Approach. Boston, MA,USA: Addison-Wesley Professional, 2016, OCLC: 1005337597.[29] A. Cobbs and Packet Design, LLC. (2002, Apr.)typed_mem—heap memory accounting system. [Online]. Avail-able: https://www.freebsd.org/cgi/man.cgi?query=FREE&manpath=FreeBSD+13.1-RELEASE+and+Ports[30] Pixar. (2022, Jul.) Universal Scene Description: The TfMallocTagMemory Tagging System. [Online]. Available: https://graphics.pixar.com/usd/dev/api/page_tf__malloc_tag.html[31] S. Baghdasaryan. (2022, Aug.) Code tagging frameworkand applications. [Online]. Available: https://lore.kernel.org/all/20220830214919.53220-1-surenb@google.com/[32] Free Software Foundation. (2022) The GNU C Library. [Online].Available: https://www.gnu.org/software/libc/[33] J. Evans, “Tick Tock, malloc Needs a Clock,” in Applicative 2015, ser.Applicative 2015. New York, NY, USA: Association for ComputingMachinery, 2015. [Online]. Available: https://doi.org/10.1145/2742580.2742807[34] ——. (2011, Jan.) Scalable memory allocation using jemalloc.[Online]. Available: https://engineering.fb.com/2011/01/03/core-data/scalable-memory-allocation-using-jemalloc/

Dissertations of the University of Groningen

   University of GroningenWho Ate My Memory? Towards Attribution in Memory ManagementKudrjavets, Gunnar; Rastogi, Ayushi; Thomas, Jeffrey; Nagappan, NachiappanPublished in:Proceedings of 45th International Conference on Software Engineering (ICSE 2023)IMPORTANT NOTE: You are advised to consult the publisher's version (publisher's PDF) if you wish to cite fromit. Please check the document version below.Document VersionEarly version, also known as pre-printPublication date:2022Link to publication in University of Groningen/UMCG research databaseCitation for published version (APA):Kudrjavets, G., Rastogi, A., Thomas, J., & Nagappan, N. (2022). Who Ate My Memory? Towards Attributionin Memory Management. Manuscript submitted for publication. In Proceedings of 45th InternationalConference on Software Engineering (ICSE 2023) arXiv. https://arxiv.org/pdf/2212.11866CopyrightOther than for strictly personal use, it is not permitted to download or to forward/distribute the text or part of it without the consent of theauthor(s) and/or copyright holder(s), unless the work is under an open content license (like Creative Commons).The publication may also be distributed here under the terms of Article 25fa of the Dutch Copyright Act, indicated by the “Taverne” license.More information can be found on the University of Groningen website: https://www.rug.nl/library/open-access/self-archiving-pure/taverne-amendment.Take-down policyIf you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediatelyand investigate your claim.Downloaded from the University of Groningen/UMCG research database (Pure): http://www.rug.nl/research/portal. For technical reasons thenumber of authors shown on this cover page is limited to 10 maximum.Download date: 21-01-2023Who Ate My Memory?Towards Attribution in Memory ManagementGunnar Kudrjavets, Ayushi RastogiUniversity of Groningen9712 CP Groningen, Netherlandsg.kudrjavets@rug.nl, a.rastogi@rug.nlJeff Thomas, Nachiappan NagappanMeta Platforms, Inc.Menlo Park, CA 94025, USAjeffdthomas@meta.com, nnachi@meta.comAbstract—To understand applications’ memory usage details,engineers use instrumented builds and profiling tools. Bothapproaches are impractical for use in production environments ordeployed mobile applications. As a result, developers can gatheronly high-level memory-related statistics for deployed software.In our experience, the lack of granular field data makes fixingperformance and reliability-related defects complex and time-consuming. The software industry needs lightweight solutionsto collect detailed data about applications’ memory usage toincrease developer productivity. Current research into memoryattribution-related data structures, techniques, and tools is in theearly stages and enables several new research avenues.Index Terms—Allocator, memory attribution, memory taggingI. BACKGROUND AND MOTIVATIONThis paper is motivated by our industry experience whileworking on problems related to optimizing and trackingmemory usage in both commercial and open-source softwareproducts. We observe the presence of a homogeneous set ofissues related to attribution (“How much memory is used bya particular component?”) and accountability (“What compo-nent is exceeding their memory budget?”). Answering thosequestions correctly and continuously is necessary to ensure theapplication’s performance and reliability.Software performance is associated with user satisfactionand how actively users engage with an application [1]. Onecriterion that impacts an application’s performance is itsmemory usage [2]. Characteristics such as the total size ofallocations, allocation rate, or the allocated memory typeserve as metrics describing how efficiently an applicationuses memory. Memory is one of the most precious systemresources on mobile platforms that do not support paging, suchas iOS [3]. A suggested design pattern for environments wherememory is limited is to make each component responsible foraccounting for its own memory usage [4].A detailed understanding of memory usage is also requiredto increase reliability. A variety of modern kernels and operat-ing systems based on them, such as Android, FreeBSD, iOS,or Linux, use a system component called an out-of-memorykiller [5], [6], [7]. An out-of-memory killer is responsible forterminating processes when their memory usage exceeds acertain quota, or excessive memory usage puts the stabilityof an entire operating system in danger. Engineers need tounderstand the application’s memory footprint in detail toavoid premature termination.In industry, we note that organizations develop their in-house ecosystems to solve company-specific memory attri-bution issues. A similar trend is present with open-sourcesoftware where projects such as Mozilla Firefox have aproduct-specific extensive framework to track and attributememory usage [8]. We observe the shortcomings in memoryattribution capabilities and tools even in popular kernels suchas Linux that has three decades of real-world usage and anactive developer community [9].Most of the research related to memory management fo-cuses on increasing the performance of custom allocators [10],[11], [12], [13], [14] or ensuring their correctness [15], [16].The limited existing research into memory attribution is inthe early stages, involves invasive profiling techniques, and isspecific to the Message Passing Interface [17].II. STATE OF AFFAIRS“. . . how it is that memory can be allocated without thekernel knowing where it went. The problem is that thetracking infrastructure just isn’t there.”— 2022 Linux Storage, Filesystem, MM and BPF SummitModern operating systems enable tracking memory usageat the process level [18], [19], [20], [21]. However, thegranularity at the process level is insufficient for engineers toperform efficient debugging or performance engineering [2].A common task that engineers encounter in practice is deter-mining what portion of the allocated amount of memory isconsumed by a specific component, scenario, or a subsystem.In the context of performance or reliability engineering thatcan mean a dynamic library, feature, function, or thread. Dataabout memory consumption is necessary to determine (a) whatcomponents to optimize, (b) if performance regressions arepresent, and (c) what performance tuning techniques to use.It is also necessary to understand how memory usage changesover time. For example, after a new commit, updates to thetoolset, such as a compiler, change in dependent libraries, orchange in some application’s configuration parameters.The standard approach to acquiring details about an ap-plication’s memory usage is to use a profiler. When anarXiv:2212.11866v1  [cs.SE]  22 Dec 2022application is executed under the profiler, such as Visual StudioProfiler [22] or Xcode [23], then the profiler tracks eachallocation and its source. An engineer can later filter, query,and visualize the resulting dataset. For example, the profilermay record the complete stack information coupled with theallocation size and originating dynamic library. Gathering theprofiler data is time-consuming, can require the application tobe compiled with specific flags (in case an instrumentationframework such as Valgrind [24] is used), and requires asignificant amount of disk storage to store the entire historyof allocations. Above all, this approach is not practical out-side the application’s development environment. The overheadcaused by instrumentation can cause an application to executean order of magnitude slower.A. Existing attribution techniquesTwo main approaches to attribute memory usage in sourcecode exist: annotating each call to allocate memory andannotating each scope. We describe the existing usage patternsfor each technique.In kernel mode, Windows uses pool tagging [19]. Eachdriver can specify a pool tag when it requests to allocatememory [25], [26], [27]. Depending on a size of a driver,it can use one or more pool tags to differentiate betweenvarious subsystems. Pool tags also have another function inWindows—kernel crashes if the driver does not release all theallocations with a specific tag when the driver is unloaded [19].On macOS, a caller can use a function such as OSMallocand associate each allocation with an opaque tag. However,that tag is used only for reference counting [28]. The tagcount is increased by one each time a specific tag is al-located. In user mode, macOS also enables passing customtags generated by VM_MAKE_TAG macro to functions such asvm_allocate [28]. Listing 1 shows a sample usage patternwhen a custom tag is associated with an allocation.Listing 1TAGGED ALLOCATION IN MACOS./* An allocation billed to networking. */err = vm_allocate(..., VM_MEMORY_LIBNETWORK);One of the FreeBSD ports is a basic heap memory account-ing system libpdel [29] that similarly requires each callerto specify a “memory type” in a form of a string.As a result of annotations, the application can duringthe runtime enumerate its virtual memory, and gather thedistribution and size of allocations per a different memory tagor a type.Hierarchically tracking memory usage by annotating sourcecode is another option. Developers need to attribute eachscope with a specific tag. All the allocator activity in thatscope and its children will be attributed to that tag. Listing 2displays how all the allocations in the function bar() and itschildren will be “billed” to the tag foo using the example ofTfMallocTag tagging system [30].Listing 2ALLOCATION TRACKING USING THE TFMALLOCTAG SYSTEM.void bar() {TfAutoMallocTag tag("foo");funcA();funcB();}B. Limitations of current techniquesAll these approaches have constraints because they require(a) annotation of each allocation or scope, (b) complete sourcecode to be available, and (c) hierarchical tracking needs tointercept all the allocations in the current process. However,a standard application has dependencies, such as system orthird-party libraries. Without modifying the dependencies,the ability to track allocations in detail is limited to theapplication’s “own code.”We are unaware of any operating systems, languages, ortools that enable engineers to query and keep track of memoryattribution at a granular level without sacrificing the applica-tion’s performance. The exploration of possible solutions isstill in the early stages. The most recent proposal for memoryallocation tracking in Linux is from August 2022 [31]. Thelack of these facilities negatively impacts each non-trivialsoftware project. Understanding the application’s memoryusage in detail is realistic only in the development environ-ment. However, in our experience, predicting or debugging anapplication’s behavior in the production environment based onthe data from the development environment is ineffective.III. FUTURE RESEARCH DIRECTIONSThe choice of the abstraction layers and a variety in theproblem space enables multiple research avenues. Ideally, thesupport for memory attribution will be integrated throughoutthe operating system, memory allocator, and a runtime librarysuch as the GNU C Library (glibc) [32]. Support for program-ming languages used for systems programming (e.g., C, C++,Rust) is imperative.The desired solution to improve engineers’ ability to at-tribute memory (a) can be enabled and disabled on demand,(b) has a minor performance overhead and will be usable inthe production environment, (c) enables querying the memoryattribution during runtime, and (d) has a well-designed set ofAPIs.One potential practical approach we envision in user modeis the usage of custom memory allocators [14], [33], [34],[13] to assist with the attribution. Custom allocators suchas jemalloc intercept each allocation request made in thecontext of a process. Therefore, the intercept mechanismcan track all the metadata such as allocation size, currentthread, timestamp, or specific flags passed to the function. Theintercept mechanism can use either data from the applicationthat specifies the current attribution scope, classify callersbased on sampling the call stack, or use some other techniques.REFERENCES[1] M. Hort, M. Kechagia, F. Sarro, and M. Harman, “A Survey ofPerformance Optimization for Mobile Applications,” IEEE Transactionson Software Engineering, vol. 48, no. 8, pp. 2879–2904, 2022. [Online].Available: https://doi.org/10.1109/TSE.2021.3071193[2] B. Gregg, Systems Performance: Enterprise and the Cloud, 2nd ed., ser.Addison-Wesley professional computing series. Boston, MA, USA:Addison-Wesley, 2020.[3] J. Levin, *OS internals. Volume 1: User space, 2nd ed. Edison, NJ,USA: Technologeeks.com, 2017.[4] J. Noble and C. Weir, Small Memory Software, ser. Software PatternsSeries. Boston, MA, USA: Addison Wesley, Nov. 2000.[5] Android OS Documentation. (2022, Oct.) Low Memory Killer Daemon.[Online]. Available: https://source.android.com/devices/tech/perf/lmkd[6] D. Xu. (2018, Jul.) Open-sourcing oomd, a new approach tohandling OOMs. Meta Platforms, Inc. [Online]. Available: https://engineering.fb.com/2018/07/19/production-engineering/oomd/[7] Eklektix, Inc. (2022) OOM killer. [Online]. Available: https://lwn.net/Kernel/Index/#OOM_killer[8] Mozilla Foundation. (2022) Performance—Firefox Source Docsdocumentation. [Online]. Available: https://firefox-source-docs.mozilla.org/performance/index.html[9] J. Corbet. (2022, May) Better tools for out-of-memory debugging.[Online]. Available: https://lwn.net/Articles/894546/[10] D. Leijen, B. Zorn, and L. de Moura, “Mimalloc: Free ListSharding in Action,” Microsoft, Tech. Rep. MSR-TR-2019-18, Jun.2019. [Online]. Available: https://www.microsoft.com/en-us/research/publication/mimalloc-free-list-sharding-in-action/[11] M. Jansson. (2017) rpmalloc—General Purpose Memory Allocator.[Online]. Available: https://github.com/mjansson/rpmalloc[12] P. Liétar, T. Butler, S. Clebsch, S. Drossopoulou, J. Franco,M. J. Parkinson, A. Shamis, C. M. Wintersteiger, and D. Chisnall,“Snmalloc: A Message Passing Allocator,” in Proceedings ofthe 2019 ACM SIGPLAN International Symposium on MemoryManagement, ser. ISMM 2019. New York, NY, USA: Associationfor Computing Machinery, 2019, pp. 122–135. [Online]. Available:https://doi.org/10.1145/3315573.3329980[13] S. Lee, T. Johnson, and E. Raman, “Feedback Directed Optimizationof TCMalloc,” in Proceedings of the Workshop on Memory SystemsPerformance and Correctness, ser. MSPC ’14. New York, NY, USA:Association for Computing Machinery, 2014. [Online]. Available:https://doi.org/10.1145/2618128.2618131[14] J. Evans, “A scalable concurrent malloc(3) implementation forFreeBSD,” in Proceedings of the BSDCan Conference, Universityof Ottawa, Ottawa, Canada, 2006. [Online]. Available: https://www.bsdcan.org/2006/papers/jemalloc.pdf[15] T. Ball, S. Chaki, and S. K. Rajamani, “Parameterized Verificationof Multithreaded Software Libraries,” in Proceedings of the 7thInternational Conference on Tools and Algorithms for the Constructionand Analysis of Systems, ser. TACAS 2001. Berlin, Heidelberg:Springer-Verlag, 2001, p. 158–173. [Online]. Available: https://doi.org/10.1007/3-540-45319-9_12[16] A. W. Appel and D. A. Naumann, “Verified Sequential Malloc/Free,”in Proceedings of the 2020 ACM SIGPLAN International Symposiumon Memory Management, ser. ISMM 2020. New York, NY, USA:Association for Computing Machinery, 2020, p. 48–59. [Online].Available: https://doi.org/10.1145/3381898.3397211[17] S. K. Gutiérrez, D. C. Arnold, K. Davis, and P. McCormick,“On the Memory Attribution Problem: A Solution and Case StudyUsing MPI,” Journal on Concurrency and Computation: Practice andExperience, vol. 32, no. 3, p. e5159, Feb. 2019. [Online]. Available:https://doi.org/10.1002/cpe.5159[18] A. S. Tanenbaum, Modern Operating Systems, 2nd ed. Upper SaddleRiver, NJ, USA: Prentice Hall, 2001.[19] M. E. Russinovich, D. A. Solomon, and A. Ionescu, Windows Inter-nals, 6th ed. Redmond, WA, USA: Microsoft Press, 2012, OCLC:ocn753301527.[20] R. Love, Linux Kernel Development, 2nd ed. Indianapolis, IN, USA:Novell Press, 2005.[21] W. Stallings, Operating Systems: Internals and Design Principles,6th ed. Upper Saddle River, NJ, USA: Pearson/Prentice Hall, 2009.[22] Microsoft Corporation. (2022) Measure performance in Visual Studio—Visual Studio (Windows). [Online]. Available: https://docs.microsoft.com/en-us/visualstudio/profiling/[23] Apple Inc. (2022) Xcode—Features. [Online]. Available: https://developer.apple.com/xcode/features/[24] Valgrind™ Developers. (2022) Valgrind Home. [Online]. Available:https://valgrind.org/[25] W. Oney, Programming the Microsoft Windows Driver Model, ser.Microsoft programming series. Redmond, WA, USA: Microsoft Press,Oct. 1999.[26] E. N. Dekker and J. M. Newcomer, Developing Windows NT devicedrivers. Boston, MA, USA: Addison-Wesley Educational, Mar. 1999.[27] P. Orwick and G. Smith, Developing drivers with the Windows DriverFoundation. Redmond, WA, USA: Microsoft Press, Apr. 2007.[28] A. Singh, Mac OS X Internals: a Systems Approach. Boston, MA,USA: Addison-Wesley Professional, 2016, OCLC: 1005337597.[29] A. Cobbs and Packet Design, LLC. (2002, Apr.)typed_mem—heap memory accounting system. [Online]. Avail-able: https://www.freebsd.org/cgi/man.cgi?query=FREE&manpath=FreeBSD+13.1-RELEASE+and+Ports[30] Pixar. (2022, Jul.) Universal Scene Description: The TfMallocTagMemory Tagging System. [Online]. Available: https://graphics.pixar.com/usd/dev/api/page_tf__malloc_tag.html[31] S. Baghdasaryan. (2022, Aug.) Code tagging frameworkand applications. [Online]. Available: https://lore.kernel.org/all/20220830214919.53220-1-surenb@google.com/[32] Free Software Foundation. (2022) The GNU C Library. [Online].Available: https://www.gnu.org/software/libc/[33] J. Evans, “Tick Tock, malloc Needs a Clock,” in Applicative 2015, ser.Applicative 2015. New York, NY, USA: Association for ComputingMachinery, 2015. [Online]. Available: https://doi.org/10.1145/2742580.2742807[34] ——. (2011, Jan.) Scalable memory allocation using jemalloc.[Online]. Available: https://engineering.fb.com/2011/01/03/core-data/scalable-memory-allocation-using-jemalloc/

Thomas, Jeff

arXiv.org e-Print Archive

   University of GroningenWho Ate My Memory? Towards Attribution in Memory ManagementKudrjavets, Gunnar; Rastogi, Ayushi; Thomas, Jeffrey; Nagappan, NachiappanPublished in:Proceedings of 45th International Conference on Software Engineering (ICSE 2023)IMPORTANT NOTE: You are advised to consult the publisher's version (publisher's PDF) if you wish to cite fromit. Please check the document version below.Document VersionEarly version, also known as pre-printPublication date:2022Link to publication in University of Groningen/UMCG research databaseCitation for published version (APA):Kudrjavets, G., Rastogi, A., Thomas, J., & Nagappan, N. (2022). Who Ate My Memory? Towards Attributionin Memory Management. Manuscript submitted for publication. In Proceedings of 45th InternationalConference on Software Engineering (ICSE 2023) arXiv. https://arxiv.org/pdf/2212.11866CopyrightOther than for strictly personal use, it is not permitted to download or to forward/distribute the text or part of it without the consent of theauthor(s) and/or copyright holder(s), unless the work is under an open content license (like Creative Commons).The publication may also be distributed here under the terms of Article 25fa of the Dutch Copyright Act, indicated by the “Taverne” license.More information can be found on the University of Groningen website: https://www.rug.nl/library/open-access/self-archiving-pure/taverne-amendment.Take-down policyIf you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediatelyand investigate your claim.Downloaded from the University of Groningen/UMCG research database (Pure): http://www.rug.nl/research/portal. For technical reasons thenumber of authors shown on this cover page is limited to 10 maximum.Download date: 11-09-2023Who Ate My Memory?Towards Attribution in Memory ManagementGunnar Kudrjavets, Ayushi RastogiUniversity of Groningen9712 CP Groningen, Netherlandsg.kudrjavets@rug.nl, a.rastogi@rug.nlJeff Thomas, Nachiappan NagappanMeta Platforms, Inc.Menlo Park, CA 94025, USAjeffdthomas@meta.com, nnachi@meta.comAbstract—To understand applications’ memory usage details,engineers use instrumented builds and profiling tools. Bothapproaches are impractical for use in production environments ordeployed mobile applications. As a result, developers can gatheronly high-level memory-related statistics for deployed software.In our experience, the lack of granular field data makes fixingperformance and reliability-related defects complex and time-consuming. The software industry needs lightweight solutionsto collect detailed data about applications’ memory usage toincrease developer productivity. Current research into memoryattribution-related data structures, techniques, and tools is in theearly stages and enables several new research avenues.Index Terms—Allocator, memory attribution, memory taggingI. BACKGROUND AND MOTIVATIONThis paper is motivated by our industry experience whileworking on problems related to optimizing and trackingmemory usage in both commercial and open-source softwareproducts. We observe the presence of a homogeneous set ofissues related to attribution (“How much memory is used bya particular component?”) and accountability (“What compo-nent is exceeding their memory budget?”). Answering thosequestions correctly and continuously is necessary to ensure theapplication’s performance and reliability.Software performance is associated with user satisfactionand how actively users engage with an application [1]. Onecriterion that impacts an application’s performance is itsmemory usage [2]. Characteristics such as the total size ofallocations, allocation rate, or the allocated memory typeserve as metrics describing how efficiently an applicationuses memory. Memory is one of the most precious systemresources on mobile platforms that do not support paging, suchas iOS [3]. A suggested design pattern for environments wherememory is limited is to make each component responsible foraccounting for its own memory usage [4].A detailed understanding of memory usage is also requiredto increase reliability. A variety of modern kernels and operat-ing systems based on them, such as Android, FreeBSD, iOS,or Linux, use a system component called an out-of-memorykiller [5], [6], [7]. An out-of-memory killer is responsible forterminating processes when their memory usage exceeds acertain quota, or excessive memory usage puts the stabilityof an entire operating system in danger. Engineers need tounderstand the application’s memory footprint in detail toavoid premature termination.In industry, we note that organizations develop their in-house ecosystems to solve company-specific memory attri-bution issues. A similar trend is present with open-sourcesoftware where projects such as Mozilla Firefox have aproduct-specific extensive framework to track and attributememory usage [8]. We observe the shortcomings in memoryattribution capabilities and tools even in popular kernels suchas Linux that has three decades of real-world usage and anactive developer community [9].Most of the research related to memory management fo-cuses on increasing the performance of custom allocators [10],[11], [12], [13], [14] or ensuring their correctness [15], [16].The limited existing research into memory attribution is inthe early stages, involves invasive profiling techniques, and isspecific to the Message Passing Interface [17].II. STATE OF AFFAIRS“. . . how it is that memory can be allocated without thekernel knowing where it went. The problem is that thetracking infrastructure just isn’t there.”— 2022 Linux Storage, Filesystem, MM and BPF SummitModern operating systems enable tracking memory usageat the process level [18], [19], [20], [21]. However, thegranularity at the process level is insufficient for engineers toperform efficient debugging or performance engineering [2].A common task that engineers encounter in practice is deter-mining what portion of the allocated amount of memory isconsumed by a specific component, scenario, or a subsystem.In the context of performance or reliability engineering thatcan mean a dynamic library, feature, function, or thread. Dataabout memory consumption is necessary to determine (a) whatcomponents to optimize, (b) if performance regressions arepresent, and (c) what performance tuning techniques to use.It is also necessary to understand how memory usage changesover time. For example, after a new commit, updates to thetoolset, such as a compiler, change in dependent libraries, orchange in some application’s configuration parameters.The standard approach to acquiring details about an ap-plication’s memory usage is to use a profiler. When anarXiv:2212.11866v1  [cs.SE]  22 Dec 2022application is executed under the profiler, such as Visual StudioProfiler [22] or Xcode [23], then the profiler tracks eachallocation and its source. An engineer can later filter, query,and visualize the resulting dataset. For example, the profilermay record the complete stack information coupled with theallocation size and originating dynamic library. Gathering theprofiler data is time-consuming, can require the application tobe compiled with specific flags (in case an instrumentationframework such as Valgrind [24] is used), and requires asignificant amount of disk storage to store the entire historyof allocations. Above all, this approach is not practical out-side the application’s development environment. The overheadcaused by instrumentation can cause an application to executean order of magnitude slower.A. Existing attribution techniquesTwo main approaches to attribute memory usage in sourcecode exist: annotating each call to allocate memory andannotating each scope. We describe the existing usage patternsfor each technique.In kernel mode, Windows uses pool tagging [19]. Eachdriver can specify a pool tag when it requests to allocatememory [25], [26], [27]. Depending on a size of a driver,it can use one or more pool tags to differentiate betweenvarious subsystems. Pool tags also have another function inWindows—kernel crashes if the driver does not release all theallocations with a specific tag when the driver is unloaded [19].On macOS, a caller can use a function such as OSMallocand associate each allocation with an opaque tag. However,that tag is used only for reference counting [28]. The tagcount is increased by one each time a specific tag is al-located. In user mode, macOS also enables passing customtags generated by VM_MAKE_TAG macro to functions such asvm_allocate [28]. Listing 1 shows a sample usage patternwhen a custom tag is associated with an allocation.Listing 1TAGGED ALLOCATION IN MACOS./* An allocation billed to networking. */err = vm_allocate(..., VM_MEMORY_LIBNETWORK);One of the FreeBSD ports is a basic heap memory account-ing system libpdel [29] that similarly requires each callerto specify a “memory type” in a form of a string.As a result of annotations, the application can duringthe runtime enumerate its virtual memory, and gather thedistribution and size of allocations per a different memory tagor a type.Hierarchically tracking memory usage by annotating sourcecode is another option. Developers need to attribute eachscope with a specific tag. All the allocator activity in thatscope and its children will be attributed to that tag. Listing 2displays how all the allocations in the function bar() and itschildren will be “billed” to the tag foo using the example ofTfMallocTag tagging system [30].Listing 2ALLOCATION TRACKING USING THE TFMALLOCTAG SYSTEM.void bar() {TfAutoMallocTag tag("foo");funcA();funcB();}B. Limitations of current techniquesAll these approaches have constraints because they require(a) annotation of each allocation or scope, (b) complete sourcecode to be available, and (c) hierarchical tracking needs tointercept all the allocations in the current process. However,a standard application has dependencies, such as system orthird-party libraries. Without modifying the dependencies,the ability to track allocations in detail is limited to theapplication’s “own code.”We are unaware of any operating systems, languages, ortools that enable engineers to query and keep track of memoryattribution at a granular level without sacrificing the applica-tion’s performance. The exploration of possible solutions isstill in the early stages. The most recent proposal for memoryallocation tracking in Linux is from August 2022 [31]. Thelack of these facilities negatively impacts each non-trivialsoftware project. Understanding the application’s memoryusage in detail is realistic only in the development environ-ment. However, in our experience, predicting or debugging anapplication’s behavior in the production environment based onthe data from the development environment is ineffective.III. FUTURE RESEARCH DIRECTIONSThe choice of the abstraction layers and a variety in theproblem space enables multiple research avenues. Ideally, thesupport for memory attribution will be integrated throughoutthe operating system, memory allocator, and a runtime librarysuch as the GNU C Library (glibc) [32]. Support for program-ming languages used for systems programming (e.g., C, C++,Rust) is imperative.The desired solution to improve engineers’ ability to at-tribute memory (a) can be enabled and disabled on demand,(b) has a minor performance overhead and will be usable inthe production environment, (c) enables querying the memoryattribution during runtime, and (d) has a well-designed set ofAPIs.One potential practical approach we envision in user modeis the usage of custom memory allocators [14], [33], [34],[13] to assist with the attribution. Custom allocators suchas jemalloc intercept each allocation request made in thecontext of a process. Therefore, the intercept mechanismcan track all the metadata such as allocation size, currentthread, timestamp, or specific flags passed to the function. Theintercept mechanism can use either data from the applicationthat specifies the current attribution scope, classify callersbased on sampling the call stack, or use some other techniques.REFERENCES[1] M. Hort, M. Kechagia, F. Sarro, and M. Harman, “A Survey ofPerformance Optimization for Mobile Applications,” IEEE Transactionson Software Engineering, vol. 48, no. 8, pp. 2879–2904, 2022. [Online].Available: https://doi.org/10.1109/TSE.2021.3071193[2] B. Gregg, Systems Performance: Enterprise and the Cloud, 2nd ed., ser.Addison-Wesley professional computing series. Boston, MA, USA:Addison-Wesley, 2020.[3] J. Levin, *OS internals. Volume 1: User space, 2nd ed. Edison, NJ,USA: Technologeeks.com, 2017.[4] J. Noble and C. Weir, Small Memory Software, ser. Software PatternsSeries. Boston, MA, USA: Addison Wesley, Nov. 2000.[5] Android OS Documentation. (2022, Oct.) Low Memory Killer Daemon.[Online]. Available: https://source.android.com/devices/tech/perf/lmkd[6] D. Xu. (2018, Jul.) Open-sourcing oomd, a new approach tohandling OOMs. Meta Platforms, Inc. [Online]. Available: https://engineering.fb.com/2018/07/19/production-engineering/oomd/[7] Eklektix, Inc. (2022) OOM killer. [Online]. Available: https://lwn.net/Kernel/Index/#OOM_killer[8] Mozilla Foundation. (2022) Performance—Firefox Source Docsdocumentation. [Online]. Available: https://firefox-source-docs.mozilla.org/performance/index.html[9] J. Corbet. (2022, May) Better tools for out-of-memory debugging.[Online]. Available: https://lwn.net/Articles/894546/[10] D. Leijen, B. Zorn, and L. de Moura, “Mimalloc: Free ListSharding in Action,” Microsoft, Tech. Rep. MSR-TR-2019-18, Jun.2019. [Online]. Available: https://www.microsoft.com/en-us/research/publication/mimalloc-free-list-sharding-in-action/[11] M. Jansson. (2017) rpmalloc—General Purpose Memory Allocator.[Online]. Available: https://github.com/mjansson/rpmalloc[12] P. Liétar, T. Butler, S. Clebsch, S. Drossopoulou, J. Franco,M. J. Parkinson, A. Shamis, C. M. Wintersteiger, and D. Chisnall,“Snmalloc: A Message Passing Allocator,” in Proceedings ofthe 2019 ACM SIGPLAN International Symposium on MemoryManagement, ser. ISMM 2019. New York, NY, USA: Associationfor Computing Machinery, 2019, pp. 122–135. [Online]. Available:https://doi.org/10.1145/3315573.3329980[13] S. Lee, T. Johnson, and E. Raman, “Feedback Directed Optimizationof TCMalloc,” in Proceedings of the Workshop on Memory SystemsPerformance and Correctness, ser. MSPC ’14. New York, NY, USA:Association for Computing Machinery, 2014. [Online]. Available:https://doi.org/10.1145/2618128.2618131[14] J. Evans, “A scalable concurrent malloc(3) implementation forFreeBSD,” in Proceedings of the BSDCan Conference, Universityof Ottawa, Ottawa, Canada, 2006. [Online]. Available: https://www.bsdcan.org/2006/papers/jemalloc.pdf[15] T. Ball, S. Chaki, and S. K. Rajamani, “Parameterized Verificationof Multithreaded Software Libraries,” in Proceedings of the 7thInternational Conference on Tools and Algorithms for the Constructionand Analysis of Systems, ser. TACAS 2001. Berlin, Heidelberg:Springer-Verlag, 2001, p. 158–173. [Online]. Available: https://doi.org/10.1007/3-540-45319-9_12[16] A. W. Appel and D. A. Naumann, “Verified Sequential Malloc/Free,”in Proceedings of the 2020 ACM SIGPLAN International Symposiumon Memory Management, ser. ISMM 2020. New York, NY, USA:Association for Computing Machinery, 2020, p. 48–59. [Online].Available: https://doi.org/10.1145/3381898.3397211[17] S. K. Gutiérrez, D. C. Arnold, K. Davis, and P. McCormick,“On the Memory Attribution Problem: A Solution and Case StudyUsing MPI,” Journal on Concurrency and Computation: Practice andExperience, vol. 32, no. 3, p. e5159, Feb. 2019. [Online]. Available:https://doi.org/10.1002/cpe.5159[18] A. S. Tanenbaum, Modern Operating Systems, 2nd ed. Upper SaddleRiver, NJ, USA: Prentice Hall, 2001.[19] M. E. Russinovich, D. A. Solomon, and A. Ionescu, Windows Inter-nals, 6th ed. Redmond, WA, USA: Microsoft Press, 2012, OCLC:ocn753301527.[20] R. Love, Linux Kernel Development, 2nd ed. Indianapolis, IN, USA:Novell Press, 2005.[21] W. Stallings, Operating Systems: Internals and Design Principles,6th ed. Upper Saddle River, NJ, USA: Pearson/Prentice Hall, 2009.[22] Microsoft Corporation. (2022) Measure performance in Visual Studio—Visual Studio (Windows). [Online]. Available: https://docs.microsoft.com/en-us/visualstudio/profiling/[23] Apple Inc. (2022) Xcode—Features. [Online]. Available: https://developer.apple.com/xcode/features/[24] Valgrind™ Developers. (2022) Valgrind Home. [Online]. Available:https://valgrind.org/[25] W. Oney, Programming the Microsoft Windows Driver Model, ser.Microsoft programming series. Redmond, WA, USA: Microsoft Press,Oct. 1999.[26] E. N. Dekker and J. M. Newcomer, Developing Windows NT devicedrivers. Boston, MA, USA: Addison-Wesley Educational, Mar. 1999.[27] P. Orwick and G. Smith, Developing drivers with the Windows DriverFoundation. Redmond, WA, USA: Microsoft Press, Apr. 2007.[28] A. Singh, Mac OS X Internals: a Systems Approach. Boston, MA,USA: Addison-Wesley Professional, 2016, OCLC: 1005337597.[29] A. Cobbs and Packet Design, LLC. (2002, Apr.)typed_mem—heap memory accounting system. [Online]. Avail-able: https://www.freebsd.org/cgi/man.cgi?query=FREE&manpath=FreeBSD+13.1-RELEASE+and+Ports[30] Pixar. (2022, Jul.) Universal Scene Description: The TfMallocTagMemory Tagging System. [Online]. Available: https://graphics.pixar.com/usd/dev/api/page_tf__malloc_tag.html[31] S. Baghdasaryan. (2022, Aug.) Code tagging frameworkand applications. [Online]. Available: https://lore.kernel.org/all/20220830214919.53220-1-surenb@google.com/[32] Free Software Foundation. (2022) The GNU C Library. [Online].Available: https://www.gnu.org/software/libc/[33] J. Evans, “Tick Tock, malloc Needs a Clock,” in Applicative 2015, ser.Applicative 2015. New York, NY, USA: Association for ComputingMachinery, 2015. [Online]. Available: https://doi.org/10.1145/2742580.2742807[34] ——. (2011, Jan.) Scalable memory allocation using jemalloc.[Online]. Available: https://engineering.fb.com/2011/01/03/core-data/scalable-memory-allocation-using-jemalloc/

   University of GroningenWho Ate My Memory? Towards Attribution in Memory ManagementKudrjavets, Gunnar; Rastogi, Ayushi; Thomas, Jeffrey; Nagappan, NachiappanPublished in:Proceedings of 45th International Conference on Software Engineering (ICSE 2023)IMPORTANT NOTE: You are advised to consult the publisher's version (publisher's PDF) if you wish to cite fromit. Please check the document version below.Document VersionEarly version, also known as pre-printPublication date:2022Link to publication in University of Groningen/UMCG research databaseCitation for published version (APA):Kudrjavets, G., Rastogi, A., Thomas, J., & Nagappan, N. (2022). Who Ate My Memory? Towards Attributionin Memory Management. Manuscript submitted for publication. In Proceedings of 45th InternationalConference on Software Engineering (ICSE 2023) arXiv. https://arxiv.org/pdf/2212.11866CopyrightOther than for strictly personal use, it is not permitted to download or to forward/distribute the text or part of it without the consent of theauthor(s) and/or copyright holder(s), unless the work is under an open content license (like Creative Commons).The publication may also be distributed here under the terms of Article 25fa of the Dutch Copyright Act, indicated by the “Taverne” license.More information can be found on the University of Groningen website: https://www.rug.nl/library/open-access/self-archiving-pure/taverne-amendment.Take-down policyIf you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediatelyand investigate your claim.Downloaded from the University of Groningen/UMCG research database (Pure): http://www.rug.nl/research/portal. For technical reasons thenumber of authors shown on this cover page is limited to 10 maximum.Download date: 31-10-2023Who Ate My Memory?Towards Attribution in Memory ManagementGunnar Kudrjavets, Ayushi RastogiUniversity of Groningen9712 CP Groningen, Netherlandsg.kudrjavets@rug.nl, a.rastogi@rug.nlJeff Thomas, Nachiappan NagappanMeta Platforms, Inc.Menlo Park, CA 94025, USAjeffdthomas@meta.com, nnachi@meta.comAbstract—To understand applications’ memory usage details,engineers use instrumented builds and profiling tools. Bothapproaches are impractical for use in production environments ordeployed mobile applications. As a result, developers can gatheronly high-level memory-related statistics for deployed software.In our experience, the lack of granular field data makes fixingperformance and reliability-related defects complex and time-consuming. The software industry needs lightweight solutionsto collect detailed data about applications’ memory usage toincrease developer productivity. Current research into memoryattribution-related data structures, techniques, and tools is in theearly stages and enables several new research avenues.Index Terms—Allocator, memory attribution, memory taggingI. BACKGROUND AND MOTIVATIONThis paper is motivated by our industry experience whileworking on problems related to optimizing and trackingmemory usage in both commercial and open-source softwareproducts. We observe the presence of a homogeneous set ofissues related to attribution (“How much memory is used bya particular component?”) and accountability (“What compo-nent is exceeding their memory budget?”). Answering thosequestions correctly and continuously is necessary to ensure theapplication’s performance and reliability.Software performance is associated with user satisfactionand how actively users engage with an application [1]. Onecriterion that impacts an application’s performance is itsmemory usage [2]. Characteristics such as the total size ofallocations, allocation rate, or the allocated memory typeserve as metrics describing how efficiently an applicationuses memory. Memory is one of the most precious systemresources on mobile platforms that do not support paging, suchas iOS [3]. A suggested design pattern for environments wherememory is limited is to make each component responsible foraccounting for its own memory usage [4].A detailed understanding of memory usage is also requiredto increase reliability. A variety of modern kernels and operat-ing systems based on them, such as Android, FreeBSD, iOS,or Linux, use a system component called an out-of-memorykiller [5], [6], [7]. An out-of-memory killer is responsible forterminating processes when their memory usage exceeds acertain quota, or excessive memory usage puts the stabilityof an entire operating system in danger. Engineers need tounderstand the application’s memory footprint in detail toavoid premature termination.In industry, we note that organizations develop their in-house ecosystems to solve company-specific memory attri-bution issues. A similar trend is present with open-sourcesoftware where projects such as Mozilla Firefox have aproduct-specific extensive framework to track and attributememory usage [8]. We observe the shortcomings in memoryattribution capabilities and tools even in popular kernels suchas Linux that has three decades of real-world usage and anactive developer community [9].Most of the research related to memory management fo-cuses on increasing the performance of custom allocators [10],[11], [12], [13], [14] or ensuring their correctness [15], [16].The limited existing research into memory attribution is inthe early stages, involves invasive profiling techniques, and isspecific to the Message Passing Interface [17].II. STATE OF AFFAIRS“. . . how it is that memory can be allocated without thekernel knowing where it went. The problem is that thetracking infrastructure just isn’t there.”— 2022 Linux Storage, Filesystem, MM and BPF SummitModern operating systems enable tracking memory usageat the process level [18], [19], [20], [21]. However, thegranularity at the process level is insufficient for engineers toperform efficient debugging or performance engineering [2].A common task that engineers encounter in practice is deter-mining what portion of the allocated amount of memory isconsumed by a specific component, scenario, or a subsystem.In the context of performance or reliability engineering thatcan mean a dynamic library, feature, function, or thread. Dataabout memory consumption is necessary to determine (a) whatcomponents to optimize, (b) if performance regressions arepresent, and (c) what performance tuning techniques to use.It is also necessary to understand how memory usage changesover time. For example, after a new commit, updates to thetoolset, such as a compiler, change in dependent libraries, orchange in some application’s configuration parameters.The standard approach to acquiring details about an ap-plication’s memory usage is to use a profiler. When anarXiv:2212.11866v1  [cs.SE]  22 Dec 2022application is executed under the profiler, such as Visual StudioProfiler [22] or Xcode [23], then the profiler tracks eachallocation and its source. An engineer can later filter, query,and visualize the resulting dataset. For example, the profilermay record the complete stack information coupled with theallocation size and originating dynamic library. Gathering theprofiler data is time-consuming, can require the application tobe compiled with specific flags (in case an instrumentationframework such as Valgrind [24] is used), and requires asignificant amount of disk storage to store the entire historyof allocations. Above all, this approach is not practical out-side the application’s development environment. The overheadcaused by instrumentation can cause an application to executean order of magnitude slower.A. Existing attribution techniquesTwo main approaches to attribute memory usage in sourcecode exist: annotating each call to allocate memory andannotating each scope. We describe the existing usage patternsfor each technique.In kernel mode, Windows uses pool tagging [19]. Eachdriver can specify a pool tag when it requests to allocatememory [25], [26], [27]. Depending on a size of a driver,it can use one or more pool tags to differentiate betweenvarious subsystems. Pool tags also have another function inWindows—kernel crashes if the driver does not release all theallocations with a specific tag when the driver is unloaded [19].On macOS, a caller can use a function such as OSMallocand associate each allocation with an opaque tag. However,that tag is used only for reference counting [28]. The tagcount is increased by one each time a specific tag is al-located. In user mode, macOS also enables passing customtags generated by VM_MAKE_TAG macro to functions such asvm_allocate [28]. Listing 1 shows a sample usage patternwhen a custom tag is associated with an allocation.Listing 1TAGGED ALLOCATION IN MACOS./* An allocation billed to networking. */err = vm_allocate(..., VM_MEMORY_LIBNETWORK);One of the FreeBSD ports is a basic heap memory account-ing system libpdel [29] that similarly requires each callerto specify a “memory type” in a form of a string.As a result of annotations, the application can duringthe runtime enumerate its virtual memory, and gather thedistribution and size of allocations per a different memory tagor a type.Hierarchically tracking memory usage by annotating sourcecode is another option. Developers need to attribute eachscope with a specific tag. All the allocator activity in thatscope and its children will be attributed to that tag. Listing 2displays how all the allocations in the function bar() and itschildren will be “billed” to the tag foo using the example ofTfMallocTag tagging system [30].Listing 2ALLOCATION TRACKING USING THE TFMALLOCTAG SYSTEM.void bar() {TfAutoMallocTag tag("foo");funcA();funcB();}B. Limitations of current techniquesAll these approaches have constraints because they require(a) annotation of each allocation or scope, (b) complete sourcecode to be available, and (c) hierarchical tracking needs tointercept all the allocations in the current process. However,a standard application has dependencies, such as system orthird-party libraries. Without modifying the dependencies,the ability to track allocations in detail is limited to theapplication’s “own code.”We are unaware of any operating systems, languages, ortools that enable engineers to query and keep track of memoryattribution at a granular level without sacrificing the applica-tion’s performance. The exploration of possible solutions isstill in the early stages. The most recent proposal for memoryallocation tracking in Linux is from August 2022 [31]. Thelack of these facilities negatively impacts each non-trivialsoftware project. Understanding the application’s memoryusage in detail is realistic only in the development environ-ment. However, in our experience, predicting or debugging anapplication’s behavior in the production environment based onthe data from the development environment is ineffective.III. FUTURE RESEARCH DIRECTIONSThe choice of the abstraction layers and a variety in theproblem space enables multiple research avenues. Ideally, thesupport for memory attribution will be integrated throughoutthe operating system, memory allocator, and a runtime librarysuch as the GNU C Library (glibc) [32]. Support for program-ming languages used for systems programming (e.g., C, C++,Rust) is imperative.The desired solution to improve engineers’ ability to at-tribute memory (a) can be enabled and disabled on demand,(b) has a minor performance overhead and will be usable inthe production environment, (c) enables querying the memoryattribution during runtime, and (d) has a well-designed set ofAPIs.One potential practical approach we envision in user modeis the usage of custom memory allocators [14], [33], [34],[13] to assist with the attribution. Custom allocators suchas jemalloc intercept each allocation request made in thecontext of a process. Therefore, the intercept mechanismcan track all the metadata such as allocation size, currentthread, timestamp, or specific flags passed to the function. Theintercept mechanism can use either data from the applicationthat specifies the current attribution scope, classify callersbased on sampling the call stack, or use some other techniques.REFERENCES[1] M. Hort, M. Kechagia, F. Sarro, and M. Harman, “A Survey ofPerformance Optimization for Mobile Applications,” IEEE Transactionson Software Engineering, vol. 48, no. 8, pp. 2879–2904, 2022. [Online].Available: https://doi.org/10.1109/TSE.2021.3071193[2] B. Gregg, Systems Performance: Enterprise and the Cloud, 2nd ed., ser.Addison-Wesley professional computing series. Boston, MA, USA:Addison-Wesley, 2020.[3] J. Levin, *OS internals. Volume 1: User space, 2nd ed. Edison, NJ,USA: Technologeeks.com, 2017.[4] J. Noble and C. Weir, Small Memory Software, ser. Software PatternsSeries. Boston, MA, USA: Addison Wesley, Nov. 2000.[5] Android OS Documentation. (2022, Oct.) Low Memory Killer Daemon.[Online]. Available: https://source.android.com/devices/tech/perf/lmkd[6] D. Xu. (2018, Jul.) Open-sourcing oomd, a new approach tohandling OOMs. Meta Platforms, Inc. [Online]. Available: https://engineering.fb.com/2018/07/19/production-engineering/oomd/[7] Eklektix, Inc. (2022) OOM killer. [Online]. Available: https://lwn.net/Kernel/Index/#OOM_killer[8] Mozilla Foundation. (2022) Performance—Firefox Source Docsdocumentation. [Online]. Available: https://firefox-source-docs.mozilla.org/performance/index.html[9] J. Corbet. (2022, May) Better tools for out-of-memory debugging.[Online]. Available: https://lwn.net/Articles/894546/[10] D. Leijen, B. Zorn, and L. de Moura, “Mimalloc: Free ListSharding in Action,” Microsoft, Tech. Rep. MSR-TR-2019-18, Jun.2019. [Online]. Available: https://www.microsoft.com/en-us/research/publication/mimalloc-free-list-sharding-in-action/[11] M. Jansson. (2017) rpmalloc—General Purpose Memory Allocator.[Online]. Available: https://github.com/mjansson/rpmalloc[12] P. Liétar, T. Butler, S. Clebsch, S. Drossopoulou, J. Franco,M. J. Parkinson, A. Shamis, C. M. Wintersteiger, and D. Chisnall,“Snmalloc: A Message Passing Allocator,” in Proceedings ofthe 2019 ACM SIGPLAN International Symposium on MemoryManagement, ser. ISMM 2019. New York, NY, USA: Associationfor Computing Machinery, 2019, pp. 122–135. [Online]. Available:https://doi.org/10.1145/3315573.3329980[13] S. Lee, T. Johnson, and E. Raman, “Feedback Directed Optimizationof TCMalloc,” in Proceedings of the Workshop on Memory SystemsPerformance and Correctness, ser. MSPC ’14. New York, NY, USA:Association for Computing Machinery, 2014. [Online]. Available:https://doi.org/10.1145/2618128.2618131[14] J. Evans, “A scalable concurrent malloc(3) implementation forFreeBSD,” in Proceedings of the BSDCan Conference, Universityof Ottawa, Ottawa, Canada, 2006. [Online]. Available: https://www.bsdcan.org/2006/papers/jemalloc.pdf[15] T. Ball, S. Chaki, and S. K. Rajamani, “Parameterized Verificationof Multithreaded Software Libraries,” in Proceedings of the 7thInternational Conference on Tools and Algorithms for the Constructionand Analysis of Systems, ser. TACAS 2001. Berlin, Heidelberg:Springer-Verlag, 2001, p. 158–173. [Online]. Available: https://doi.org/10.1007/3-540-45319-9_12[16] A. W. Appel and D. A. Naumann, “Verified Sequential Malloc/Free,”in Proceedings of the 2020 ACM SIGPLAN International Symposiumon Memory Management, ser. ISMM 2020. New York, NY, USA:Association for Computing Machinery, 2020, p. 48–59. [Online].Available: https://doi.org/10.1145/3381898.3397211[17] S. K. Gutiérrez, D. C. Arnold, K. Davis, and P. McCormick,“On the Memory Attribution Problem: A Solution and Case StudyUsing MPI,” Journal on Concurrency and Computation: Practice andExperience, vol. 32, no. 3, p. e5159, Feb. 2019. [Online]. Available:https://doi.org/10.1002/cpe.5159[18] A. S. Tanenbaum, Modern Operating Systems, 2nd ed. Upper SaddleRiver, NJ, USA: Prentice Hall, 2001.[19] M. E. Russinovich, D. A. Solomon, and A. Ionescu, Windows Inter-nals, 6th ed. Redmond, WA, USA: Microsoft Press, 2012, OCLC:ocn753301527.[20] R. Love, Linux Kernel Development, 2nd ed. Indianapolis, IN, USA:Novell Press, 2005.[21] W. Stallings, Operating Systems: Internals and Design Principles,6th ed. Upper Saddle River, NJ, USA: Pearson/Prentice Hall, 2009.[22] Microsoft Corporation. (2022) Measure performance in Visual Studio—Visual Studio (Windows). [Online]. Available: https://docs.microsoft.com/en-us/visualstudio/profiling/[23] Apple Inc. (2022) Xcode—Features. [Online]. Available: https://developer.apple.com/xcode/features/[24] Valgrind™ Developers. (2022) Valgrind Home. [Online]. Available:https://valgrind.org/[25] W. Oney, Programming the Microsoft Windows Driver Model, ser.Microsoft programming series. Redmond, WA, USA: Microsoft Press,Oct. 1999.[26] E. N. Dekker and J. M. Newcomer, Developing Windows NT devicedrivers. Boston, MA, USA: Addison-Wesley Educational, Mar. 1999.[27] P. Orwick and G. Smith, Developing drivers with the Windows DriverFoundation. Redmond, WA, USA: Microsoft Press, Apr. 2007.[28] A. Singh, Mac OS X Internals: a Systems Approach. Boston, MA,USA: Addison-Wesley Professional, 2016, OCLC: 1005337597.[29] A. Cobbs and Packet Design, LLC. (2002, Apr.)typed_mem—heap memory accounting system. [Online]. Avail-able: https://www.freebsd.org/cgi/man.cgi?query=FREE&manpath=FreeBSD+13.1-RELEASE+and+Ports[30] Pixar. (2022, Jul.) Universal Scene Description: The TfMallocTagMemory Tagging System. [Online]. Available: https://graphics.pixar.com/usd/dev/api/page_tf__malloc_tag.html[31] S. Baghdasaryan. (2022, Aug.) Code tagging frameworkand applications. [Online]. Available: https://lore.kernel.org/all/20220830214919.53220-1-surenb@google.com/[32] Free Software Foundation. (2022) The GNU C Library. [Online].Available: https://www.gnu.org/software/libc/[33] J. Evans, “Tick Tock, malloc Needs a Clock,” in Applicative 2015, ser.Applicative 2015. New York, NY, USA: Association for ComputingMachinery, 2015. [Online]. Available: https://doi.org/10.1145/2742580.2742807[34] ——. (2011, Jan.) Scalable memory allocation using jemalloc.[Online]. Available: https://engineering.fb.com/2011/01/03/core-data/scalable-memory-allocation-using-jemalloc/

Who Ate My Memory? Towards Attribution in Memory Management

Abstract

Similar works

Full text

Available Versions

ARTS repository - University of Groningen

Proceedings - University of Groningen

University of Groningen

Dissertations of the University of Groningen

arXiv.org e-Print Archive

Dissertations of the University of Groningen

Proceedings - University of Groningen

ARTS repository - University of Groningen