Spark 内存管理之StaticMemoryManager

xiaoxiao2021-02-28  84

概要

Spark 内存管理概述 介绍了内存管理接口MemoryManager对内存的大致划分,这篇研究其子类StaticMemoryManager,静态内存管理器,的行为,主要是各部分内存的占比。StaticMemoryManager是Spark 1.6之前唯一的内存管理器。

StaticMemoryManager

Spark 内存管理概述 介绍了MemoryManager的UML,StaticMemoryManager继承自MemoryManager,如下 

StaticMemoryManager分别使用ExecutionMemoryPool、StorageMemoryPool管理execution、storage、unroll内存,UML如下 

内存管理

回顾MemoryManager

MemoryManager将内存划分为如下几部分,接下来研究静态内存管理中下面各部分占比 

静态内存管理

之所以称为静态内存管理,是因为storage、execution内存占比和界限是固定的。看下代码中对storage、execution、unroll内存大小的限制

storage  execution  unroll 

总结如下

storage 0.6 storage-safety 0.6 * 0.9 * (1.0-0.2) unroll 0.6 * 0.9 * 0.2 storage-reserved 0.6 * 0.1 execution 0.2 execution-safety 0.2 * 0.8 execution-reserved 0.2 * 0.2 other 0.2  

饼图表示如下  因为无法准确计算实际使用中的内存,因此,storage、execution都会保留一部分内存,就是上图中的storage-reservedexecution-reserved

设置

spark 1.6版本之后默认使用统一资源管理器,由参数spark.memory.useLegacyMode控制,如下 

总结

介绍StaticMemoryManager的结构,及其管理的内存storage、execution、unroll等占比,这里所说的管理只是数值的维护。

(完)

https://www.douban.com/search?q=玩家汇娱乐平台总代QQ61777449 https://www.douban.com/search?q=ABC娱乐平台总代QQ61777449 https://www.douban.com/search?q=金尊娱乐平台总代QQ61777449 https://www.douban.com/search?q=名爵娱乐平台总代QQ61777449 https://www.douban.com/search?q=伟徳亚洲娱乐平台总代QQ61777449 https://www.douban.com/search?q=牛彩娱乐平台总代QQ61777449 https://www.douban.com/search?q=天音娱乐平台总代QQ61777449 https://www.douban.com/search?q=天云阁娱乐平台总代QQ61777449

http://home.babytree.com/u12992707376 http://home.babytree.com/u12992707376/journal/show/14230250 http://www.hdb.com/article/cycrb.html http://www.hdb.com/article/aqcrb.html http://www.hdb.com/article/tqcrb.html http://www.hdb.com/article/eqcrb.html http://www.hdb.com/article/jqcrb.html http://www.hdb.com/article/6qcrb.html http://www.hdb.com/article/uccrb.html http://www.hdb.com/article/accrb.html http://www.hdb.com/article/7ccrb.html http://www.hdb.com/article/eccrb.html http://www.hdb.com/article/jccrb.html http://www.hdb.com/article/8ccrb.html http://www.hdb.com/article/lccrb.html http://www.hdb.com/article/4ccrb.html http://www.hdb.com/article/qccrb.html http://www.hdb.com/article/cccrb.html http://www.hdb.com/article/gccrb.html http://www.hdb.com/article/zccrb.html http://www.hdb.com/article/9ccrb.html http://www.hdb.com/article/wccrb.html http://www.hdb.com/article/sccrb.html http://www.hdb.com/article/0gcrb.html http://www.hdb.com/article/2gcrb.html http://www.hdb.com/article/agcrb.html http://www.hdb.com/article/5gcrb.html http://www.hdb.com/article/7gcrb.html http://www.hdb.com/article/tgcrb.html http://www.hdb.com/article/pgcrb.html http://www.hdb.com/article/egcrb.html http://www.hdb.com/article/fgcrb.html http://www.hdb.com/article/sgcrb.html http://www.hdb.com/article/uzcrb.html http://www.hdb.com/article/kzcrb.html http://www.hdb.com/article/ezcrb.html http://www.hdb.com/article/6zcrb.html http://www.hdb.com/article/ozcrb.html http://www.hdb.com/article/rzcrb.html http://www.hdb.com/article/8zcrb.html http://www.hdb.com/article/yzcrb.html http://www.hdb.com/article/gzcrb.html http://www.hdb.com/article/zzcrb.html http://www.hdb.com/article/izcrb.html http://www.hdb.com/article/1zcrb.html http://www.hdb.com/article/hzcrb.html http://www.hdb.com/article/szcrb.html http://www.hdb.com/article/0icrb.html http://www.hdb.com/article/2icrb.html http://www.hdb.com/article/aicrb.html http://www.hdb.com/article/5icrb.html http://www.hdb.com/article/ticrb.html http://www.hdb.com/article/picrb.html http://www.hdb.com/article/gicrb.html http://www.hdb.com/article/b1crb.html http://www.hdb.com/article/21crb.html http://www.hdb.com/article/51crb.html http://www.hdb.com/article/t1crb.html http://www.hdb.com/article/p1crb.html http://www.hdb.com/article/k1crb.html http://www.hdb.com/article/e1crb.html http://www.hdb.com/article/31crb.html http://www.hdb.com/article/j1crb.html http://www.hdb.com/article/61crb.html http://www.hdb.com/article/z1crb.html http://www.hdb.com/article/11crb.html http://www.hdb.com/article/h1crb.html http://www.hdb.com/article/91crb.html http://www.hdb.com/article/d1crb.html http://www.hdb.com/article/w1crb.html http://www.hdb.com/article/f1crb.html http://www.hdb.com/article/s1crb.html http://www.hdb.com/article/x1crb.html http://www.hdb.com/article/uhcrb.html http://www.hdb.com/article/2hcrb.html http://www.hdb.com/article/7hcrb.html http://www.hdb.com/article/thcrb.html http://www.hdb.com/article/phcrb.html http://www.hdb.com/article/khcrb.html http://www.hdb.com/article/ehcrb.html http://www.hdb.com/article/jhcrb.html http://www.hdb.com/article/ohcrb.html http://www.hdb.com/article/rhcrb.html http://www.hdb.com/article/8hcrb.html http://www.hdb.com/article/lhcrb.html http://www.hdb.com/article/mhcrb.html http://www.hdb.com/article/yhcrb.html http://www.hdb.com/article/ghcrb.html http://www.hdb.com/article/zhcrb.html http://www.hdb.com/article/ihcrb.html http://www.hdb.com/article/1hcrb.html http://www.hdb.com/article/hhcrb.html http://www.hdb.com/article/vhcrb.html http://www.hdb.com/article/9hcrb.html http://www.hdb.com/article/dhcrb.html http://www.hdb.com/article/whcrb.html http://www.hdb.com/article/fhcrb.html http://www.hdb.com/article/shcrb.html http://www.hdb.com/article/xhcrb.html http://www.hdb.com/article/nvcrb.html http://www.hdb.com/article/avcrb.html http://www.hdb.com/article/5vcrb.html http://www.hdb.com/article/7vcrb.html http://www.hdb.com/article/tvcrb.html http://www.hdb.com/article/pvcrb.html http://www.hdb.com/article/kvcrb.html http://www.hdb.com/article/evcrb.html http://www.hdb.com/article/3vcrb.html http://www.hdb.com/article/jvcrb.html http://www.hdb.com/article/6vcrb.html http://www.hdb.com/article/ovcrb.html http://www.hdb.com/article/8vcrb.html http://www.hdb.com/article/b9crb.html http://www.hdb.com/article/29crb.html http://www.hdb.com/article/a9crb.html http://www.hdb.com/article/59crb.html http://www.hdb.com/article/79crb.html http://www.hdb.com/article/t9crb.html http://www.hdb.com/article/p9crb.html http://www.hdb.com/article/k9crb.html http://www.hdb.com/article/39crb.html http://www.hdb.com/article/j9crb.html http://www.hdb.com/article/69crb.html http://www.hdb.com/article/o9crb.html http://www.hdb.com/article/r9crb.html http://www.hdb.com/article/49crb.html http://www.hdb.com/article/q9crb.html http://www.hdb.com/article/g9crb.html http://www.hdb.com/article/z9crb.html http://www.hdb.com/article/i9crb.html http://www.hdb.com/article/19crb.html http://www.hdb.com/article/h9crb.html http://www.hdb.com/article/v9crb.html http://www.hdb.com/article/d9crb.html http://www.hdb.com/article/w9crb.html http://www.hdb.com/article/f9crb.html http://www.hdb.com/article/s9crb.html http://www.hdb.com/article/x9crb.html http://www.hdb.com/article/udcrb.html http://www.hdb.com/article/2dcrb.html http://www.hdb.com/article/ndcrb.html http://www.hdb.com/article/adcrb.html http://www.hdb.com/article/5dcrb.html http://www.hdb.com/article/7dcrb.html http://www.hdb.com/article/edcrb.html http://www.hdb.com/article/3dcrb.html http://www.hdb.com/article/jdcrb.html http://www.hdb.com/article/6dcrb.html http://www.hdb.com/article/odcrb.html http://www.hdb.com/article/rdcrb.html http://www.hdb.com/article/ldcrb.html http://www.hdb.com/article/mdcrb.html http://www.hdb.com/article/ydcrb.html http://www.hdb.com/article/4dcrb.html http://www.hdb.com/article/qdcrb.html http://www.hdb.com/article/cdcrb.html http://www.hdb.com/article/gdcrb.html http://www.hdb.com/article/zdcrb.html http://www.hdb.com/article/idcrb.html http://www.hdb.com/article/1dcrb.html

https://www.douban.com/search?q=优游娱乐平台总代-官网QQ61777449 https://www.douban.com/search?q=优游娱乐注册开户QQ61777449 https://www.douban.com/search?q=利信娱乐平台总代QQ61777449 https://www.douban.com/search?q=利信平台注册开户QQ61777449 https://www.douban.com/search?q=易购娱乐平台总代QQ61777449 https://www.douban.com/search?q=易购开户注册QQ61777449 https://www.douban.com/search?q=博猫娱乐平台总代QQ61777449 https://www.douban.com/search?q=博猫游戏平台总代QQ61777449 https://www.douban.com/search?q=博猫游戏平台注册QQ61777449 https://www.douban.com/search?q=博猫游戏平台开户QQ61777449 https://www.douban.com/search?q=易购娱乐平台总代QQ61777449 https://www.douban.com/search?q=亿万先生娱乐平台总代QQ61777449 https://www.douban.com/search?q=东森娱乐平台总代QQ61777449 https://www.douban.com/search?q=东森平台注册开户QQ61777449 https://www.douban.com/search?q=华人娱乐平台总代QQ61777449 https://www.douban.com/search?q=易购游戏开户注册QQ61777449 https://www.douban.com/search?q=易购娱乐平台总代官网QQ61777449 https://www.douban.com/search?q=优博娱乐平台总代QQ61777449 https://www.douban.com/search?q=大圣娱乐平台总代QQ61777449 https://www.douban.com/search?q=大圣游戏平台总代官网QQ61777449 https://www.douban.com/search?q=百利娱乐平台总代QQ61777449 https://www.douban.com/search?q=保时捷娱乐平台总代QQ61777449 https://www.douban.com/search?q=掌中彩娱乐平台总代QQ61777449 https://www.douban.com/search?q=帝苑club娱乐平台总代QQ61777449 https://www.douban.com/search?q=欢乐吧娱乐平台总代QQ61777449 https://www.douban.com/search?q=金沙娱乐平台总代QQ61777449

转载请注明原文地址: https://www.6miu.com/read-44370.html

最新回复(0)