解决retrofit OKhttp创建大量对外连接时内存溢出

xiaoxiao2025-04-23  14

解决retrofit OKhttp创建大量对外连接时内存溢出: https://blog.csdn.net/tianyaleixiaowu/article/details/78811488

这个问题是这样发生的,我的表中有一批数据,量级较大,数百万个,它们有个地址Address字段,标明了地理位置。我需要对这一批数据根据地址去百度或者高德地图去查询经纬度,并且保存下来。 原本是直接分页读取该表,每次读取几百条,然后一条一条去获取经纬度并且保存。后来发现实在太慢,一秒也就能处理个三五条。所以开启了多线程,大约30个线程,每个线程处理不同id范围的数据。 此时问题出现了,每个线程中都有for循环来分页读取DB中的地址数据,然后每条数据都要去百度地图请求一次,网络请求用的是retrofit,retrofit是包装的OKHttp。

可以看到线程数干到2000多时程序崩了

java.lang.OutOfMemoryError: unable to create new native thread at java.lang.Thread.start0(Native Method) at java.lang.Thread.start(Thread.java:714) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:950) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1368) at okhttp3.ConnectionPool.put(ConnectionPool.java:153) at okhttp3.OkHttpClient 1. p u t ( O k H t t p C l i e n t . j a v a : 163 ) a t o k h t t p 3. i n t e r n a l . c o n n e c t i o n . S t r e a m A l l o c a t i o n . f i n d C o n n e c t i o n ( S t r e a m A l l o c a t i o n . j a v a : 201 ) a t o k h t t p 3. i n t e r n a l . c o n n e c t i o n . S t r e a m A l l o c a t i o n . f i n d H e a l t h y C o n n e c t i o n ( S t r e a m A l l o c a t i o n . j a v a : 121 ) a t o k h t t p 3. i n t e r n a l . c o n n e c t i o n . S t r e a m A l l o c a t i o n . n e w S t r e a m ( S t r e a m A l l o c a t i o n . j a v a : 100 ) a t o k h t t p 3. i n t e r n a l . c o n n e c t i o n . C o n n e c t I n t e r c e p t o r . i n t e r c e p t ( C o n n e c t I n t e r c e p t o r . j a v a : 42 ) a t o k h t t p 3. i n t e r n a l . h t t p . R e a l I n t e r c e p t o r C h a i n . p r o c e e d ( R e a l I n t e r c e p t o r C h a i n . j a v a : 92 ) a t o k h t t p 3. i n t e r n a l . h t t p . R e a l I n t e r c e p t o r C h a i n . p r o c e e d ( R e a l I n t e r c e p t o r C h a i n . j a v a : 67 ) a t o k h t t p 3. i n t e r n a l . c a c h e . C a c h e I n t e r c e p t o r . i n t e r c e p t ( C a c h e I n t e r c e p t o r . j a v a : 93 ) a t o k h t t p 3. i n t e r n a l . h t t p . R e a l I n t e r c e p t o r C h a i n . p r o c e e d ( R e a l I n t e r c e p t o r C h a i n . j a v a : 92 ) a t o k h t t p 3. i n t e r n a l . h t t p . R e a l I n t e r c e p t o r C h a i n . p r o c e e d ( R e a l I n t e r c e p t o r C h a i n . j a v a : 67 ) a t o k h t t p 3. i n t e r n a l . h t t p . B r i d g e I n t e r c e p t o r . i n t e r c e p t ( B r i d g e I n t e r c e p t o r . j a v a : 93 ) a t o k h t t p 3. i n t e r n a l . h t t p . R e a l I n t e r c e p t o r C h a i n . p r o c e e d ( R e a l I n t e r c e p t o r C h a i n . j a v a : 92 ) a t o k h t t p 3. i n t e r n a l . h t t p . R e t r y A n d F o l l o w U p I n t e r c e p t o r . i n t e r c e p t ( R e t r y A n d F o l l o w U p I n t e r c e p t o r . j a v a : 120 ) a t o k h t t p 3. i n t e r n a l . h t t p . R e a l I n t e r c e p t o r C h a i n . p r o c e e d ( R e a l I n t e r c e p t o r C h a i n . j a v a : 92 ) a t o k h t t p 3. i n t e r n a l . h t t p . R e a l I n t e r c e p t o r C h a i n . p r o c e e d ( R e a l I n t e r c e p t o r C h a i n . j a v a : 67 ) a t c o m . m i n d a t a . e c s e r v e r . g l o b a l . h t t p . R e t r o f i t S e r v i c e B u i l d e r . l a m b d a 1.put(OkHttpClient.java:163) at okhttp3.internal.connection.StreamAllocation.findConnection(StreamAllocation.java:201) at okhttp3.internal.connection.StreamAllocation.findHealthyConnection(StreamAllocation.java:121) at okhttp3.internal.connection.StreamAllocation.newStream(StreamAllocation.java:100) at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:42) at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92) at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67) at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:93) at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92) at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67) at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93) at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92) at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:120) at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92) at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67) at com.mindata.ecserver.global.http.RetrofitServiceBuilder.lambda 1.put(OkHttpClient.java:163)atokhttp3.internal.connection.StreamAllocation.findConnection(StreamAllocation.java:201)atokhttp3.internal.connection.StreamAllocation.findHealthyConnection(StreamAllocation.java:121)atokhttp3.internal.connection.StreamAllocation.newStream(StreamAllocation.java:100)atokhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:42)atokhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)atokhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67)atokhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:93)atokhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)atokhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67)atokhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93)atokhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)atokhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:120)atokhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)atokhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67)atcom.mindata.ecserver.global.http.RetrofitServiceBuilder.lambdagenerateClient 0 ( R e t r o f i t S e r v i c e B u i l d e r . j a v a : 72 ) a t c o m . m i n d a t a . e c s e r v e r . g l o b a l . h t t p . R e t r o f i t S e r v i c e B u i l d e r 0(RetrofitServiceBuilder.java:72) at com.mindata.ecserver.global.http.RetrofitServiceBuilder 0(RetrofitServiceBuilder.java:72)atcom.mindata.ecserver.global.http.RetrofitServiceBuilder$Lambda$11/620324267.intercept(Unknown Source) at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92) at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67) at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:185) at okhttp3.RealCall.execute(RealCall.java:69) at retrofit2.OkHttpCall.execute(OkHttpCall.java:180) at com.mindata.ecserver.global.http.CallManager.execute(CallManager.java:25)

通过监控界面可以看到,大量的线程OKHttp ConnectionPool,也是导致内存溢出的主要原因。

作者:天涯泪小武 来源: 原文:https://blog.csdn.net/tianyaleixiaowu/article/details/78811488 版权声明:本文为博主原创文章,转载请附上博文链接!


作者:天涯泪小武 来源: 原文:https://blog.csdn.net/tianyaleixiaowu/article/details/78811488 版权声明:本文为博主原创文章,转载请附上博文链接!

转载请注明原文地址: https://www.6miu.com/read-5028919.html

最新回复(0)