HTTP客户端警告：Going to buffer response body of large or unknown size

HTTP客户端警告：Going to buffer response body of large or unknown size

   点关注不迷路，欢迎再访！	

精简博客内容，尽量已行业术语来分享。

努力做到对每一位认可自己的读者负责。

帮助别人的同时更是丰富自己的良机。

目录

• • [HTTP客户端警告：Going to buffer response body of large or unknown size](#HTTP客户端警告：Going to buffer response body of large or unknown size)
  • • • 原编码问题
      • • [源码分析 getResponseBodyAsString()源码](#源码分析 getResponseBodyAsString()源码)
      • getResponseBodyAsStream()源码
      • • 优化原编码

原编码问题

HttpClient发现抛出Going to buffer response body of large or unknown size. Using getResponseBodyAsStream instead is recommended，查看原有代码中的逻辑如下：

HttpClient httpclient = new HttpClient();
GetMethod getMethod = new GetMethod(url);
int statusCode = httpclient.executeMethod(getMethod);
String respContent = getMethod.getResponseBodyAsString();

源码分析 getResponseBodyAsString()源码

	//getResponseBodyAsString()方法源码
    public String getResponseBodyAsString() throws IOException {
        byte[] rawdata = null;
        if (this.responseAvailable()) {
        	//调用了getResponseBody()，容易消耗内存
            rawdata = this.getResponseBody();
        }

        return rawdata != null ? EncodingUtil.getString(rawdata, this.getResponseCharSet()) : null;
    }
    //responseAvailable()方法源码
    private boolean responseAvailable() {
        return this.responseBody != null || this.responseStream != null;
    }
    
    //getResponseBody()方法源码
    public byte[] getResponseBody() throws IOException {
        if (this.responseBody == null) {
            InputStream instream = this.getResponseBodyAsStream();
            if (instream != null) {
                long contentLength = this.getResponseContentLength();
                if (contentLength > 2147483647L) {
                    throw new IOException("Content too large to be buffered: " + contentLength + " bytes");
                }

                int limit = this.getParams().getIntParameter("http.method.response.buffer.warnlimit", 1048576);
                if (contentLength == -1L || contentLength > (long)limit) {
                //这里是warn的原文
                    LOG.warn("Going to buffer response body of large or unknown size. Using getResponseBodyAsStream instead is recommended.");
                }

                LOG.debug("Buffering response body");
                ByteArrayOutputStream outstream = new ByteArrayOutputStream(contentLength > 0L ? (int)contentLength : 4096);
                byte[] buffer = new byte[4096];

                int len;
                while((len = instream.read(buffer)) > 0) {
                    outstream.write(buffer, 0, len);
                }

                outstream.close();
                this.setResponseStream((InputStream)null);
                this.responseBody = outstream.toByteArray();
            }
        }

        return this.responseBody;
    }

从源码中可以看出，warn的条件是(contentLength == -1L || contentLength > (long)limit)，如果http头没有指定contentLength或大于上限值（默认1M），就会抛异常。其实，如果返回的结果比较确定，对程序没有太大影响。而对于返回结果不确定时，源码也建议我们使用下面的getResponseBodyAsStream()方法。

getResponseBodyAsStream()源码

public InputStream getResponseBodyAsStream() throws IOException {
        if (this.responseStream != null) {
            return this.responseStream;
        } else if (this.responseBody != null) {
            InputStream byteResponseStream = new ByteArrayInputStream(this.responseBody);
            LOG.debug("re-creating response stream from byte array");
            return byteResponseStream;
        } else {
            return null;
        }
    }

从源码中可以看出，getResponseBodyAsStream()内部没有使用getResponseBody()方法，避免了内存耗尽问题，而是使用了InputStream流方式处理。

优化原编码

HttpClient httpclient = new HttpClient();
GetMethod getMethod = new GetMethod(url);
int statusCode = httpclient.executeMethod(getMethod);
//String respContent = getMethod.getResponseBodyAsString();
//使用getResponseBodyAsStream()
InputStream inputStream = getMethod.getResponseBodyAsStream();
BufferedReader br = new BufferedReader(new InputStreamReader(inputStream));
StringBuffer stringBuffer = new StringBuffer();
String str = "";
while ((str = br.readLine()) != null) {
       stringBuffer.append(str);
}
LOGGER.info("respContent: {}", stringBuffer.toString());