Jackson+Feign反序列化问题排查

概述

本文记录在使用Spring Cloud微服务开发时遇到的一个反序列化问题,RPC/HTTP框架使用的是Feign,JSON序列化反序列化工具是Jackson。

问题

测试环境的ELK告警日志如下:

复制代码
- [43f42bf7] 500 Server Error for HTTP POST "/api/open/dialog/nextQuestion"
feign.codec.DecodeException: Error while extracting response for type [AbaResponse<UserAccountVO>] 
and content type [application/json;charset=UTF-8]; 
nested exception is org.springframework.http.converter.HttpMessageNotReadableException:
JSON parse error: Expected array or string.; 
nested exception is com.fasterxml.jackson.databind.exc.MismatchedInputException: Expected array or string.
at [Source: (ByteArrayInputStream); line: 1, column: 295] (through reference chain: com.aba.common.utils.context.AbaResponse["data"]->com.aba.enduser.common.vo.UserAccountVO["privacySettings"]->java.util.LinkedHashMap["MINIMUM_LEGAL_AGE"]->com.aba.enduser.common.dto.account.PrivacySettings["timestamp"])
at feign.SynchronousMethodHandler.decode(SynchronousMethodHandler.java:180)
at feign.SynchronousMethodHandler.executeAndDecode(SynchronousMethodHandler.java:140)
at feign.SynchronousMethodHandler.invoke(SynchronousMethodHandler.java:78)
at feign.ReflectiveFeign$FeignInvocationHandler.invoke(ReflectiveFeign.java:103)

报错产生自gateway-open服务,gateway-open服务把接口请求/api/open/dialog/nextQuestion转发到dialog服务,dialog服务在Feign调用另外一个enduser服务时发生。很熟悉的报错,Feign反序列化问题。

排查

no Creators, like default construct, exist: cannot deserialize from Object value no delegate- or property-based Creator

为了排查问题,首先想到本地复现问题。本地启动dialog和enduser服务,postman请求dialog服务的接口/dialog/nextQuestion。却出现另一个问题,且这个报错发生在解析requestBody时。在Controller层方法里第一行加断点,程序都没在断点处停止,直接报错:

复制代码
Caught unhandled generic exception in com.aba.dialog.controller.DialogController
org.springframework.http.converter.HttpMessageConversionException: Type definition error: [simple type, class com.aba.dialog.service.domain.assessment.dialog.answer.DialogAnswerItem]; nested exception is com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Cannot construct instance of `com.aba.dialog.service.domain.assessment.dialog.answer.DialogAnswerItem` (no Creators, like default construct, exist): cannot deserialize from Object value (no delegate- or property-based Creator)
 at [Source: (PushbackInputStream); line: 1, column: 2]
at org.springframework.http.converter.json.AbstractJackson2HttpMessageConverter.readJavaType(AbstractJackson2HttpMessageConverter.java:242)
at org.springframework.http.converter.json.AbstractJackson2HttpMessageConverter.read(AbstractJackson2HttpMessageConverter.java:227)
at org.springframework.web.servlet.mvc.method.annotation.AbstractMessageConverterMethodArgumentResolver.readWithMessageConverters(AbstractMessageConverterMethodArgumentResolver.java:204)

dialog服务最近没有任何改动啊。enduser服务有改动,也和dialog服务无关;毕竟dialog服务断点没进去。

报错代码:

java 复制代码
@PostMapping(value = "/nextQuestion")
public DialogDTO handleDialog(@RequestBody DialogAnswerItem item) {
	// 断点行
    String platform = httpServletRequest.getHeader("dialogPlatform");
}

@RequestBody注解的POJO类:

kotlin 复制代码
data class DialogAnswerItem(val stateId: StateId,
                            var answer: GivenAnswer,
                            val progress: Double = 0.0,
                            val entryPoint: String? = null)

不甚熟悉的kotlin语言。

看起来一时半会搞不定。

Expected array or string

既然上面的问题没搞定,先解决测试环境的问题。本地启动第三个应用gateway服务,postman模拟调用gateway服务,由gateway负责转发。问题重现:

诸多分析,Google搜到一个靠谱的stackoverflow答案:feign-client-decodeexception-error-while-extracting-response

修改enduser服务代码:

java 复制代码
@Data
@AllArgsConstructor
@NoArgsConstructor
public class PrivacySettings implements Serializable {

    private Boolean value;

    @JsonSerialize(using = LocalDateTimeSerializer.class)
    @JsonDeserialize(using = LocalDateTimeDeserializer.class)
    private LocalDateTime timestamp;
}

本地调试,问题解决。

wait but why。

上面也提到【enduser服务有改动,也和dialog服务无关】,现在为了解决Feign + Jackson远程调用反序列化失败问题,去修改enduser代码,增加2个Jackson提供的注解@JsonSerialize@JsonDeserialize

问题虽然解决,总感觉哪里不对劲。但是测试环境里,前端等着使用相关接口,没成多想,发布测试环境。

Feign

结果发布到测试环境后,测试环境里ELK也记录到我一开始在本地调试重现问题时遇到的另外一个问题:
no Creators, like default construct, exist: cannot deserialize from Object value no delegate- or property-based Creator

看来这个问题是绕不过去的坎。诸般Google/百度搜索与尝试,始终没解决问题。

最后还是仔仔细细看Google给出的第一篇stackoverflow文章no-creators-like-default-construct-exist-cannot-deserialize-from-object-valu,看到:

register jackson module kotlin to ObjectMapper.

才突然意识到,最近对一个common-web组件库做了mvn clean deploy操作。deploy包括install,所以本地环境和测试环境都有相同问题。

再检查common-web下面的配置类:

java 复制代码
@Component
public class JsonConfig {
    /**
     * 解决JSON parse error: Unrecognized field "xxx"异常问题
     */
    @Bean
    public MappingJackson2HttpMessageConverter mappingJackson2HttpMessageConverter() {
        MappingJackson2HttpMessageConverter converter = new MappingJackson2HttpMessageConverter();
        ObjectMapper objectMapper = new ObjectMapper();
        objectMapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);
        converter.setObjectMapper(objectMapper);
        return converter;
    }
}

如上述代码里注释所述,增加此配置是为了解决JSON: Unrecognized field, not marked as ignorable问题,参考stackoverflow的问答jackson-with-json-unrecognized-field-not-marked-as-ignorable

之前在另外2个服务都出现过此问题,出现此问题的场景都是A服务调用B服务,B服务在业务开发时增加字段(杜绝修改字段和删除字段的开发bad practice)。A服务在微服务体系里还是在使用旧版本的B-api.jar,也就是说A服务的镜像里的jar里还是使用旧的版本,但是在Feign调用B服务时,B服务返回一个新版本的B-api.jar,多了一个字段。于是报错??

A服务重新编译新版本,则会把新版本的B-api.jar纳入到镜像里,也就是说发布新版本即可解决问题。

想要一劳永逸解决此类问题,在A服务里新增上述配置类就可以了吗?待验证。

考虑到Spring Cloud微服务体系,加字段是很常见的事情,那是不是可以把配置类放在common-web组件库,让所有服务都有此配置类。待验证。

正是因为上述猜想待验证,代码一直在本地。common-web组件库里其他类加以调整时,把JsonConfig配置类编译到dialog服务。

最后,两个问题的解决方法都是移除JsonConfig配置类,并且enduser服务的两个Jackson注解都可以revert。

问题是得以"解决",但是为啥呢?

后面仔细看dialog服务代码,好几个Jackson配置:

java 复制代码
@Configuration
@EnableAsync
open class ApplicationConfig {
    private val log = LoggerFactory.getLogger(this.javaClass)

    @Bean
    open fun restTemplateCommon(): RestTemplate {
        val restTemplate = RestTemplate()
        addOwnMappingJackson2HttpMessageConverter(restTemplate)
        val interceptors = listOf(
            ClientHttpRequestInterceptor { request, body, execution ->
                val headers = request.headers
                headers.add("Accept", MediaType.APPLICATION_JSON_VALUE)
                headers.add("Content-Type", MediaType.APPLICATION_JSON_VALUE)
                execution.execute(request, body)
            }
        )
        restTemplate.interceptors = interceptors
        return restTemplate
    }

    private fun addOwnMappingJackson2HttpMessageConverter(restTemplate: RestTemplate) {
        val converter = MappingJackson2HttpMessageConverter()
        val objectMapper = ObjectMapper()
            .findAndRegisterModules()
            // needed that the LocalDate is not serialized to [2000,1,1] but to "2000-01-01"
            .configure(SerializationFeature.WRITE_DATES_AS_TIMESTAMPS, false)
            .configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);
        converter.objectMapper = objectMapper

        val jacksonMappers = restTemplate.messageConverters
            .filter { httpMessageConverter -> httpMessageConverter is MappingJackson2HttpMessageConverter }

        if (jacksonMappers.isNotEmpty()) {
            restTemplate.messageConverters.remove(jacksonMappers.first())
        }
        restTemplate.messageConverters.add(1, converter)
    }

}

上面这个是kotlin语言。以及

java 复制代码
@Configuration
public class HttpConverterConfig implements WebMvcConfigurer {

    @Bean
    public MappingJackson2HttpMessageConverter mappingJackson2HttpMessageConverter() {
        AdaJackson2ObjectMapperBuilder adaJackson2ObjectMapperBuilder = new AdaJackson2ObjectMapperBuilder();
        return new MappingJackson2HttpMessageConverter(adaJackson2ObjectMapperBuilder.build()) {

            @Override
            protected void writeInternal(@NotNull Object object, Type type, @NotNull HttpOutputMessage outputMessage) throws IOException, HttpMessageNotWritableException {
                if (object instanceof String) {
                    Charset charset = this.getDefaultCharset();
                    StreamUtils.copy((String) object, charset, outputMessage.getBody());
                } else {
                    super.writeInternal(object, type, outputMessage);
                }
            }
        };
    }
}

以及:

java 复制代码
@Component
public class AdaJackson2ObjectMapperBuilder extends Jackson2ObjectMapperBuilder {

    public AdaJackson2ObjectMapperBuilder() {
        serializationInclusion(JsonInclude.Include.NON_NULL);
        serializationInclusion(JsonInclude.Include.NON_ABSENT);

        featuresToDisable(SerializationFeature.WRITE_DATES_AS_TIMESTAMPS,
                SerializationFeature.WRITE_DATE_TIMESTAMPS_AS_NANOSECONDS);
        modules(new AdaModule(), new GuavaModule(), new JavaTimeModule(), new Jdk8Module(), new ParameterNamesModule());
    }

    @Override
    public void configure(@NotNull ObjectMapper objectMapper) {
        super.configure(objectMapper);
        // disable constructor, getter and setter detection
        objectMapper.setVisibility(PropertyAccessor.ALL, JsonAutoDetect.Visibility.NONE);
        objectMapper.setVisibility(PropertyAccessor.FIELD, JsonAutoDetect.Visibility.ANY);
        objectMapper.registerModule(new KotlinModule());
    }

	private static class AdaModule extends SimpleModule {
	    public AdaModule() {
	        addSerializer(JSONError.class, new JSONErrorSerializer());
	    }
	}
}

以及:

java 复制代码
public class JSONErrorSerializer extends JsonSerializer<JSONError> {

    private static final String KEY_STATUS_CODE = "statusCode";
    private static final String KEY_ERROR = "error";
    private static final String KEY_MESSAGE = "message";

    @Override
    public void serialize(JSONError jsonError, JsonGenerator jsonGenerator, SerializerProvider serializerProvider) throws IOException {
        jsonGenerator.writeStartObject();
        jsonGenerator.writeStringField(KEY_STATUS_CODE, String.valueOf(jsonError.getStatusCode()));
        jsonGenerator.writeStringField(KEY_ERROR, jsonError.getError());
        if (jsonError.getMessage() != null && !jsonError.getMessage().isEmpty()) {
            jsonGenerator.writeStringField(KEY_MESSAGE, jsonError.getMessage());
        }
        jsonGenerator.writeEndObject();
    }
}

参考

相关推荐
hsx66610 小时前
Kotlin 协程中的 Dispatchers
kotlin
每次的天空13 小时前
Android-重学kotlin(协程源码第二阶段)新学习总结
android·学习·kotlin
stevenzqzq13 小时前
Kotlin 中主构造函数和次构造函数的区别
android·kotlin
开发者如是说16 小时前
言叶是如何对文件进行端到端加密的
android·kotlin·swift
小李飞飞砖17 小时前
kotlin
开发语言·单例模式·kotlin
小李飞飞砖17 小时前
kotlin中的冷流和热流
android·开发语言·kotlin
Kotlin上海用户组1 天前
Koin vs. Hilt——最流行的 Android DI 框架全方位对比
android·架构·kotlin
Kapaseker2 天前
当Object遇到Json你可能会碰到的坑
kotlin
RichardLai882 天前
Kotlin Flow:构建响应式流的现代 Kotlin 之道
android·前端·kotlin
程序员江同学2 天前
Kotlin/Native 编译流程浅析
android·kotlin