SpringBoot离线应用的5种实现方式

在当今高度依赖网络的环境中,离线应用的价值日益凸显。无论是在网络不稳定的区域运行的现场系统,还是需要在断网环境下使用的企业内部应用,具备离线工作能力已成为许多应用的必备特性。

本文将介绍基于SpringBoot实现离线应用的5种不同方式。

一、离线应用的概念与挑战

离线应用(Offline Application)是指能够在网络连接不可用的情况下,仍然能够正常运行并提供核心功能的应用程序。这类应用通常具备以下特点:

  1. 本地数据存储:能够在本地存储和读取数据
  2. 操作缓存:能够缓存用户操作,待网络恢复后同步
  3. 资源本地化:应用资源(如静态资源、配置等)可以在本地访问
  4. 状态管理:维护应用状态,处理在线/离线切换

实现离线应用面临的主要挑战包括:数据存储与同步、冲突解决、用户体验设计以及安全性考虑。

二、嵌入式数据库实现离线数据存储

原理介绍

嵌入式数据库直接集成在应用程序中,无需外部数据库服务器,非常适合离线应用场景。

在SpringBoot中,可以轻松集成H2、SQLite、HSQLDB等嵌入式数据库。

实现步骤

  1. 添加依赖
xml 复制代码
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-data-jpa</artifactId>
</dependency>
<dependency>
    <groupId>com.h2database</groupId>
    <artifactId>h2</artifactId>
    <scope>runtime</scope>
</dependency>
  1. 配置文件
ini 复制代码
# 使用文件模式的H2数据库,支持持久化
spring.datasource.url=jdbc:h2:file:./data/offlinedb
spring.datasource.driverClassName=org.h2.Driver
spring.datasource.username=sa
spring.datasource.password=password
spring.jpa.database-platform=org.hibernate.dialect.H2Dialect

# 自动创建表结构
spring.jpa.hibernate.ddl-auto=update

# 启用H2控制台(开发环境)
spring.h2.console.enabled=true
spring.h2.console.path=/h2-console
  1. 创建实体类
less 复制代码
@Entity
@Table(name = "offline_data")
public class OfflineData {
    
    @Id
    @GeneratedValue(strategy = GenerationType.IDENTITY)
    private Long id;
    
    private String content;
    
    @Column(name = "is_synced")
    private boolean synced;
    
    @Column(name = "created_at")
    private LocalDateTime createdAt;
    
    // 构造函数、getter和setter
}
  1. 创建Repository
csharp 复制代码
@Repository
public interface OfflineDataRepository extends JpaRepository<OfflineData, Long> {
    List<OfflineData> findBySyncedFalse();
}
  1. 创建Service
typescript 复制代码
@Service
public class OfflineDataService {
    
    private final OfflineDataRepository repository;
    
    @Autowired
    public OfflineDataService(OfflineDataRepository repository) {
        this.repository = repository;
    }
    
    // 保存本地数据
    public OfflineData saveData(String content) {
        OfflineData data = new OfflineData();
        data.setContent(content);
        data.setSynced(false);
        data.setCreatedAt(LocalDateTime.now());
        return repository.save(data);
    }
    
    // 获取所有未同步的数据
    public List<OfflineData> getUnsyncedData() {
        return repository.findBySyncedFalse();
    }
    
    // 标记数据为已同步
    public void markAsSynced(Long id) {
        repository.findById(id).ifPresent(data -> {
            data.setSynced(true);
            repository.save(data);
        });
    }
    
    // 当网络恢复时,同步数据到远程服务器
    @Scheduled(fixedDelay = 60000) // 每分钟检查一次
    public void syncDataToRemote() {
        List<OfflineData> unsyncedData = getUnsyncedData();
        if (!unsyncedData.isEmpty()) {
            try {
                // 尝试连接远程服务器
                if (isNetworkAvailable()) {
                    for (OfflineData data : unsyncedData) {
                        boolean syncSuccess = sendToRemoteServer(data);
                        if (syncSuccess) {
                            markAsSynced(data.getId());
                        }
                    }
                }
            } catch (Exception e) {
                // 同步失败,下次再试
                log.error("Failed to sync data: " + e.getMessage());
            }
        }
    }
    
    private boolean isNetworkAvailable() {
        // 实现网络检测逻辑
        try {
            InetAddress address = InetAddress.getByName("api.example.com");
            return address.isReachable(3000); // 3秒超时
        } catch (Exception e) {
            return false;
        }
    }
    
    private boolean sendToRemoteServer(OfflineData data) {
        // 实现发送数据到远程服务器的逻辑
        // 这里使用RestTemplate示例
        try {
            RestTemplate restTemplate = new RestTemplate();
            ResponseEntity<String> response = restTemplate.postForEntity(
                "https://api.example.com/data", 
                data, 
                String.class
            );
            return response.getStatusCode().isSuccessful();
        } catch (Exception e) {
            log.error("Failed to send data: " + e.getMessage());
            return false;
        }
    }
}
  1. 创建Controller
kotlin 复制代码
@RestController
@RequestMapping("/api/data")
public class OfflineDataController {
    
    private final OfflineDataService service;
    
    @Autowired
    public OfflineDataController(OfflineDataService service) {
        this.service = service;
    }
    
    @PostMapping
    public ResponseEntity<OfflineData> createData(@RequestBody String content) {
        OfflineData savedData = service.saveData(content);
        return ResponseEntity.ok(savedData);
    }
    
    @GetMapping("/unsynced")
    public ResponseEntity<List<OfflineData>> getUnsyncedData() {
        return ResponseEntity.ok(service.getUnsyncedData());
    }
    
    @PostMapping("/sync")
    public ResponseEntity<String> triggerSync() {
        service.syncDataToRemote();
        return ResponseEntity.ok("Sync triggered");
    }
}

优缺点分析

优点:

  • 完全本地化的数据存储,无需网络连接
  • 支持完整的SQL功能,可以进行复杂查询
  • 数据持久化到本地文件,应用重启不丢失

缺点:

  • 嵌入式数据库性能和并发处理能力有限
  • 占用本地存储空间,需要注意容量管理
  • 数据同步逻辑需要自行实现
  • 复杂的冲突解决场景处理困难

适用场景

  • 需要结构化数据存储的单机应用
  • 定期需要将数据同步到中心服务器的现场应用
  • 对数据查询有SQL需求的离线系统
  • 数据量适中的企业内部工具

三、本地缓存与离线数据访问策略

原理介绍

本方案利用Java内存缓存框架(如Caffeine、Ehcache)结合本地持久化存储,实现数据的本地缓存和离线访问。该方案特别适合读多写少的应用场景。

实现步骤

  1. 添加依赖
xml 复制代码
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-cache</artifactId>
</dependency>
<dependency>
    <groupId>com.github.ben-manes.caffeine</groupId>
    <artifactId>caffeine</artifactId>
</dependency>
<dependency>
    <groupId>com.fasterxml.jackson.core</groupId>
    <artifactId>jackson-databind</artifactId>
</dependency>
  1. 配置缓存
typescript 复制代码
@Configuration
@EnableCaching
public class CacheConfig {
    
    @Bean
    public Caffeine<Object, Object> caffeineConfig() {
        return Caffeine.newBuilder()
                .expireAfterWrite(1, TimeUnit.DAYS)
                .initialCapacity(100)
                .maximumSize(1000)
                .recordStats();
    }
    
    @Bean
    public CacheManager cacheManager(Caffeine<Object, Object> caffeine) {
        CaffeineCacheManager cacheManager = new CaffeineCacheManager();
        cacheManager.setCaffeine(caffeine);
        return cacheManager;
    }
    
    @Bean
    public CacheSerializer cacheSerializer() {
        return new CacheSerializer();
    }
}
  1. 创建缓存序列化器
typescript 复制代码
@Component
public class CacheSerializer {
    
    private final ObjectMapper objectMapper = new ObjectMapper();
    private final File cacheDir = new File("./cache");
    
    public CacheSerializer() {
        if (!cacheDir.exists()) {
            cacheDir.mkdirs();
        }
    }
    
    public void serializeCache(String cacheName, Map<Object, Object> entries) {
        try {
            File cacheFile = new File(cacheDir, cacheName + ".json");
            objectMapper.writeValue(cacheFile, entries);
        } catch (IOException e) {
            throw new RuntimeException("Failed to serialize cache: " + cacheName, e);
        }
    }
    
    @SuppressWarnings("unchecked")
    public Map<Object, Object> deserializeCache(String cacheName) {
        File cacheFile = new File(cacheDir, cacheName + ".json");
        if (!cacheFile.exists()) {
            return new HashMap<>();
        }
        
        try {
            return objectMapper.readValue(cacheFile, Map.class);
        } catch (IOException e) {
            throw new RuntimeException("Failed to deserialize cache: " + cacheName, e);
        }
    }
}
  1. 创建离线数据服务
typescript 复制代码
@Service
@Slf4j
public class ProductService {
    
    private final RestTemplate restTemplate;
    private final CacheSerializer cacheSerializer;
    
    private static final String CACHE_NAME = "products";
    
    @Autowired
    public ProductService(RestTemplate restTemplate, CacheSerializer cacheSerializer) {
        this.restTemplate = restTemplate;
        this.cacheSerializer = cacheSerializer;
        // 初始化时加载持久化的缓存
        loadCacheFromDisk();
    }
    
    @Cacheable(cacheNames = CACHE_NAME, key = "#id")
    public Product getProductById(Long id) {
        try {
            // 尝试从远程服务获取
            return restTemplate.getForObject("https://api.example.com/products/" + id, Product.class);
        } catch (Exception e) {
            // 网络不可用时,尝试从持久化缓存获取
            Map<Object, Object> diskCache = cacheSerializer.deserializeCache(CACHE_NAME);
            Product product = (Product) diskCache.get(id.toString());
            if (product != null) {
                return product;
            }
            throw new ProductNotFoundException("Product not found in cache: " + id);
        }
    }
    
    @Cacheable(cacheNames = CACHE_NAME)
    public List<Product> getAllProducts() {
        try {
            // 尝试从远程服务获取
            Product[] products = restTemplate.getForObject("https://api.example.com/products", Product[].class);
            return products != null ? Arrays.asList(products) : Collections.emptyList();
        } catch (Exception e) {
            // 网络不可用时,返回所有持久化缓存的产品
            Map<Object, Object> diskCache = cacheSerializer.deserializeCache(CACHE_NAME);
            return new ArrayList<>(diskCache.values());
        }
    }
    
    @CachePut(cacheNames = CACHE_NAME, key = "#product.id")
    public Product saveProduct(Product product) {
        try {
            // 尝试保存到远程服务
            return restTemplate.postForObject("https://api.example.com/products", product, Product.class);
        } catch (Exception e) {
            // 网络不可用时,只保存到本地缓存
            product.setOfflineSaved(true);
            
            // 同时更新持久化缓存
            Map<Object, Object> diskCache = cacheSerializer.deserializeCache(CACHE_NAME);
            diskCache.put(product.getId().toString(), product);
            cacheSerializer.serializeCache(CACHE_NAME, diskCache);
            
            return product;
        }
    }
    
    @Scheduled(fixedDelay = 300000) // 每5分钟
    public void persistCacheToDisk() {
        Cache cache = cacheManager.getCache(CACHE_NAME);
        if (cache != null) {
            Map<Object, Object> entries = new HashMap<>();
            cache.getNativeCache().asMap().forEach(entries::put);
            cacheSerializer.serializeCache(CACHE_NAME, entries);
        }
    }
    
    @Scheduled(fixedDelay = 600000) // 每10分钟
    public void syncOfflineData() {
        if (!isNetworkAvailable()) {
            return;
        }
        
        Map<Object, Object> diskCache = cacheSerializer.deserializeCache(CACHE_NAME);
        for (Object value : diskCache.values()) {
            Product product = (Product) value;
            if (product.isOfflineSaved()) {
                try {
                    restTemplate.postForObject("https://api.example.com/products", product, Product.class);
                    product.setOfflineSaved(false);
                } catch (Exception e) {
                    // 同步失败,下次再试
                    log.error(e.getMessage(),e);
                }
            }
        }
        
        // 更新持久化缓存
        cacheSerializer.serializeCache(CACHE_NAME, diskCache);
    }
    
    private void loadCacheFromDisk() {
        Map<Object, Object> diskCache = cacheSerializer.deserializeCache(CACHE_NAME);
        Cache cache = cacheManager.getCache(CACHE_NAME);
        if (cache != null) {
            diskCache.forEach((key, value) -> cache.put(key, value));
        }
    }
    
    private boolean isNetworkAvailable() {
        try {
            return InetAddress.getByName("api.example.com").isReachable(3000);
        } catch (Exception e) {
            return false;
        }
    }
}
  1. 创建数据模型
typescript 复制代码
@Data
public class Product implements Serializable {
    private Long id;
    private String name;
    private String description;
    private BigDecimal price;
    private boolean offlineSaved;
}
  1. 创建Controller
kotlin 复制代码
@RestController
@RequestMapping("/api/products")
public class ProductController {
    
    private final ProductService productService;
    
    @Autowired
    public ProductController(ProductService productService) {
        this.productService = productService;
    }
    
    @GetMapping("/{id}")
    public ResponseEntity<Product> getProductById(@PathVariable Long id) {
        try {
            return ResponseEntity.ok(productService.getProductById(id));
        } catch (ProductNotFoundException e) {
            return ResponseEntity.notFound().build();
        }
    }
    
    @GetMapping
    public ResponseEntity<List<Product>> getAllProducts() {
        return ResponseEntity.ok(productService.getAllProducts());
    }
    
    @PostMapping
    public ResponseEntity<Product> createProduct(@RequestBody Product product) {
        return ResponseEntity.ok(productService.saveProduct(product));
    }
    
    @GetMapping("/sync")
    public ResponseEntity<String> triggerSync() {
        productService.syncOfflineData();
        return ResponseEntity.ok("Sync triggered");
    }
}

优缺点分析

优点:

  • 内存缓存访问速度快,用户体验好
  • 结合本地持久化,支持应用重启后恢复缓存
  • 适合读多写少的应用场景

缺点:

  • 缓存同步和冲突解决逻辑复杂
  • 大量数据缓存会占用较多内存
  • 不适合频繁写入的场景
  • 缓存序列化和反序列化有性能开销

适用场景

  • 产品目录、知识库等读多写少的应用
  • 需要快速响应的用户界面
  • 有限的数据集合且结构相对固定
  • 偶尔离线使用的Web应用

四、离线优先架构与本地存储引擎

原理介绍

离线优先架构(Offline-First)是一种设计理念,它将离线状态视为应用的默认状态,而不是异常状态。

在这种架构中,数据首先存储在本地,然后在条件允许时同步到服务器。

该方案使用嵌入式KV存储(如LevelDB、RocksDB)作为本地存储引擎。

实现步骤

  1. 添加依赖
xml 复制代码
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
    <groupId>org.iq80.leveldb</groupId>
    <artifactId>leveldb</artifactId>
    <version>0.12</version>
</dependency>
<dependency>
    <groupId>com.fasterxml.jackson.core</groupId>
    <artifactId>jackson-databind</artifactId>
</dependency>
  1. 创建LevelDB存储服务
java 复制代码
@Component
public class LevelDBStore implements InitializingBean, DisposableBean {
    
    private DB db;
    private final ObjectMapper objectMapper = new ObjectMapper();
    private final File dbDir = new File("./leveldb");
    
    @Override
    public void afterPropertiesSet() throws Exception {
        Options options = new Options();
        options.createIfMissing(true);
        db = factory.open(dbDir, options);
    }
    
    @Override
    public void destroy() throws Exception {
        if (db != null) {
            db.close();
        }
    }
    
    public <T> void put(String key, T value) {
        try {
            byte[] serialized = objectMapper.writeValueAsBytes(value);
            db.put(bytes(key), serialized);
        } catch (Exception e) {
            throw new RuntimeException("Failed to store data: " + key, e);
        }
    }
    
    public <T> T get(String key, Class<T> type) {
        try {
            byte[] data = db.get(bytes(key));
            if (data == null) {
                return null;
            }
            return objectMapper.readValue(data, type);
        } catch (Exception e) {
            throw new RuntimeException("Failed to retrieve data: " + key, e);
        }
    }
    
    public <T> List<T> getAll(String prefix, Class<T> type) {
        List<T> result = new ArrayList<>();
        try (DBIterator iterator = db.iterator()) {
            byte[] prefixBytes = bytes(prefix);
            for (iterator.seek(prefixBytes); iterator.hasNext(); iterator.next()) {
                String key = asString(iterator.peekNext().getKey());
                if (!key.startsWith(prefix)) {
                    break;
                }
                T value = objectMapper.readValue(iterator.peekNext().getValue(), type);
                result.add(value);
            }
        } catch (Exception e) {
            throw new RuntimeException("Failed to retrieve data with prefix: " + prefix, e);
        }
        return result;
    }
    
    public boolean delete(String key) {
        try {
            db.delete(bytes(key));
            return true;
        } catch (Exception e) {
            return false;
        }
    }
    
    private byte[] bytes(String s) {
        return s.getBytes(StandardCharsets.UTF_8);
    }
    
    private String asString(byte[] bytes) {
        return new String(bytes, StandardCharsets.UTF_8);
    }
}
  1. 创建离线同步管理器
typescript 复制代码
@Component
public class SyncManager {
    
    private final LevelDBStore store;
    private final RestTemplate restTemplate;
    
    @Value("${sync.server.url}")
    private String syncServerUrl;
    
    @Autowired
    public SyncManager(LevelDBStore store, RestTemplate restTemplate) {
        this.store = store;
        this.restTemplate = restTemplate;
    }
    
    // 保存并跟踪离线操作
    public <T> void saveOperation(String type, String id, T data) {
        String key = "op:" + type + ":" + id;
        OfflineOperation<T> operation = new OfflineOperation<>(
            UUID.randomUUID().toString(),
            type,
            id,
            data,
            System.currentTimeMillis()
        );
        store.put(key, operation);
    }
    
    // 同步所有未同步的操作
    @Scheduled(fixedDelay = 60000) // 每分钟尝试同步
    public void syncOfflineOperations() {
        if (!isNetworkAvailable()) {
            return;
        }
        
        List<OfflineOperation<?>> operations = store.getAll("op:", OfflineOperation.class);
        
        // 按时间戳排序,确保按操作顺序同步
        operations.sort(Comparator.comparing(OfflineOperation::getTimestamp));
        
        for (OfflineOperation<?> operation : operations) {
            boolean success = sendToServer(operation);
            if (success) {
                // 同步成功后删除本地操作记录
                store.delete("op:" + operation.getType() + ":" + operation.getId());
            } else {
                // 同步失败,下次再试
                break;
            }
        }
    }
    
    private boolean sendToServer(OfflineOperation<?> operation) {
        try {
            HttpMethod method;
            switch (operation.getType()) {
                case "CREATE":
                    method = HttpMethod.POST;
                    break;
                case "UPDATE":
                    method = HttpMethod.PUT;
                    break;
                case "DELETE":
                    method = HttpMethod.DELETE;
                    break;
                default:
                    return false;
            }
            
            // 构建请求URL
            String url = syncServerUrl + "/" + operation.getId();
            if ("DELETE".equals(operation.getType())) {
                // DELETE请求通常不需要请求体
                ResponseEntity<Void> response = restTemplate.exchange(
                    url, method, null, Void.class
                );
                return response.getStatusCode().is2xxSuccessful();
            } else {
                // POST和PUT请求需要请求体
                HttpEntity<Object> request = new HttpEntity<>(operation.getData());
                ResponseEntity<Object> response = restTemplate.exchange(
                    url, method, request, Object.class
                );
                return response.getStatusCode().is2xxSuccessful();
            }
        } catch (Exception e) {
            return false;
        }
    }
    
    private boolean isNetworkAvailable() {
        try {
            URL url = new URL(syncServerUrl);
            HttpURLConnection connection = (HttpURLConnection) url.openConnection();
            connection.setConnectTimeout(3000);
            connection.connect();
            return connection.getResponseCode() == 200;
        } catch (Exception e) {
            return false;
        }
    }
    
    @Data
    @AllArgsConstructor
    private static class OfflineOperation<T> {
        private String operationId;
        private String type; // CREATE, UPDATE, DELETE
        private String id;
        private T data;
        private long timestamp;
    }
}
  1. 创建任务服务
arduino 复制代码
@Service
public class TaskService {
    
    private final LevelDBStore store;
    private final SyncManager syncManager;
    
    @Autowired
    public TaskService(LevelDBStore store, SyncManager syncManager) {
        this.store = store;
        this.syncManager = syncManager;
    }
    
    public Task getTaskById(String id) {
        return store.get("task:" + id, Task.class);
    }
    
    public List<Task> getAllTasks() {
        return store.getAll("task:", Task.class);
    }
    
    public Task createTask(Task task) {
        // 生成ID
        if (task.getId() == null) {
            task.setId(UUID.randomUUID().toString());
        }
        
        // 设置时间戳
        task.setCreatedAt(System.currentTimeMillis());
        task.setUpdatedAt(System.currentTimeMillis());
        
        // 保存到本地存储
        store.put("task:" + task.getId(), task);
        
        // 记录离线操作,等待同步
        syncManager.saveOperation("CREATE", task.getId(), task);
        
        return task;
    }
    
    public Task updateTask(String id, Task task) {
        Task existingTask = getTaskById(id);
        if (existingTask == null) {
            throw new RuntimeException("Task not found: " + id);
        }
        
        // 更新字段
        task.setId(id);
        task.setCreatedAt(existingTask.getCreatedAt());
        task.setUpdatedAt(System.currentTimeMillis());
        
        // 保存到本地存储
        store.put("task:" + id, task);
        
        // 记录离线操作,等待同步
        syncManager.saveOperation("UPDATE", id, task);
        
        return task;
    }
    
    public boolean deleteTask(String id) {
        Task existingTask = getTaskById(id);
        if (existingTask == null) {
            return false;
        }
        
        // 从本地存储删除
        boolean deleted = store.delete("task:" + id);
        
        // 记录离线操作,等待同步
        if (deleted) {
            syncManager.saveOperation("DELETE", id, null);
        }
        
        return deleted;
    }
}
  1. 创建任务模型
arduino 复制代码
@Data
public class Task {
    private String id;
    private String title;
    private String description;
    private boolean completed;
    private long createdAt;
    private long updatedAt;
}
  1. 创建Controller
less 复制代码
@RestController
@RequestMapping("/api/tasks")
public class TaskController {
    
    private final TaskService taskService;
    
    @Autowired
    public TaskController(TaskService taskService) {
        this.taskService = taskService;
    }
    
    @GetMapping("/{id}")
    public ResponseEntity<Task> getTaskById(@PathVariable String id) {
        Task task = taskService.getTaskById(id);
        if (task == null) {
            return ResponseEntity.notFound().build();
        }
        return ResponseEntity.ok(task);
    }
    
    @GetMapping
    public ResponseEntity<List<Task>> getAllTasks() {
        return ResponseEntity.ok(taskService.getAllTasks());
    }
    
    @PostMapping
    public ResponseEntity<Task> createTask(@RequestBody Task task) {
        return ResponseEntity.ok(taskService.createTask(task));
    }
    
    @PutMapping("/{id}")
    public ResponseEntity<Task> updateTask(@PathVariable String id, @RequestBody Task task) {
        try {
            return ResponseEntity.ok(taskService.updateTask(id, task));
        } catch (Exception e) {
            return ResponseEntity.notFound().build();
        }
    }
    
    @DeleteMapping("/{id}")
    public ResponseEntity<Void> deleteTask(@PathVariable String id) {
        boolean deleted = taskService.deleteTask(id);
        if (deleted) {
            return ResponseEntity.noContent().build();
        }
        return ResponseEntity.notFound().build();
    }
    
    @PostMapping("/sync")
    public ResponseEntity<String> triggerSync() {
        return ResponseEntity.ok("Sync triggered");
    }
}
  1. 配置文件
ini 复制代码
# 同步服务器地址
sync.server.url=https://api.example.com/tasks

优缺点分析

优点:

  • 离线优先设计,保证应用在任何网络状态下可用
  • 高性能的本地存储引擎,适合大量数据
  • 支持完整的CRUD操作和离线同步
  • 细粒度的操作跟踪,便于解决冲突

缺点:

  • 实现复杂度较高
  • 同步策略需要根据业务场景定制
  • 不支持复杂的关系型查询

适用场景

  • 需要全面离线支持的企业应用
  • 现场操作类系统,如仓库管理、物流系统
  • 数据量较大的离线应用
  • 需要严格保证离线和在线数据一致性的场景

五、嵌入式消息队列与异步处理

原理介绍

该方案使用嵌入式消息队列(如ActiveMQ Artemis嵌入模式)实现离线操作的异步处理和持久化。

操作被发送到本地队列,在网络恢复后批量处理。

实现步骤

  1. 添加依赖
xml 复制代码
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-artemis</artifactId>
</dependency>
<dependency>
    <groupId>org.apache.activemq</groupId>
    <artifactId>artemis-server</artifactId>
</dependency>
<dependency>
    <groupId>org.apache.activemq</groupId>
    <artifactId>artemis-jms-server</artifactId>
</dependency>
  1. 配置嵌入式Artemis
arduino 复制代码
@Configuration
@Slf4j
public class ArtemisConfig {
    
    @Value("${artemis.embedded.data-directory:./artemis-data}")
    private String dataDirectory;
    
    @Value("${artemis.embedded.queues:offlineOperations}")
    private String queues;
    
    @Bean
    public ActiveMQServer activeMQServer() throws Exception {
        Configuration config = new ConfigurationImpl();
        config.setPersistenceEnabled(true);
        config.setJournalDirectory(dataDirectory + "/journal");
        config.setBindingsDirectory(dataDirectory + "/bindings");
        config.setLargeMessagesDirectory(dataDirectory + "/largemessages");
        config.setPagingDirectory(dataDirectory + "/paging");
        
        config.addAcceptorConfiguration("in-vm", "vm://0");
        config.addAddressSetting("#", 
                new AddressSettings()
                    .setDeadLetterAddress(SimpleString.toSimpleString("DLQ"))
                    .setExpiryAddress(SimpleString.toSimpleString("ExpiryQueue")));
        
        ActiveMQServer server = new ActiveMQServerImpl(config);
        server.start();
        
        // 创建队列
        Arrays.stream(queues.split(","))
                .forEach(queue -> {
                    try {
                        server.createQueue(
                            SimpleString.toSimpleString(queue),
                            RoutingType.ANYCAST,
                            SimpleString.toSimpleString(queue),
                            null,
                            true,
                            false
                        );
                    } catch (Exception e) {
                        log.error(e.getMessage(),e);
                    }
                });
        
        return server;
    }
    
    @Bean
    public ConnectionFactory connectionFactory() {
        return new ActiveMQConnectionFactory("vm://0");
    }
    
    @Bean
    public JmsTemplate jmsTemplate(ConnectionFactory connectionFactory) {
        JmsTemplate template = new JmsTemplate(connectionFactory);
        template.setDeliveryPersistent(true);
        return template;
    }
}
  1. 创建离线操作消息服务
typescript 复制代码
@Service
public class OfflineMessageService {
    
    private final JmsTemplate jmsTemplate;
    private final ObjectMapper objectMapper;
    
    @Value("${artemis.queue.operations:offlineOperations}")
    private String operationsQueue;
    
    @Autowired
    public OfflineMessageService(JmsTemplate jmsTemplate) {
        this.jmsTemplate = jmsTemplate;
        this.objectMapper = new ObjectMapper();
    }
    
    public void sendOperation(OfflineOperation operation) {
        try {
            String json = objectMapper.writeValueAsString(operation);
            jmsTemplate.convertAndSend(operationsQueue, json);
        } catch (Exception e) {
            throw new RuntimeException("Failed to send operation to queue", e);
        }
    }
    
    public OfflineOperation receiveOperation() {
        try {
            String json = (String) jmsTemplate.receiveAndConvert(operationsQueue);
            if (json == null) {
                return null;
            }
            return objectMapper.readValue(json, OfflineOperation.class);
        } catch (Exception e) {
            throw new RuntimeException("Failed to receive operation from queue", e);
        }
    }
    
    @Data
    @AllArgsConstructor
    @NoArgsConstructor
    public static class OfflineOperation {
        private String type;      // CREATE, UPDATE, DELETE
        private String endpoint;  // API endpoint
        private String id;        // resource id
        private String payload;   // JSON payload
        private long timestamp;
    }
}
  1. 创建离线操作处理服务
java 复制代码
@Service
public class OrderService {
    
    private final OfflineMessageService messageService;
    private final RestTemplate restTemplate;
    private final ObjectMapper objectMapper = new ObjectMapper();
    
    @Value("${api.base-url}")
    private String apiBaseUrl;
    
    @Autowired
    public OrderService(OfflineMessageService messageService, RestTemplate restTemplate) {
        this.messageService = messageService;
        this.restTemplate = restTemplate;
    }
    
    // 创建订单 - 直接进入离线队列
    public void createOrder(Order order) {
        try {
            // 生成ID
            if (order.getId() == null) {
                order.setId(UUID.randomUUID().toString());
            }
            
            order.setCreatedAt(System.currentTimeMillis());
            order.setStatus("PENDING");
            
            String payload = objectMapper.writeValueAsString(order);
            
            OfflineMessageService.OfflineOperation operation = new OfflineMessageService.OfflineOperation(
                    "CREATE",
                    "orders",
                    order.getId(),
                    payload,
                    System.currentTimeMillis()
            );
            
            messageService.sendOperation(operation);
        } catch (Exception e) {
            throw new RuntimeException("Failed to create order", e);
        }
    }
    
    // 更新订单状态 - 直接进入离线队列
    public void updateOrderStatus(String orderId, String status) {
        try {
            Map<String, Object> update = new HashMap<>();
            update.put("status", status);
            update.put("updatedAt", System.currentTimeMillis());
            
            String payload = objectMapper.writeValueAsString(update);
            
            OfflineMessageService.OfflineOperation operation = new OfflineMessageService.OfflineOperation(
                    "UPDATE",
                    "orders",
                    orderId,
                    payload,
                    System.currentTimeMillis()
            );
            
            messageService.sendOperation(operation);
        } catch (Exception e) {
            throw new RuntimeException("Failed to update order status", e);
        }
    }
    
    // 处理离线队列中的操作 - 由定时任务触发
    @Scheduled(fixedDelay = 60000) // 每分钟执行一次
    public void processOfflineOperations() {
        if (!isNetworkAvailable()) {
            return; // 网络不可用,跳过处理
        }
        
        int processedCount = 0;
        while (processedCount < 50) { // 一次处理50条,防止阻塞太久
            OfflineMessageService.OfflineOperation operation = messageService.receiveOperation();
            if (operation == null) {
                break; // 队列为空
            }
            
            boolean success = processOperation(operation);
            if (!success) {
                // 处理失败,重新入队(可以考虑添加重试次数限制)
                messageService.sendOperation(operation);
                break; // 暂停处理,等待下一次调度
            }
            
            processedCount++;
        }
    }
    
    private boolean processOperation(OfflineMessageService.OfflineOperation operation) {
        try {
            String url = apiBaseUrl + "/" + operation.getEndpoint();
            if (operation.getId() != null && !operation.getType().equals("CREATE")) {
                url += "/" + operation.getId();
            }
            
            HttpMethod method;
            switch (operation.getType()) {
                case "CREATE":
                    method = HttpMethod.POST;
                    break;
                case "UPDATE":
                    method = HttpMethod.PUT;
                    break;
                case "DELETE":
                    method = HttpMethod.DELETE;
                    break;
                default:
                    return false;
            }
            
            HttpHeaders headers = new HttpHeaders();
            headers.setContentType(MediaType.APPLICATION_JSON);
            
            HttpEntity<String> request = operation.getType().equals("DELETE") ? 
                    new HttpEntity<>(headers) : 
                    new HttpEntity<>(operation.getPayload(), headers);
            
            ResponseEntity<String> response = restTemplate.exchange(url, method, request, String.class);
            
            return response.getStatusCode().isSuccessful();
        } catch (Exception e) {
            log.error(e.getMessage(),e);
            return false;
        }
    }
    
    private boolean isNetworkAvailable() {
        try {
            URL url = new URL(apiBaseUrl);
            HttpURLConnection connection = (HttpURLConnection) url.openConnection();
            connection.setConnectTimeout(3000);
            connection.connect();
            return connection.getResponseCode() == 200;
        } catch (Exception e) {
            return false;
        }
    }
}
  1. 创建订单模型
vbnet 复制代码
@Data
public class Order {
    private String id;
    private String customerName;
    private List<OrderItem> items;
    private BigDecimal totalAmount;
    private String status;
    private long createdAt;
    private Long updatedAt;
}

@Data
public class OrderItem {
    private String productId;
    private String productName;
    private int quantity;
    private BigDecimal price;
}
  1. 创建Controller
less 复制代码
@RestController
@RequestMapping("/api/orders")
public class OrderController {
    
    private final OrderService orderService;
    
    @Autowired
    public OrderController(OrderService orderService) {
        this.orderService = orderService;
    }
    
    @PostMapping
    public ResponseEntity<String> createOrder(@RequestBody Order order) {
        orderService.createOrder(order);
        return ResponseEntity.ok("Order submitted for processing");
    }
    
    @PutMapping("/{id}/status")
    public ResponseEntity<String> updateOrderStatus(
            @PathVariable String id, 
            @RequestParam String status) {
        orderService.updateOrderStatus(id, status);
        return ResponseEntity.ok("Status update submitted for processing");
    }
    
    @PostMapping("/process")
    public ResponseEntity<String> triggerProcessing() {
        orderService.processOfflineOperations();
        return ResponseEntity.ok("Processing triggered");
    }
}
  1. 配置文件
ini 复制代码
# API配置
api.base-url=https://api.example.com

# Artemis配置
artemis.embedded.data-directory=./artemis-data
artemis.embedded.queues=offlineOperations
artemis.queue.operations=offlineOperations

优缺点分析

优点:

  • 强大的消息持久化能力,确保操作不丢失
  • 异步处理模式,非阻塞用户操作
  • 支持大批量数据处理
  • 内置的消息重试和死信机制

缺点:

  • 资源消耗较大,尤其是内存和磁盘
  • 配置相对复杂
  • 需要处理消息幂等性问题
  • 不适合需要即时反馈的场景

适用场景

  • 批量数据处理场景,如订单处理系统
  • 需要可靠消息处理的工作流应用
  • 高并发写入场景
  • 对操作顺序有严格要求的业务场景

六、方案对比与选择建议

方案对比

方案 复杂度 数据容量 冲突处理 适用场景 开发维护成本
嵌入式数据库 较复杂 单机应用、结构化数据
本地缓存 简单 读多写少、数据量小
离线优先架构 完善 企业应用、现场系统
嵌入式消息队列 中等 批量处理、异步操作

总结

在实际应用中,可以根据项目特点选择合适的方案,也可以结合多种方案的优点,定制最适合自己需求的离线解决方案。

无论选择哪种方案,完善的数据同步策略和良好的用户体验都是成功实现离线应用的关键因素。

相关推荐
sss191s9 分钟前
Java 集合面试题从数据结构到 HashMap 源码剖析详解及常见考点梳理
java·开发语言·数据结构
LI JS@你猜啊17 分钟前
window安装docker
java·spring cloud·eureka
书中自有妍如玉27 分钟前
.net 使用MQTT订阅消息
java·前端·.net
风铃儿~1 小时前
Spring AI 入门:Java 开发者的生成式 AI 实践之路
java·人工智能·spring
斯普信专业组1 小时前
Tomcat全方位监控实施方案指南
java·tomcat
忆雾屿1 小时前
云原生时代 Kafka 深度实践:06原理剖析与源码解读
java·后端·云原生·kafka
武昌库里写JAVA1 小时前
iview Switch Tabs TabPane 使用提示Maximum call stack size exceeded堆栈溢出
java·开发语言·spring boot·学习·课程设计
gaoliheng0061 小时前
Redis看门狗机制
java·数据库·redis
我是唐青枫2 小时前
.NET AOT 详解
java·服务器·.net
小白杨树树2 小时前
【WebSocket】SpringBoot项目中使用WebSocket
spring boot·websocket·网络协议