1. 需求
- 实现文件上传进度条展示
- 实现耗时异步任务完成消息通知
- 其他消息通知
2. 方案
文件上传进度消息：
- 后台使用commons-fileupload提供的功能，替代Spirng的文件解析器，注册自定义监听器，通过文件上传监听获取当前Spring框架已经读取的文件进度
- 服务模块通过Feign接口向消息模块发送文件上传进度消息
- 消息模块收到文件上传进度消息，并通过WebSocket发送给文件上传的用户
- 客户端收到进度，渲染上传进度条
异步耗时任务完成消息：
- 创建自定义注解@SendMessage
- 在需要发送消息的方法上注解@SendMessage
- 创建消息通知切面类MessageAspect，对@SendMessage进行环绕切面
- 在方法前后通过Feign接口向消息模块发送任务开始、结束消息
- 消息模块收到开始、结束消息，通过WebSocket向浏览器发送消息
3. 方案对比
常见方案：
AJAX异步轮询
优点：简单好用
缺点：轮询任务很多时效率较低，无法实现服务端通知WebSocket集群
WebSocket属于全双工通讯，与服务端建立会话后无法实现多个服务器间的会话共享，需要应用其他方案处理WebSocket集群问题。水平受限，暂未寻找到合适的集群方案，在此不做讨论。
优点：支持大量用户同时维持WebSocket通讯，服务可拓展集群实现高并发高可用单WebSocket消息模块部署
这个是本案例中采用的方案，仅部署一个消息服务，该消息服务维护着所有与浏览器建立的WebSocket连接，其他模块可以多服务部署，通过Feign接口向消息服务发送消息，消息服务将消息转发给指定用户，消息服务充当中间人角色。
优点：部署方便，可以实现服务端通知
缺点：单服务处理能力受限，不支持大量用户，不适用于在线用户多的互联网应用
4. 文件上传进度消息实现
4.1 引入依赖
<dependency>
<groupId>commons-io</groupId>
<artifactId>commons-io</artifactId>
<version>2.4</version>
</dependency>
<dependency>
<groupId>commons-fileupload</groupId>
<artifactId>commons-fileupload</artifactId>
<version>1.3.1</version>
</dependency>
- 编写自定义文件上传监听器
- update方法为框架自行调用，因此避免性能问题应限制发送消息的次数
- update方法参数中pBytesRead pContentLength 均是当前Item，一次上传多个文件时注意需要计算整个文件数量的百分比，但该百分比并不能反映真实进度，因为文件的大小不一致，仅能反映模拟的一个上传进度。
- MessageDto为自定义消息实体，这个可以根据实际发送消息的格式进行自定义
package com.tba.sc.common.listener;
import com.tba.sc.common.dto.message.MessageDto;
import com.tba.sc.common.enums.EnumMessageType;
import com.tba.sc.common.feign.message.FeignMessageService;
import lombok.Data;
import lombok.extern.slf4j.Slf4j;
import org.apache.commons.fileupload.ProgressListener;
/**
* @author wangqichang
* @since 2020/4/9
*/
@Data
@Slf4j
public class RedisFileUploadProgressListener implements ProgressListener {
/**
* 上传UUID
*/
private String uploadUUID;
private String taskName;
private int itemNum = 1;
private FeignMessageService messageService;
/**
* 已读字节数
*/
private long megaBytes = -1;
public RedisFileUploadProgressListener(String uploadUUID, String taskName, Integer itemNum, FeignMessageService messageService) {
this.uploadUUID = uploadUUID;
this.taskName = taskName;
this.itemNum = itemNum;
this.messageService = messageService;
}
@Override
public void update(long pBytesRead, long pContentLength, int pItems) {
//避免性能问题，每读取1M更新状态
long mBytes = pBytesRead / 1000000;
if (megaBytes == mBytes) {
return;
}
megaBytes = mBytes;
Double doubleLength = new Double(pContentLength);
if (pContentLength > 0 && pItems > 0) {
Double ps = pBytesRead / doubleLength * 100 * pItems / itemNum;
log.info("文件上传监听：上传UUID：{} 当前ITEM：{} 百分比：{}", uploadUUID, pItems, ps);
try {
messageService.send(MessageDto.builder().type(EnumMessageType.FILE_UPLOAD_PROCESS.getType()).msgId(uploadUUID).percentage(ps).message(taskName).build());
} catch (Exception e) {
log.error("调用Message模块失败，未能发送上传百分比消息");
}
}
}
}
4. 2编写自定义文件上传解析器，封装参数，注册监听
- 该解析器执行时，springmvc尚未封装参数，因此如果监听器必要参数需要获取时，本例是由前端拼接URL参数，此处从URL中获取必要参数
- cleanupMultipart方法在整个上传方法结束后调用做清理工作，上传文件后进行业务逻辑处理完毕后才会调用，并不是Controller获取到文件后清理。
package com.tba.sc.common.config;
import com.tba.sc.common.feign.message.FeignMessageService;
import com.tba.sc.common.listener.RedisFileUploadProgressListener;
import org.apache.commons.fileupload.FileItem;
import org.apache.commons.fileupload.servlet.ServletFileUpload;
import org.springframework.web.multipart.MultipartHttpServletRequest;
import org.springframework.web.multipart.commons.CommonsMultipartResolver;
import javax.servlet.http.HttpServletRequest;
/**
* @author wangqichang
* @since 2020/4/10
*/
@Slf4j
public class MyCommonsMultipartResolver extends CommonsMultipartResolver {
RedisTemplate redisTemplate;
FeignMessageService feignMessageService;
public MyCommonsMultipartResolver(RedisTemplate redisTemplate, FeignMessageService feignMessageService) {
this.redisTemplate = redisTemplate;
this.feignMessageService = feignMessageService;
}
/**
* 注册上传监听
*
* @param request
* @return
* @throws MultipartException
*/
@Override
protected MultipartParsingResult parseRequest(HttpServletRequest request) throws MultipartException {
//向request设置上传文件ID
String uuid = IdUtil.fastUUID();
request.setAttribute(SystemConstants.MSG_ID_PARAM, uuid);
String encoding = determineEncoding(request);
FileUpload fileUpload = prepareFileUpload(encoding);
String queryString = request.getQueryString();
try {
RedisFileUploadProgressListener redisFileUploadProgressListener = null;
if (StrUtil.isNotBlank(queryString)) {
String[] split = queryString.split("&");
if (ArrayUtil.isNotEmpty(split) && split.length > 1) {
String[] param = split[0].split("=");
String[] itemParam = split[1].split("=");
//设置监听
if (ArrayUtil.isNotEmpty(param) && param.length > 1 && SystemConstants.UPLOAD_TASK_NAME.equals(param[0])) {
String taskName = URLDecoder.decode(param[1], "UTF-8");
request.setAttribute(SystemConstants.UPLOAD_TASK_NAME, taskName);
Integer item = 1;
if (SystemConstants.UPLOAD_ITEM_NUM.equals(itemParam[0])) {
item = Integer.valueOf(itemParam[1]);
}
redisFileUploadProgressListener = new RedisFileUploadProgressListener(uuid, taskName, item, feignMessageService);
fileUpload.setProgressListener(redisFileUploadProgressListener);
}
}
}
List<FileItem> fileItems = ((ServletFileUpload) fileUpload).parseRequest(request);
return parseFileItems(fileItems, encoding);
} catch (FileUploadBase.SizeLimitExceededException ex) {
throw new MaxUploadSizeExceededException(fileUpload.getSizeMax(), ex);
} catch (FileUploadBase.FileSizeLimitExceededException ex) {
throw new MaxUploadSizeExceededException(fileUpload.getFileSizeMax(), ex);
} catch (FileUploadException ex) {
throw new MultipartException("Failed to parse multipart servlet request", ex);
} catch (Exception e) {
e.printStackTrace();
throw new RuntimeException(e.getMessage());
}
}
/**
* 上传文件结束
* @param request
*/
@Override
public void cleanupMultipart(MultipartHttpServletRequest request) {
super.cleanupMultipart(request);
String uploadId = (String) request.getAttribute(SystemConstants.MSG_ID_PARAM);
String taskName = (String) request.getAttribute(SystemConstants.UPLOAD_TASK_NAME);
if (StrUtil.isNotBlank(taskName)) {
feignMessageService.send(MessageDto.builder().type(EnumMessageType.FILE_UPLOAD_PROCESS.getType()).msgId(uploadId).message(taskName + "任务文件上传完成").percentage(100D).finalNotice(Boolean.TRUE).build());
}
}
}
4. 3 向spring容器中注入解析器
根据解析器构造，传入必要参数。该解析器将替代默认实现
@Bean
MyCommonsMultipartResolver commonsMultipartResolver(RedisTemplate redisTemplate, FeignMessageService feignMessageService) {
return new MyCommonsMultipartResolver(redisTemplate,feignMessageService);
}
5 搭建消息服务模块
5.1 核心依赖
spring为WebSocket提供了很好的支持，参照官方文档即可完成服务搭建
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-websocket</artifactId>
</dependency>
5.2创建WebSocket配置类
继承WebSocketMessageBrokerConfigurer类，重写registerStompEndpoints() configureMessageBroker() configureClientInboundChannel()方法。
- registerStompEndpoints方法为注册Stomp端点，暴露用于建立WebSocket的端点接口。其中DefaultHandshakeHandler为端口握手处理，重写determineUser方法，name为当前WebSocket的唯一标识，本例中为用户名（注意，需保证同一时间一个用户只能在一个客户端建立WebSocket连接）
- configureMessageBroker为配置消息代理，设置前缀及配置消息订阅主题
- configureClientInboundChannel配置websocket权限，本例中使用stomp携带token标头，实际上仅在建立连接时做判断也是可以的
package com.tba.message.config;
import org.springframework.messaging.Message;
import org.springframework.messaging.simp.config.MessageBrokerRegistry;
import org.springframework.messaging.simp.stomp.StompCommand;
import org.springframework.messaging.support.ChannelInterceptor;
import org.springframework.web.socket.WebSocketHandler;
import org.springframework.web.socket.config.annotation.EnableWebSocketMessageBroker;
import org.springframework.web.socket.config.annotation.StompEndpointRegistry;
import org.springframework.web.socket.config.annotation.WebSocketMessageBrokerConfigurer;
import org.springframework.web.socket.server.support.DefaultHandshakeHandler;
import java.security.Principal;
/**
* STOMP over WebSocket support is available in the spring-messaging and spring-websocket modules. Once you have those dependencies, you can expose a STOMP endpoints, over WebSocket with SockJS Fallback, as the following example shows:
*
* @author wangqichang
* @since 2020/3/13
*/
@Slf4j
@Configuration
@EnableWebSocketMessageBroker
public class WebSocketConfig implements WebSocketMessageBrokerConfigurer {
@Autowired
RedisTemplate redisTemplate;
@Override
public void registerStompEndpoints(StompEndpointRegistry registry) {
/**
* is the HTTP URL for the endpoint to which a WebSocket (or SockJS) client needs to connect for the WebSocket handshake.
*/
registry
.addEndpoint("/ws")
.setHandshakeHandler(new DefaultHandshakeHandler() {
@Override
protected Principal determineUser(ServerHttpRequest request, WebSocketHandler wsHandler, Map<String, Object> attributes) {
return new UserPrincipal() {
@Override
public String getName() {
//使用了spring security框架时，框架将自动封装Principal
// Principal principal = request.getPrincipal();
//根据自行权限框架，根据token自行封装Principal
List<String> authToken = request.getHeaders().get(SystemConstants.TOKEN_HEADER);
if (CollUtil.isNotEmpty(authToken)) {
String token = authToken.get(0);
String redisTokenKey = RedisKeyConstants.TOKEN_PREFIX + token;
CurrentUser user = (CurrentUser) redisTemplate.opsForValue().get(redisTokenKey);
if (ObjectUtil.isNotNull(user)) {
return user.getUsername();
}
}
throw new ServiceException("无法注册当前连接的用户，请检查是否携带用户凭证");
}
};
}
})
.setAllowedOrigins("*")
.withSockJS();
}
@Override
public void configureMessageBroker(MessageBrokerRegistry config) {
/**
* STOMP messages whose destination header begins with /app are routed to @MessageMapping methods in @Controller classes.
* Use the built-in message broker for subscriptions and broadcasting and route messages whose destination header begins with /topic `or `/queue to the broker.
*/
config.setApplicationDestinationPrefixes("/app");
//topic 广播主题消息 queue 一对一消息
config.enableSimpleBroker("/topic", "/queue");
}
/**
* 从stomp中获取token标头
*
* @param registration
*/
@Override
public void configureClientInboundChannel(ChannelRegistration registration) {
registration.interceptors(new ChannelInterceptor() {
@Override
public Message<?> preSend(Message<?> message, MessageChannel channel) {
StompHeaderAccessor accessor =
MessageHeaderAccessor.getAccessor(message, StompHeaderAccessor.class);
if (StompCommand.CONNECT.equals(accessor.getCommand())) {
List<String> nativeHeader = accessor.getNativeHeader(SystemConstants.TOKEN_HEADER);
String token = nativeHeader.get(0);
Assert.notNull(token, "未携带用户凭证的请求");
//根据token从redis中获取当前用户
CurrentUser user = (CurrentUser) redisTemplate.opsForValue().get(RedisKeyConstants.TOKEN_PREFIX + token);
if (ObjectUtil.isNotNull(user)) {
String username = user.getUsername();
accessor.setUser(new com.tba.message.security.UserPrincipal(username));
return message;
}
throw new ServiceException("用户凭证已过期");
}
return message;
}
});
}
}
5.3 编写Controller，暴露发送消息的Restful接口
- 此接口暴露给其他服务调用，通过Message服务，向客户端发送消息。Message服务相当于中间代理，因为客户端仅与Message服务维持WebSocket连接
- 这个方法从线程变量中取出当前用户username（线程变量中用户信息为拦截器拦截token，查询用户并设置），向该用户发送消息，历史未结束消息放在redis缓存中，每次从redis中查询该用户历史数据，通过msgId更新消息或者新增消息。最后一次提示消息发送成功则从list删除，不进行历史未结束消息的缓存。
package com.tba.message.controller;
import com.tba.sc.common.dto.message.MessageDto;
import com.tba.sc.common.user.CurrentUser;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.messaging.simp.SimpMessagingTemplate;
/**
* @author wangqichang
* @since 2020/4/10
*/
@Slf4j
@Controller
@RequestMapping("/msg")
@RestController
public class MsgController {
@Autowired
RedisTemplate redisTemplate;
@Autowired
private SimpMessagingTemplate simpMessagingTemplate;
@PostMapping(value = "/send")
public InvokeResult send(@RequestBody MessageDto message) {
//根据当前http请求中获取用户信息
CurrentUser current = UserContext.current();
Assert.notNull(current);
//从redis中获取当前用户的消息列表
List<MessageDto> list = (List<MessageDto>) redisTemplate.opsForValue().get(RedisKeyConstants.TOKEN_MSG + current.getToken());
if (ObjectUtil.isNull(list)) {
list = new ArrayList<>();
}
if (CollUtil.isNotEmpty(list) && StrUtil.isNotBlank(message.getMsgId())) {
for (int i = 0; i < list.size(); i++) {
//更新消息
if (message.getMsgId().equals(list.get(i).getMsgId())) {
list.set(i, message);
message.setCreateDate(list.get(i).getCreateDate());
}
}
} else {
//新增消息
list.add(message);
}
try {
this.simpMessagingTemplate.convertAndSendToUser(current.getUsername(), "/queue", list);
log.info("用户：{} 消息数量：{} 发送新消息：{}", current.getRealname(), list.size(), message.toString());
//发送成功，删除消息
if (message.isFinalNotice()) {
list.remove(message);
}
return InvokeResult.success();
} catch (Exception e) {
e.printStackTrace();
log.error(e.getMessage());
return InvokeResult.failure("消息发送失败");
} finally {
//发送失败，进缓存
redisTemplate.opsForValue().set(RedisKeyConstants.TOKEN_MSG + current.getToken(), list, 7, TimeUnit.DAYS);
}
}
}
5.4 暴露消息Feign接口
@FeignClient(name = "消息服务实例名称", path = "/msg")
public interface FeignMessageService {
@PostMapping(value = "/send")
InvokeResult send(@RequestBody MessageDto message);
}
6 耗时任务消息发送
此处通过注解切面，在需要执行的方法前后想Message服务发送消息
6.1 自定义@SendMessage注解
@Target({ElementType.METHOD})
@Retention(RetentionPolicy.RUNTIME)
@Documented
public @interface SendMessage {
}
6.2 定义MessageAspect切面类
该切面将以@SendMessage注解为切入点，利用反射获取形参名及参数值，封装MessageDto，调用Feign接口向消息模块发送消息
package com.tba.sc.common.advice;
import com.tba.sc.common.dto.message.MessageDto;
import com.tba.sc.common.enums.EnumMessageType;
import com.tba.sc.common.feign.message.FeignMessageService;
import org.aspectj.lang.ProceedingJoinPoint;
import org.aspectj.lang.annotation.Around;
import org.aspectj.lang.annotation.Aspect;
import org.aspectj.lang.annotation.Pointcut;
import org.aspectj.lang.reflect.MethodSignature;
import java.lang.reflect.Method;
import java.lang.reflect.Parameter;
/**
* @author wangqichang
* @since 2020/4/15
*/
@Slf4j
@Aspect
@Component
public class MessageAspect {
@Autowired
FeignMessageService feignMessageService;
@Around("@annotation(com.tba.sc.common.annotation.SendMessage)")
public Object BeforeMethod(ProceedingJoinPoint jp) throws Throwable {
MethodSignature methodSignature = (MethodSignature) jp.getSignature();
Method method = methodSignature.getMethod();
Object[] args = jp.getArgs();
//注意：该方法需指定编译插件-parameters参数，否则无法获取到形参名称。配置在pom中maven-compiler-plugin
Parameter[] parameters = method.getParameters();
String taskName = null;
String taskId = null;
String url = null;
methodSignature.getParameterNames();
for (Parameter parameter : parameters) {
Integer index = (Integer) ReflectUtil.getFieldValue(parameter, "index");
if ("taskName".equals(parameter.getName()) && ArrayUtil.isNotEmpty(args)) {
taskName = (String) args[index];
} else if ("id".equals(parameter.getName())) {
taskId = (String) args[index];
} else if ("url".equals(parameter.getName())) {
url = (String) args[index];
}
}
log.info("taskName:{} id:{} url:{}", taskName, taskId, url);
if (StrUtil.isNotBlank(taskName)) {
String msgId = IdUtil.fastUUID();
MessageDto msg = MessageDto.builder()
.msgId(msgId)
.type(EnumMessageType.BUSINESS_NOTICE.getType())
.finalNotice(Boolean.FALSE)
.createDate(new Date())
.message(taskName + "任务开始")
.taskId(taskId)
.url(url)
.build();
try {
log.info("发送消息：{}", msg.toString());
feignMessageService.send(msg);
Object proceed = jp.proceed();
msg.setFinalNotice(Boolean.TRUE);
msg.setMessage(taskName + "任务完成");
msg.setSuccess(Boolean.TRUE);
log.info("发送消息：{}", msg.toString());
feignMessageService.send(msg);
return proceed;
} catch (Throwable throwable) {
msg.setFinalNotice(Boolean.TRUE);
msg.setMessage(taskName + "任务异常结束");
msg.setSuccess(Boolean.FALSE);
log.info("发送消息：{}", msg.toString());
feignMessageService.send(msg);
throw throwable;
}
}
log.info("未能获取到任务名称参数，未发送消息");
return jp.proceed();
}
}
6.2在所需接口上注解@SendMessage，并声明形参
- 此处部分参数并未传递给Service，目的是为了切面类可以拿到形参及实参封装消息实体
@PostMapping("/xxx")
@SendMessage
public InvokeResult xxx(String id, String taskName,String url) {
xxxService.xxx(id);
return InvokeResult.success();
}
7. 效果展示
文件上传监听日志，成功监听上传进度
2020-05-18 18:01:28.706 INFO 16744 --- [io-6001-exec-30] .t.s.c.l.RedisFileUploadProgressListener : 文件上传监听：上传UUID：fdbdf76f-3421-436a-8614-837aa8fe7972 当前ITEM：1 百分比：93.90534762273536
2020-05-18 18:01:28.743 INFO 16744 --- [io-6001-exec-30] .t.s.c.l.RedisFileUploadProgressListener : 文件上传监听：上传UUID：fdbdf76f-3421-436a-8614-837aa8fe7972 当前ITEM：1 百分比：96.51222534735146
2020-05-18 18:01:28.787 INFO 16744 --- [io-6001-exec-30] .t.s.c.l.RedisFileUploadProgressListener : 文件上传监听：上传UUID：fdbdf76f-3421-436a-8614-837aa8fe7972 当前ITEM：1 百分比：99.11910307196756
文件上传进度消息发送日志
2020-05-15 14:37:23.033 INFO 2924 --- [nio-9015-exec-9] c.tba.message.controller.MsgController : 用户：超管 消息数量：1 发送新消息：MessageDto{msgId='3cce247c-5e67-46e1-9d18-3e4bc25cc1e4', type=2, taskId='null', message='11', percentage=99.57479978077085, finalNotice=false, success=false, createDate=null, url='null'}
2020-05-15 14:37:24.125 INFO 2924 --- [io-9015-exec-13] c.tba.message.controller.MsgController : 用户：超管 消息数量：1 发送新消息：MessageDto{msgId='3cce247c-5e67-46e1-9d18-3e4bc25cc1e4', type=2, taskId='null', message='11', percentage=99.79995204151234, finalNotice=false, success=false, createDate=null, url='null'}
耗时任务消息模块发送日志
2020-05-15 10:50:40.501 INFO 2924 --- [MessageBroker-5] o.s.w.s.c.WebSocketMessageBrokerStats : WebSocketSession[3 current WS(2)-HttpStream(0)-HttpPoll(1), 13 total, 0 closed abnormally (0 connect failure, 0 send limit, 0 transport error)], stompSubProtocol[processed CONNECT(11)-CONNECTED(10)-DISCONNECT(0)], stompBrokerRelay[null], inboundChannel[pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 120], outboundChannel[pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 43], sockJsScheduler[pool size = 8, active threads = 1, queued tasks = 4, completed tasks = 651]
2020-05-15 10:50:57.728 INFO 2924 --- [io-9015-exec-10] c.tba.message.controller.MsgController : 用户：超管 消息数量：1 发送新消息：MessageDto{msgId='debbafbf-63a3-432e-8107-15cf03becebe', type=3, taskId='8afad4bd7202f283017212458b9d0111', message='测试002N3-N4任务开始', percentage=null, finalNotice=false, success=false, createDate=Fri May 15 18:50:57 CST 2020, url='/operation?type=2&id=8afad4bd7202f28301721245d3cf0112&taskName=%E6%B5%8B%E8%AF%95002&taskType=1'}
2020-05-15 10:51:06.304 INFO 2924 --- [io-9015-exec-11] c.tba.message.controller.MsgController : 用户：超管 消息数量：1 发送新消息：MessageDto{msgId='debbafbf-63a3-432e-8107-15cf03becebe', type=3, taskId='8afad4bd7202f283017212458b9d0111', message='测试002N3-N4任务完成', percentage=null, finalNotice=true, success=true, createDate=Fri May 15 18:50:57 CST 2020, url='/operation?type=2&id=8afad4bd7202f28301721245d3cf0112&taskName=%E6%B5%8B%E8%AF%95002&taskType=1'}
前端消息渲染效果
image.png
大功告成！
尚有诸多缺点，但保证了基础功能够用，诸位大佬可以做个小参考。
