2. 协议设计与解析

2.1 为什么需要协议？

TCP/IP 中消息传输基于流的方式，没有边界。

协议的目的就是划定消息的边界，制定通信双方要共同遵守的通信规则

例如：在网络上传输

下雨天留客天留我不留

是中文一句著名的无标点符号句子，在没有标点符号情况下，这句话有数种拆解方式，而意思却是完全不同，所以常被用作讲述标点符号的重要性

一种解读

下雨天留客，天留，我不留

另一种解读

下雨天，留客天，留我不？留

如何设计协议呢？其实就是给网络传输的信息加上“标点符号”。但通过分隔符来断句不是很好，因为分隔符本身如果用于传输，那么必须加以区分。因此，下面一种协议较为常用

定长字节表示内容长度 + 实际内容

例如，假设一个中文字符长度为 3，按照上述协议的规则，发送信息方式如下，就不会被接收方弄错意思了

0f下雨天留客06天留09我不留

小故事

很久很久以前，一位私塾先生到一家任教。双方签订了一纸协议：“无鸡鸭亦可无鱼肉亦可白菜豆腐不可少不得束修金”。此后，私塾先生虽然认真教课，但主人家则总是给私塾先生以白菜豆腐为菜，丝毫未见鸡鸭鱼肉的款待。私塾先生先是很不解，可是后来也就想通了：主人把鸡鸭鱼肉的钱都会换为束修金的，也罢。至此双方相安无事。

年关将至，一个学年段亦告结束。私塾先生临行时，也不见主人家为他交付束修金，遂与主家理论。然主家亦振振有词：“有协议为证——无鸡鸭亦可，无鱼肉亦可，白菜豆腐不可少，不得束修金。这白纸黑字明摆着的，你有什么要说的呢？”

私塾先生据理力争：“协议是这样的——无鸡，鸭亦可；无鱼，肉亦可；白菜豆腐不可，少不得束修金。”

双方唇枪舌战，你来我往，真个是不亦乐乎！

这里的束修金，也作“束脩”，应当是泛指教师应当得到的报酬

2.2 redis 协议举例

@Slf4j
public class TestRedis {/* redis协议格式set name zhangsan*3$3set$4name$8zhangsan*/public static void main(String[] args) {final byte[] LINE = {13, 10}; //回车换行 \r\nNioEventLoopGroup worker = new NioEventLoopGroup();try {Bootstrap bootstrap = new Bootstrap();bootstrap.channel(NioSocketChannel.class);bootstrap.group(worker);bootstrap.handler(new ChannelInitializer<SocketChannel>() {@Overrideprotected void initChannel(SocketChannel ch) {ch.pipeline().addLast(new LoggingHandler());ch.pipeline().addLast(new ChannelInboundHandlerAdapter() {@Overridepublic void channelActive(ChannelHandlerContext ctx) {ByteBuf buf = ctx.alloc().buffer();buf.writeBytes("*3".getBytes());buf.writeBytes(LINE);buf.writeBytes("$3".getBytes());buf.writeBytes(LINE);buf.writeBytes("set".getBytes());buf.writeBytes(LINE);buf.writeBytes("$4".getBytes());buf.writeBytes(LINE);buf.writeBytes("name".getBytes());buf.writeBytes(LINE);buf.writeBytes("$8".getBytes());buf.writeBytes(LINE);buf.writeBytes("zhangsan".getBytes());buf.writeBytes(LINE);ctx.writeAndFlush(buf);}@Overridepublic void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {ByteBuf buf = (ByteBuf) msg;System.out.println(buf.toString(Charset.defaultCharset()));}});}});ChannelFuture channelFuture = bootstrap.connect("localhost", 6379).sync();channelFuture.channel().closeFuture().sync();} catch (InterruptedException e) {log.error("client error", e);} finally {worker.shutdownGracefully();}}
}

2.3 http 协议举例

@Slf4j
public class TestHttp {public static void main(String[] args) {NioEventLoopGroup boss = new NioEventLoopGroup();NioEventLoopGroup worker = new NioEventLoopGroup();try {ServerBootstrap serverBootstrap = new ServerBootstrap();serverBootstrap.channel(NioServerSocketChannel.class);serverBootstrap.group(boss, worker);serverBootstrap.childHandler(new ChannelInitializer<SocketChannel>() {@Overrideprotected void initChannel(SocketChannel ch) throws Exception {ch.pipeline().addLast(new LoggingHandler(LogLevel.DEBUG));//http编码器，解码器ch.pipeline().addLast(new HttpServerCodec());ch.pipeline().addLast(new SimpleChannelInboundHandler<HttpRequest>() {@Overrideprotected void channelRead0(ChannelHandlerContext ctx, HttpRequest msg) throws Exception {// 获取请求log.debug(msg.uri());//返回响应DefaultFullHttpResponse response = new DefaultFullHttpResponse(msg.protocolVersion(), HttpResponseStatus.OK);byte[] bytes = "<h1>Hello, world!</h1>".getBytes();//不加这个，浏览器一直转圈response.headers().set(CONTENT_LENGTH, bytes.length);response.content().writeBytes(bytes);ctx.channel().writeAndFlush(response);}});}});ChannelFuture channelFuture = serverBootstrap.bind(8080).sync();channelFuture.channel().closeFuture().sync();} catch (Exception e) {log.error("server error", e);} finally {boss.shutdownGracefully();worker.shutdownGracefully();}}
}

浏览器输入：http://127.0.0.1:8080/index，结果

17:26:54 [DEBUG] [nioEventLoopGroup-3-1] i.n.h.l.LoggingHandler - [id: 0x986cb0df, L:/127.0.0.1:8080 - R:/127.0.0.1:55551] REGISTERED
17:26:54 [DEBUG] [nioEventLoopGroup-3-2] i.n.h.l.LoggingHandler - [id: 0x0bc48b95, L:/127.0.0.1:8080 - R:/127.0.0.1:55552] REGISTERED
17:26:54 [DEBUG] [nioEventLoopGroup-3-2] i.n.h.l.LoggingHandler - [id: 0x0bc48b95, L:/127.0.0.1:8080 - R:/127.0.0.1:55552] ACTIVE
17:26:54 [DEBUG] [nioEventLoopGroup-3-1] i.n.h.l.LoggingHandler - [id: 0x986cb0df, L:/127.0.0.1:8080 - R:/127.0.0.1:55551] ACTIVE
17:26:54 [DEBUG] [nioEventLoopGroup-3-1] i.n.h.l.LoggingHandler - [id: 0x986cb0df, L:/127.0.0.1:8080 - R:/127.0.0.1:55551] READ: 662B+-------------------------------------------------+|  0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f |
+--------+-------------------------------------------------+----------------+
|00000000| 47 45 54 20 2f 69 6e 64 65 78 20 48 54 54 50 2f |GET /index HTTP/|
|00000010| 31 2e 31 0d 0a 48 6f 73 74 3a 20 31 32 37 2e 30 |1.1..Host: 127.0|
|00000020| 2e 30 2e 31 3a 38 30 38 30 0d 0a 43 6f 6e 6e 65 |.0.1:8080..Conne|
|00000030| 63 74 69 6f 6e 3a 20 6b 65 65 70 2d 61 6c 69 76 |ction: keep-aliv|
|00000040| 65 0d 0a 73 65 63 2d 63 68 2d 75 61 3a 20 22 47 |e..sec-ch-ua: "G|
|00000050| 6f 6f 67 6c 65 20 43 68 72 6f 6d 65 22 3b 76 3d |oogle Chrome";v=|
|00000060| 22 31 31 37 22 2c 20 22 4e 6f 74 3b 41 3d 42 72 |"117", "Not;A=Br|
|00000070| 61 6e 64 22 3b 76 3d 22 38 22 2c 20 22 43 68 72 |and";v="8", "Chr|
|00000080| 6f 6d 69 75 6d 22 3b 76 3d 22 31 31 37 22 0d 0a |omium";v="117"..|
|00000090| 73 65 63 2d 63 68 2d 75 61 2d 6d 6f 62 69 6c 65 |sec-ch-ua-mobile|
|000000a0| 3a 20 3f 30 0d 0a 73 65 63 2d 63 68 2d 75 61 2d |: ?0..sec-ch-ua-|
|000000b0| 70 6c 61 74 66 6f 72 6d 3a 20 22 57 69 6e 64 6f |platform: "Windo|
|000000c0| 77 73 22 0d 0a 55 70 67 72 61 64 65 2d 49 6e 73 |ws"..Upgrade-Ins|
|000000d0| 65 63 75 72 65 2d 52 65 71 75 65 73 74 73 3a 20 |ecure-Requests: |
|000000e0| 31 0d 0a 55 73 65 72 2d 41 67 65 6e 74 3a 20 4d |1..User-Agent: M|
|000000f0| 6f 7a 69 6c 6c 61 2f 35 2e 30 20 28 57 69 6e 64 |ozilla/5.0 (Wind|
|00000100| 6f 77 73 20 4e 54 20 31 30 2e 30 3b 20 57 69 6e |ows NT 10.0; Win|
|00000110| 36 34 3b 20 78 36 34 29 20 41 70 70 6c 65 57 65 |64; x64) AppleWe|
|00000120| 62 4b 69 74 2f 35 33 37 2e 33 36 20 28 4b 48 54 |bKit/537.36 (KHT|
|00000130| 4d 4c 2c 20 6c 69 6b 65 20 47 65 63 6b 6f 29 20 |ML, like Gecko) |
|00000140| 43 68 72 6f 6d 65 2f 31 31 37 2e 30 2e 30 2e 30 |Chrome/117.0.0.0|
|00000150| 20 53 61 66 61 72 69 2f 35 33 37 2e 33 36 0d 0a | Safari/537.36..|
|00000160| 41 63 63 65 70 74 3a 20 74 65 78 74 2f 68 74 6d |Accept: text/htm|
|00000170| 6c 2c 61 70 70 6c 69 63 61 74 69 6f 6e 2f 78 68 |l,application/xh|
|00000180| 74 6d 6c 2b 78 6d 6c 2c 61 70 70 6c 69 63 61 74 |tml+xml,applicat|
|00000190| 69 6f 6e 2f 78 6d 6c 3b 71 3d 30 2e 39 2c 69 6d |ion/xml;q=0.9,im|
|000001a0| 61 67 65 2f 61 76 69 66 2c 69 6d 61 67 65 2f 77 |age/avif,image/w|
|000001b0| 65 62 70 2c 69 6d 61 67 65 2f 61 70 6e 67 2c 2a |ebp,image/apng,*|
|000001c0| 2f 2a 3b 71 3d 30 2e 38 2c 61 70 70 6c 69 63 61 |/*;q=0.8,applica|
|000001d0| 74 69 6f 6e 2f 73 69 67 6e 65 64 2d 65 78 63 68 |tion/signed-exch|
|000001e0| 61 6e 67 65 3b 76 3d 62 33 3b 71 3d 30 2e 37 0d |ange;v=b3;q=0.7.|
|000001f0| 0a 53 65 63 2d 46 65 74 63 68 2d 53 69 74 65 3a |.Sec-Fetch-Site:|
|00000200| 20 6e 6f 6e 65 0d 0a 53 65 63 2d 46 65 74 63 68 | none..Sec-Fetch|
|00000210| 2d 4d 6f 64 65 3a 20 6e 61 76 69 67 61 74 65 0d |-Mode: navigate.|
|00000220| 0a 53 65 63 2d 46 65 74 63 68 2d 55 73 65 72 3a |.Sec-Fetch-User:|
|00000230| 20 3f 31 0d 0a 53 65 63 2d 46 65 74 63 68 2d 44 | ?1..Sec-Fetch-D|
|00000240| 65 73 74 3a 20 64 6f 63 75 6d 65 6e 74 0d 0a 41 |est: document..A|
|00000250| 63 63 65 70 74 2d 45 6e 63 6f 64 69 6e 67 3a 20 |ccept-Encoding: |
|00000260| 67 7a 69 70 2c 20 64 65 66 6c 61 74 65 2c 20 62 |gzip, deflate, b|
|00000270| 72 0d 0a 41 63 63 65 70 74 2d 4c 61 6e 67 75 61 |r..Accept-Langua|
|00000280| 67 65 3a 20 7a 68 2d 43 4e 2c 7a 68 3b 71 3d 30 |ge: zh-CN,zh;q=0|
|00000290| 2e 39 0d 0a 0d 0a                               |.9....          |
+--------+-------------------------------------------------+----------------+
17:26:54 [DEBUG] [nioEventLoopGroup-3-1] c.i.n.j.c.TestHttp - /index
17:26:54 [DEBUG] [nioEventLoopGroup-3-1] i.n.h.l.LoggingHandler - [id: 0x986cb0df, L:/127.0.0.1:8080 - R:/127.0.0.1:55551] WRITE: 61B+-------------------------------------------------+|  0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f |
+--------+-------------------------------------------------+----------------+
|00000000| 48 54 54 50 2f 31 2e 31 20 32 30 30 20 4f 4b 0d |HTTP/1.1 200 OK.|
|00000010| 0a 43 6f 6e 74 65 6e 74 2d 6c 65 6e 67 74 68 3a |.Content-length:|
|00000020| 20 32 32 0d 0a 0d 0a 3c 68 31 3e 48 65 6c 6c 6f | 22....<h1>Hello|
|00000030| 2c 20 77 6f 72 6c 64 21 3c 2f 68 31 3e          |, world!</h1>   |
+--------+-------------------------------------------------+----------------+

2.4 自定义协议要素

• 魔数，用来在第一时间判定是否是无效数据包
  • 魔数（Magic Number）通常是一个固定长度的字节序列，用于识别协议或数据结构的开头。魔数的存在可以帮助接收方快速验证数据的有效性，并确保它们理解并按照正确的协议进行处理。
• 版本号，可以支持协议的升级
• 序列化算法，消息正文到底采用哪种序列化反序列化方式，可以由此扩展，例如：json、protobuf、hessian、jdk
• 指令类型，是登录、注册、单聊、群聊… 跟业务相关
• 请求序号，为了双工通信，提供异步能力
• 正文长度
• 消息正文

编解码器

根据上面的要素，设计一个登录请求消息和登录响应消息，并使用 Netty 完成收发

@Slf4j
public class MessageCodec extends ByteToMessageCodec<Message> {// 编码 出站调用@Overrideprotected void encode(ChannelHandlerContext ctx, Message msg, ByteBuf out) throws Exception {// 4 字节的魔数out.writeBytes(new byte[]{1, 2, 3, 4});// 1 字节的版本out.writeByte(1);// 1 字节的序列化方式 jdk 0 , json 1out.writeByte(0);// 1 字节的指令类型out.writeByte(msg.getMessageType());// 4 个字节的请求序号out.writeInt(msg.getSequenceId());// 1 字节，无意义，对齐填充，为了将协议头补齐至2的n次幂out.writeByte(0xff);// 将message序列化为字节数组ByteArrayOutputStream bos = new ByteArrayOutputStream();ObjectOutputStream oos = new ObjectOutputStream(bos);oos.writeObject(msg);byte[] bytes = bos.toByteArray();// 4 个字节的正文长度out.writeInt(bytes.length);// 消息正文out.writeBytes(bytes);}// 解码 进站调用@Overrideprotected void decode(ChannelHandlerContext ctx, ByteBuf in, List<Object> out) throws Exception {int magicNum = in.readInt();byte version = in.readByte();byte serializerType = in.readByte();byte messageType = in.readByte();int sequenceId = in.readInt();in.readByte();int length = in.readInt();byte[] bytes = new byte[length];in.readBytes(bytes, 0, length);ObjectInputStream ois = new ObjectInputStream(new ByteArrayInputStream(bytes));Message message = (Message) ois.readObject();log.debug("{}, {}, {}, {}, {}, {}", magicNum, version, serializerType, messageType, sequenceId, length);log.debug("{}", message);out.add(message);}
}

测试

public class TestMessageCodec {public static void main(String[] args) throws Exception {EmbeddedChannel channel = new EmbeddedChannel(new LoggingHandler(LogLevel.DEBUG),new LengthFieldBasedFrameDecoder(1024, 12, 4, 0, 0),//解决黏包半包new MessageCodec());//出站 encodeLoginRequestMessage message = new LoginRequestMessage("zhangsan", "123", "张三");channel.writeOutbound(message);//入站 decodeByteBuf buf = ByteBufAllocator.DEFAULT.buffer();new MessageCodec().encode(null, message, buf);//channel.writeInbound(buf);//模拟半包问题ByteBuf s1 = buf.slice(0, 100);ByteBuf s2 = buf.slice(100, buf.readableBytes() - 100);s1.retain();channel.writeInbound(s1); // release 1channel.writeInbound(s2);}
}

11:02:40 [DEBUG] [main] i.n.h.l.LoggingHandler - [id: 0xembedded, L:embedded - R:embedded] READ: 100B+-------------------------------------------------+|  0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f |
+--------+-------------------------------------------------+----------------+
|00000000| 01 02 03 04 01 00 00 00 00 00 00 ff 00 00 00 fd |................|
|00000010| ac ed 00 05 73 72 00 36 63 6e 2e 69 74 63 61 73 |....sr.6cn.itcas|
|00000020| 74 2e 6e 65 74 74 79 2e 6a 69 6e 6a 69 65 2e 63 |t.netty.jinjie.c|
|00000030| 32 2e 70 72 6f 74 6f 63 6f 6c 2e 4c 6f 67 69 6e |2.protocol.Login|
|00000040| 52 65 71 75 65 73 74 4d 65 73 73 61 67 65 43 a4 |RequestMessageC.|
|00000050| f9 f0 14 8a f7 ce 02 00 03 4c 00 04 6e 61 6d 65 |.........L..name|
|00000060| 74 00 12 4c                                     |t..L            |
+--------+-------------------------------------------------+----------------+
11:02:40 [DEBUG] [main] i.n.h.l.LoggingHandler - [id: 0xembedded, L:embedded - R:embedded] READ COMPLETE
11:02:40 [DEBUG] [main] i.n.h.l.LoggingHandler - [id: 0xembedded, L:embedded - R:embedded] READ: 169B+-------------------------------------------------+|  0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f |
+--------+-------------------------------------------------+----------------+
|00000000| 6a 61 76 61 2f 6c 61 6e 67 2f 53 74 72 69 6e 67 |java/lang/String|
|00000010| 3b 4c 00 08 70 61 73 73 77 6f 72 64 71 00 7e 00 |;L..passwordq.~.|
|00000020| 01 4c 00 08 75 73 65 72 6e 61 6d 65 71 00 7e 00 |.L..usernameq.~.|
|00000030| 01 78 72 00 2a 63 6e 2e 69 74 63 61 73 74 2e 6e |.xr.*cn.itcast.n|
|00000040| 65 74 74 79 2e 6a 69 6e 6a 69 65 2e 63 32 2e 70 |etty.jinjie.c2.p|
|00000050| 72 6f 74 6f 63 6f 6c 2e 4d 65 73 73 61 67 65 78 |rotocol.Messagex|
|00000060| d7 24 ab 71 27 ac 87 02 00 02 49 00 0b 6d 65 73 |.$.q'.....I..mes|
|00000070| 73 61 67 65 54 79 70 65 49 00 0a 73 65 71 75 65 |sageTypeI..seque|
|00000080| 6e 63 65 49 64 78 70 00 00 00 00 00 00 00 00 74 |nceIdxp........t|
|00000090| 00 06 e5 bc a0 e4 b8 89 74 00 03 31 32 33 74 00 |........t..123t.|
|000000a0| 08 7a 68 61 6e 67 73 61 6e                      |.zhangsan       |
+--------+-------------------------------------------------+----------------+
11:02:40 [DEBUG] [main] c.i.n.j.c.p.MessageCodec - 16909060, 1, 0, 0, 0, 253
11:02:40 [DEBUG] [main] c.i.n.j.c.p.MessageCodec - LoginRequestMessage(super=Message(sequenceId=0, messageType=0), username=zhangsan, password=123, name=张三)
11:02:40 [DEBUG] [main] i.n.h.l.LoggingHandler - [id: 0xembedded, L:embedded - R:embedded] READ COMPLETE

解读

💡 什么时候可以加 @Sharable

• 当 handler 不保存状态时，就可以安全地在多线程下被共享，可以加@Sharable
• 但要注意对于编解码器类，不能继承 ByteToMessageCodec 或 CombinedChannelDuplexHandler 父类，他们的构造方法对 @Sharable 有限制
• 如果能确保编解码器不会保存状态，可以继承 MessageToMessageCodec 父类，再加@Sharable注解

@Slf4j
@ChannelHandler.Sharable
/*** 必须和 LengthFieldBasedFrameDecoder 一起使用，确保接到的 ByteBuf 消息是完整的*/
public class MessageCodecSharable extends MessageToMessageCodec<ByteBuf, Message> {@Overrideprotected void encode(ChannelHandlerContext ctx, Message msg, List<Object> outList) throws Exception {ByteBuf out = ctx.alloc().buffer();// 1. 4 字节的魔数out.writeBytes(new byte[]{1, 2, 3, 4});// 2. 1 字节的版本,out.writeByte(1);// 3. 1 字节的序列化方式 jdk 0 , json 1out.writeByte(0);// 4. 1 字节的指令类型out.writeByte(msg.getMessageType());// 5. 4 个字节out.writeInt(msg.getSequenceId());// 无意义，对齐填充out.writeByte(0xff);// 6. 获取内容的字节数组ByteArrayOutputStream bos = new ByteArrayOutputStream();ObjectOutputStream oos = new ObjectOutputStream(bos);oos.writeObject(msg);byte[] bytes = bos.toByteArray();// 7. 长度out.writeInt(bytes.length);// 8. 写入内容out.writeBytes(bytes);outList.add(out);}@Overrideprotected void decode(ChannelHandlerContext ctx, ByteBuf in, List<Object> out) throws Exception {int magicNum = in.readInt();byte version = in.readByte();byte serializerType = in.readByte();byte messageType = in.readByte();int sequenceId = in.readInt();in.readByte();int length = in.readInt();byte[] bytes = new byte[length];in.readBytes(bytes, 0, length);ObjectInputStream ois = new ObjectInputStream(new ByteArrayInputStream(bytes));Message message = (Message) ois.readObject();log.debug("{}, {}, {}, {}, {}, {}", magicNum, version, serializerType, messageType, sequenceId, length);log.debug("{}", message);out.add(message);}
}