mikeash.com pyblog/friday-qa-2013-12-06-network-protocol-design.html commentshttp://www.mikeash.com/?page=pyblog/friday-qa-2013-12-06-network-protocol-design.html#commentsmikeash.com Recent CommentsFri, 29 Mar 2024 10:34:47 GMTPyRSS2Gen-1.0.0http://blogs.law.harvard.edu/tech/rssPeter N Lewis - 2014-05-08 05:23:20http://www.mikeash.com/?page=pyblog/friday-qa-2013-12-06-network-protocol-design.html#commentsA lot of good advice in here. One thing I'd say is to more thoroughly discourage UDP. <br /> <br />I would advise using UDP only in the case where what you're sending fits entirely in a single packet, and can be duplicated or lost without consequence. <br /> <br />As soon as you have to worry about retransmitting or anything more than completely trivial sequencing its time to use TCP - do not try to implement your own retransmittion scheme in UDP, it is practically guarentees to end badly.2bcf4c9c22efefb35812f1a72f6889f8Thu, 08 May 2014 05:23:20 GMTmikeash - 2014-01-24 15:36:22http://www.mikeash.com/?page=pyblog/friday-qa-2013-12-06-network-protocol-design.html#commentsChecksums aren't really necessary for higher level protocols, since you can generally trust the underlying transport protocol (e.g. TCP) to handle checksums for you already. Of course, there's no harm and it can soothe the paranoid soul, and they are indeed absolutely necessary for the bottom of the stack where you're not riding on something that checks integrity for you already. <br /> <br />For resynchronization, that's one of the beautiful properties of the COBS system I touched on. If you ever lose your place, just wait until the next zero byte, and thes keep going from there.59cabf9ec576b64eae956adc5e4d97f5Fri, 24 Jan 2014 15:36:22 GMTSteve Weller - 2014-01-22 05:43:05http://www.mikeash.com/?page=pyblog/friday-qa-2013-12-06-network-protocol-design.html#commentsTwo things I would add. I've designed and implemented protocols for use over serial ports and radio modems that have done some fairly important control functions (such as flying large airplanes remotely). Typically these were implemented in assembler and at least partially inside interrupt routines. <br /> <br />One: always include checksums of some kind. CRC32 is good as long as your packets are small, and it's fast to calculate. Use the parity the bit too if you can. <br /> <br />Two: provide enough inefficiency to allow for the identification of sequences that must always occur at certain points, and sequences that must never occur anywhere. These can be used to reject bad data wholesale and allow you to resynchronize if needed.e50c8edaf5cb5fe63110c3912f8e13aaWed, 22 Jan 2014 05:43:05 GMTGuilherme Rambo - 2013-12-24 19:59:17http://www.mikeash.com/?page=pyblog/friday-qa-2013-12-06-network-protocol-design.html#commentsOn the next friday Q&amp;A you could talk about customizing NSWindows, I've been messing around with this and It's pretty tricky, but totally doable :)81789e08f00d8c0388f3e2f4523a3177Tue, 24 Dec 2013 19:59:17 GMTRichard Clark - 2013-12-09 15:11:28http://www.mikeash.com/?page=pyblog/friday-qa-2013-12-06-network-protocol-design.html#comments@Slava To clarify, all standards-compliant WebSocket implementations (RFC-6455) have binary support. The interim protocol versions didn't define this until mid-2011 and some implementations of that persisted until recently (esp. in iOS). If you need to support older browsers, there are both open-source and commercial implementations available that will do the job.1bdfce162710ddb9d4c16d601b875b56Mon, 09 Dec 2013 15:11:28 GMTnevyn - 2013-12-07 02:29:21http://www.mikeash.com/?page=pyblog/friday-qa-2013-12-06-network-protocol-design.html#commentsWhen playing around with simple networked stuff, I found myself sending length-prefixed JSON on top of AsyncSocket often enough that I abstracted it out into a pretty nice and simple class: <a href="https://github.com/nevyn/TCAsyncHashProtocol/tree/0a438239e1ee970e14b1d4186b2ca9e84baa7133">https://github.com/nevyn/TCAsyncHashProtocol/tree/0a438239e1ee970e14b1d4186b2ca9e84baa7133</a> . Supports request/response, NSData payloads (as an attachment after the JSON), and automatic selector dispatch based on the incoming message (that last part might be a terrible idea). (linking to an old hash as I made the lib too complex on master). <br /> <br />AsyncSocket is great. I love it. <br /> <br />We used quite a bit of protobuf at Spotify, and weren't super happy. I don't remember all the reasons... Generating sources is a bit of a PITA. It was something about runtime version compatibility that was problematic. I've heard good things about Cap'n Proto though ( <a href="http://kentonv.github.io/capnproto/">http://kentonv.github.io/capnproto/</a> ).7a4d74706422fc40715a634e209fa0cfSat, 07 Dec 2013 02:29:21 GMTSlava Zakovyrya - 2013-12-06 16:44:34http://www.mikeash.com/?page=pyblog/friday-qa-2013-12-06-network-protocol-design.html#comments@Jens Alfke: WebSockets are fine until you need support for binary frames. Not all implementations of WebSockets have that, which was a deal breaker for me. Ended up with custom solution utilizing STOMP: <a href="http://stomp.github.io">http://stomp.github.io</a>c1f443db8ba5e02265068875fb1b8d01Fri, 06 Dec 2013 16:44:34 GMTJens Alfke - 2013-12-06 15:21:50http://www.mikeash.com/?page=pyblog/friday-qa-2013-12-06-network-protocol-design.html#commentsAnyone wanting to do this should really read Marshall Rose’s “On The Design Of Application Protocols” (RFC 3117): <br /> <br /><a href="http://tools.ietf.org/html/rfc3117">http://tools.ietf.org/html/rfc3117</a> <br /> <br />(Note that this refers to a “BXXP” protocol, which was later renamed “BEEP”.) <br /> <br />For the most part I feel the same way about designing new wire protocols as I do about designing new crypto systems: don’t do it unless you’re an expert, or your code will be a rich source of bugs and potentially security exploits. It’s easy to make mistakes in framing or encoding/decoding data. <br /> <br />These days, if you need a simple request/response flow, HTTP is a good bet. If you need something a bit fancier, where each side can initiate messages, use WebSockets.3abc76c24fd36671e8de26cb76894544Fri, 06 Dec 2013 15:21:50 GMT