注册 登录  
 加关注
   显示下一条  |  关闭
温馨提示!由于新浪微博认证机制调整,您的新浪微博帐号绑定已过期,请重新绑定!立即重新绑定新浪微博》  |  关闭

老和山小和尚

敬天爱人

 
 
 

日志

 
 
 
 

SPDY 协议草案  

2009-11-13 17:04:26|  分类: 协议研究 |  标签: |举报 |字号 订阅

  下载LOFTER 我的照片书  |
chromium.org 被墙了,转发一下,方便大家观看

Mike Belshe (mbelshe at google.com) & Roberto Peon (fenix at google.com)

DRAFT

Overview

One of the bottlenecks of current HTTP is that it relies solely on multiple connections for concurrency. This causes several problems, including additional round trips for connection setup, slow-start delays, and a constant rationing by the client where it tries to avoid opening too many connections to a single server. HTTP "pipelining" doesn't help, as each connection may be blocked on the request at the head of the line; in addition, many proxies apparently have poor support for pipelining. Applications, in their desire to create many connections, create many sub-domains to work around browser per-domain connection throttling.


SPDY aims to address this and other application-layer problems associated with modern web applications, while requiring little or no change from the perspective of web application writers.


In a nutshell, SPDY adds a framing layer for multiplexing multiple, concurrent streams across a single TCP connection.  The framing layer is optimized for HTTP-like request-response streams.


The SPDY session offers three basic improvements over HTTP:

  • Multiplexed requests. There is no limit to the number of requests that can be issued concurrently over a single SPDY connection.  Because requests are interleaved on a single channel, the efficiency of TCP is much higher.
  • Prioritized requests. Clients can request certain resources to be delivered first.  This avoids the problem of congesting the network channel with non-critical resources when a high-priority request is pending.
  • Compressed headers.  Clients today send a significant amount of redundant data in the form of HTTP headers.  Because a single web page may require 50 or 100 subrequests, this data is significant. Compressing the headers saves a significant amount of latency and bandwidth compared to HTTP.
Note that for the most part, SPDY attempts to preserve the existing semantics of HTTP features.  All features such as cookies, etags, vary headers, content-encoding negotiations, etc work exactly as they do with HTTP; SPDY only replaces the way the data is written to the network.

Definitions

  • connection: A TCP-level connection between two endpoints.
  • endpoint: Either the client or server of a connection.
  • session: A framed sequence of data chunks. Frames are defined as SPDY frames; see Framing below.
  • stream: A bi-directional flow of bytes across a virtual channel within a SPDY session.

Main differences from HTTP

SPDY is intended to be as compatible as possible with current web-based applications. This means that, from the perspective of the server business logic or application API, nothing has changed. To achieve this, all of the application request and response header semantics are preserved.  SPDY introduces a "session" which resides between the HTTP application layer and the TCP transport to regulate the flow of data. This "session" is akin to an HTTP request-response pair. The following changes represent the differences between SPDY and HTTP:

The request

To initiate a new request, clients first create a new SPDY session.  Once the session is created, the client can create a new SPDY stream to carry the request.  Part of creating the stream is sending the HTTP header block.  The HTTP header block in SPDY is almost unchanged from today's HTTP header block, with the following differences:

  • The first line of the request is unfolded into name/value pairs like other HTTP headers.  The names of the first line fields are de style="color: rgb(0, 96, 0); "<methodde<, de style="color: rgb(0, 96, 0); "<urlde<, and de style="color: rgb(0, 96, 0); "<versionde<.  These keys are required to be present.  The 'url' is the fully-qualified URL, containing protocol, host, port, and path.
  • Duplicate header names are not allowed.
  • Header names are all lowercase.
  • The de style="color: rgb(0, 96, 0); "<Connectionde< and de style="color: rgb(0, 96, 0); "<Keep-Alivede< headers are no longer valid and are ignored if present.
  • Clients are assumed to support de style="color: rgb(0, 96, 0); "<Accept-Encoding: gzipde<.  Clients that do not specify any body encodings receive gzip-encoded data from the server.
  • HTTP request headers are compressed.  This is accomplished by compressing all data sent by the client with gzip encoding.
  • The "host" header is ignored.  The host:port portion of the HTTP URL is the definitive host.
  • POST-specific changes:
    • POST requests are expected to contain a data stream as part of the post; see Data flow below.
    • de style="color: rgb(0, 96, 0); "<Content-lengthde< is not a valid header.
    • Chunked encoding is no longer valid.
    • The POST data stream is terminated by a zero-length data frame.

The response

When responding to a HTTP request, servers will send data frames using the SPDY stream created by the client.  The response is similar to HTTP/1.1 in that it consists of a header block followed by a body. However, there are a few notable changes:

  • The response status line is unfolded into name/value pairs like other HTTP headers.  The names of the status line are de style="color: rgb(0, 96, 0); "<status de<and de style="color: rgb(0, 96, 0); "<versionde<.  These keys are required to be present
  • If the SPDY reply happens before a SYN_STREAM, then it includes parameters that inform the client regarding the request that would have been made to receive this response, by including de style="color: rgb(0, 96, 0); "<url de<and de style="color: rgb(0, 96, 0); "<methodde< keys. 
  • All header names must be lowercase.
  • The de style="color: rgb(0, 96, 0); "<Connectionde< and de style="color: rgb(0, 96, 0); "<Keep-alivede< response headers are no longer valid.
  • de style="color: rgb(0, 96, 0); "<Content-lengthde< is no longer valid.
  • Chunked encoding is no longer valid.
  • Duplicate header names are not allowed.

Connections

The first implementation of the SPDY session runs atop TCP, similarly to how HTTP works today. The client is expected to be the TCP connection initiator. Because it runs on TCP, we have a reliable transport. Unlike HTTP, all connections with SPDY are persistent connections. The HTTP connection header does not apply.

For best performance, it is expected that clients will not close open connections until the user navigates away from all web pages referencing a connection, or until the server closes the connection. Servers are encouraged to leave connections open for as long as possible, but can terminate idle connections after some amount of inactivity if necessary.

Framing

Once the TCP connection is established, clients and servers exchange framed messages. There are two types of frames: control frames and data frames.  Frames always have a common header which is 8 bytes.

The first bit is a control bit indicating whether a frame is a control frame or data frame. Control frames carry a version number, a frame type, flags, and a length. Data frames contain the stream ID, flags, and the length for the payload carried after the common header. The simple header is designed to make reading and writing of frames easy.

Control frames

  +----------------------------------+
  |C| Version(15bits) | Type(16bits) |
  +----------------------------------+
  | Flags (8)  |  Length (24 bits)   |
  +----------------------------------+
  |               Data               |
  +----------------------------------+

Control frame fields:

Control bit: The 'C' bit is a single bit indicating that this is a control message. For control frames this value is always 1. 

Version: The version number of the session protocol (currently 1).

Type: The type of control frame. Control frames are SYN_STREAM, SYN_REPLY, etc.

Flags: Flags related to this frame. Flags for control frames and data frames are different.

Length: An unsigned 24-bit value representing the number of bytes after the length field.

Data: data associated with this control frame. The format and length of this data is controlled by the control frame type.

Data frames
  +----------------------------------+
  |C|       Stream-ID (31bits)       |
  +----------------------------------+
  | Flags (8)  |  Length (24 bits)   |
  +----------------------------------+
  |               Data               |
  +----------------------------------+

Data frame fields:

Control bit: For data frames this value is always 0.

Stream-ID: A 31-bit value identifying the stream.

Flags: Flags related to this frame. Valid flags are:
  • 0x01 = FLAG_FIN - signifies that this frame represents the half-close of this stream. See Stream half-close below.
Length: An unsigned 24-bit value representing the number of bytes after the length field. The total size of a data frame is 8 bytes + length. It is valid to have a zero-length data frame.

Data: A variable-length field containing the number of bytes in the payload.

Hello message

After the connection has been established, SPDY employs an asynchronous Hello sequence where each side informs the other about the communication details it knows about.  Unlike most protocols, this Hello sequence is optional and fully asynchronous.  Because it is asynchronous, it does not add a round-trip latency to the connection startup.  But because it is asynchronous and optional, both sides must be prepared for this message to arrive at any time or not at all.

To initiate a Hello sequence, either side can send a HELLO control frame.  The Hello frame is optional, but if it is to be sent, it must be the first frame sent.  When a Hello message is received, the receiver is not obligated to reply with a Hello message in return.  The message is therefore completely informational.

HELLO control message:
  +----------------------------------+
  |1|       1          |       4     |
  +----------------------------------+
  | Flags (8)  |  Length (24 bits)   |
  +----------------------------------+
  |  Unused       |Number of entries |
  +----------------------------------|
  |          ID/Value Pairs          |
  |             ...                  |

HELLO message fields:

Control bit: The control bit is always 1 for this message.


Version: The SPDY version number.

Type: The message type for a HELLO message is 4.

Unused: 16 bits of unused space, reserved for future use.

Number of entries: A 16-bit value representing the number of ID/value pairs in this message.

ID: A 32-bit ID number. The following IDs are valid:
  • 1 - HELLO_BANDWIDTH_TO_YOU_ID allows the sender to send its expected upload bandwidth on this channel. This number is an estimate. The value should be the integral number of bytes per second that the sender predicts as an expected maximum upload channel capacity.
  • 2 - HELLO_BANDWIDTH_FROM_YOU_ID allows the sender to send its expected download bandwidth on this channel. This number is an estimate. The value should be the integral number of bytes per second that the sender predicts as an expected maximum download channel capacity.
  • 3 - HELLO_ROUND_TRIP_ID allows the sender to send its expected round-trip-time on this channel. The round trip time is defined as the minimum amount of time to send a control frame from this client to the remote and receive a response. The value is represented in milliseconds.
  • 4 - HELLO_MAX_CONCURRENT_STREAMS allows the sender to inform the remote endpoint the maximum number of concurrent streams which it will allow. By default there is no limit. For implementors it is recommended that this value be no smaller than 100.
Value: A 32-bit value.

The message is intentionally expandable for future information which may improve client-server communications. The sender does not need to send every type of ID/value. It must only send those for which it has accurate values to convey. When multiple ID/value pairs are sent, they should be sent in order of lowest value to highest value.

Streams

Streams are independent sequences of bi-directional data cut into frames.  Streams can be created either by the client or the server, can concurrently send data interleaved with other streams, and can be cancelled.

Upon stream initiation, streams allow for each side to transmit a fixed-length list of name/value pairs to the other endpoint.

Stream creation

A stream is created by sending a control packet with the type set to SYN_STREAM(1).  If the server is initiating the stream, the Stream-ID must be even.  If the client is initiating the stream, the Stream-ID must be odd.  0 is not a valid Stream-ID. Stream-IDs must increase monotonically as new streams are created.

Upon receipt of a SYN_STREAM frame, the server replies with a SYN_REPLY frame.  The client does not need to wait for a SYN_REPLY before sending any data frames.


If the endpoint receiving a SYN_STREAM does not want to accept the new stream, it can immediately respond with a FIN_STREAM control frame.  Note, however, that the initiating endpoint may have already sent data on the stream as well; this data must be ignored.



SYN_STREAM control message:
  +----------------------------------+
  |1|       1          |       1     |
  +----------------------------------+
  | Flags (8)  |  Length (24 bits)   |
  +----------------------------------+
  |X|          Stream-ID (31bits)    |
  +----------------------------------+
  | Pri | Unused    |   NV Entries   |
  +----------------------------------|
  |        Name/value pairs          |
  |             ...                  |


SYN_STREAM message fields:

Flags: Flags related to this frame. Valid flags are:

  • 0x01 = FLAG_FIN - signifies that this frame represents the half-close of this stream. When set, it indicates that the stream will not contain any data frames.

Length: An unsigned 24 bit value representing the number of bytes after the length field. The total size of a SYN_STREAM frame is 8 bytes + length. The length for this frame must be greater than or equal to 8.

Priority: A 2-bit priority field. If an endpoint has initiated multiple streams, the priority field represents which streams should be given first precidence. Servers are not required to strictly enforce the priority field, although best-effort is assumed. 0 represents the lowest priority and 3 represents the highest priority. The highest-priority data is that which is most desired by the client.

Unused: 14 bits of unused space, reserved for future use.

NV entries: The number of name/value pairs that follow.

The Name/value block is described below.


SYN_REPLY control message:

 

  +----------------------------------+
  |1|        1        |        2     |
  +----------------------------------+
  | Flags (8)  |  Length (24 bits)   |
  +----------------------------------+
  |X|          Stream-ID (31bits)    |
  +----------------------------------+
  | Unused        |    NV entries    |
  +----------------------------------|
  |         Name/value pairs         |
  |              ...                 |

SYN_REPLY message fields:

Flags: Flags related to this frame. Valid flags are:

0x01 = FLAG_FIN - signifies that this frame represents the half-close of this stream. When set, it indicates that the stream will not contain any data frames.

Length: An unsigned 24-bit value representing the number of bytes after the length field. The total size of a SYN_STREAM frame is 8 bytes + length. The length for this frame must be greater than or equal to 8.

Priority: A 2-bit priority field. If an endpoint has initiated multiple streams, the priority field represents which streams should be given first precedence. Servers are not required to strictly enforce the priority field, although best-effort is assumed. 0 represents the lowest priority and 3 represents the highest priority.

Unused: 14 bits of unused space, reserved for future use.

NV entries: The number of name/value pairs that follow.

The Name/value block is described below.

Name/value header block format

Both the SYN_STREAM and SYN_REPLY frames contain a Name/value header block.  The header block used by both the request and the response is the same.  It is designed so that headers can be easily appended to the end of a list and also so that it can be easily parsed by receivers.  Each numeric value is 2 bytes.


  +----------------------------------+
  |     Number of headers (int16)    |
  +----------------------------------+
  |     Length of name (int16)       |
  +----------------------------------+
  |           Name (string)          |
  +----------------------------------+
  |     Length of value  (int16)     |
  +----------------------------------+
  |          Value   (string)        |
  +----------------------------------+
  |           (etc)                  |


Each header name must have at least one value. The length of each name and value must be greater than zero.


Duplicate header names are not allowed.  To send two identically named headers, send a header with two values.


TODO:  Specify a string encoding.


TODO: Describe compression mechanism.

Stream data exchange

Once a stream is created, it is used to send arbitrary amounts of data in either direction.  When either side has finished sending data it can send a frame with the FIN_FLAG set. (See "TCP connection teardown" below.)

Stream half-close

When one side of the stream sends a control or data frame with the FLAG_FIN flag set, the stream is considered to be half-closed from that side. The sender of the FLAG_FIN is indicating that no further data will be sent from the sender on this stream. When both sides have half-closed, the stream is considered to be closed.

Stream close

There are 3 ways that streams can be terminated: normal termination, abrupt termination, and TCP connection teardown.

Normal termination

Normal stream termination occurs when both sides of the stream have half-closed the stream.

Abrupt termination

Either the client or server can send a FIN_STREAM control packet at any time.  Upon receipt of the FIN_STREAM, both sides must ignore further data received on the stream and both sides must stop transmitting to the stream.

FIN_STREAM control frame:
  +-------------------------------+
  |1|       1        |      3     |
  +-------------------------------+
  | Flags (8)  |         8        |
  +-------------------------------+
  |X|          Stream-ID (31bits) |
  +-------------------------------+
  |          Status code          | 
  +-------------------------------+

FIN_STREAM message fields:

Flags: Flags related to this frame. Valid flags are:

  • 0x01 = FLAG_FIN - signifies that this frame represents the half-close of this stream. See "Stream half-close" above.
Length: An unsigned 24-bit value representing the number of bytes after the length field. For FIN_STREAM control frames, this value is always 8.

Status code: An indicator for why the stream is being terminated.The following status codes are defined:
  • 1 - PROTOCOL_ERROR. This is a generic error, and should only be used if a more specific error is not available. The receiver of this error will likely abort the entire session and possibly return an error to the user.
  • 2 - INVALID_STREAM should be returned when a frame is received for a frame which is not active. The receiver of this error will likely log a communications error.
TODO - Define more specific errors.

TCP connection teardown


If the TCP connection is torn down while unterminated streams are active (no FIN_STREAM frames have been sent or received for the stream), then the endpoint must assume that the stream was abnormally interrupted and may be incomplete.


If a client or server receives data on a stream which has already been torn down, it must ignore the data received after the teardown.

Data flow

Because TCP provides a single stream of data on which SPDY multiplexes multiple logical streams, it is important for clients and servers to interleave data messages for concurrent sessions.

Implementors should note that sending data in smallish chunks will result in lower end-user latency for a page as this allows the browser to begin parsing metadata early, and, in turn, to finalize the page layout. Sending large chunks yields a small increase in bandwidth efficiency at the cost of slowing down the user experience on pages with many resources.

Other control frames

NOOP

The NOOP control frame is a no-operation frame.  It can be sent from the client or the server.  Receivers of a NO-OP frame should simply ignore it.

NOTE: This control frame may ultimately be removed.  It has been implemented for experimentation purposes.

NOOP control message:
  +----------------------------------+
  |1|       1          |       5     |
  +----------------------------------+
  | 0 (Flags)  |    0 (Length)       |
  +----------------------------------+

Control-bit: The control bit is always 1 for this message.

Version: The SPDY version number.

Type: The message type for a NOOP message is 5.

Length: This frame carries no data, so the length is always 0.

PING

The PING control frame is a mechanism for measuring a minimal round-trip time from the sender.  It can be sent from the client or the server.  Receivers of a PING frame should send an identical frame to the sender as soon as possible (if there is other pending data waiting to be sent, PING should take highest priority).  Each ping sent by a sender should use a unique ID.

NOTE: This control frame may ultimately be removed.  It has been implemented for experimentation purposes.
PING  control message:
  +----------------------------------+
  |1|       1          |       6     |
  +----------------------------------+
  | 0 (flags) |     4 (length)       |
  +----------------------------------|
  |            32-bit ID             |
  +----------------------------------|

Control bit: The control bit is always 1 for this message.

Version: The SPDY version number.

Type: The message type for a PING message is 6.

Length: This frame is always 4 bytes long.

ID: A unique ID for this ping.

Note: If a sender uses all possible PING ids (e.g. has sent all 2^32 possible IDs), it can "wrap" and start re-using IDs.

GOAWAY

The GOAWAY control frame is a mechanism to tell the remote side of the connection that it should no longer use this session for further requests.  It can be sent from the client or the server. Receivers of a GOAWAY frame must not send additional requests on this session, although a new session can be established for new requests.  The purpose of this message is to allow the server to gracefully stop accepting new requests (perhaps for a reboot or maintenance), while still finishing processing of previously established requests.

There is an inherent race condition between a client sending requests and the server sending a GOAWAY message.  To deal with this case, the GOAWAY message contains a last-stream identifier indicating the last stream which was accepted in this session.  If the client issued requests for sessions after this stream-id, they were not accepted by the server and should be re-issued later at the client's discretion.

NOTE: This control frame may ultimately be removed.  It has been implemented for experimentation purposes.

GOAWAY control message:
  +----------------------------------+
  |1|       1          |       7     |
  +----------------------------------+
  | 0 (flags) |     4 (length)       |
  +----------------------------------|
  |       Last-good-stream-ID        |
  +----------------------------------|

Control bit: The control bit is always 1 for this message.

Version: The SPDY version number.

Type: The message type for a GOAWAY message is 7.

Length: This frame is always 4 bytes long.

Last-good-stream-Id: The last stream id which was accepted by the sender of the GOAWAY message.

Future work/experiments

The following are thoughts/brainstorms for future work and experiments.
  • Caching
    Since we're proposing to do almost everything over an encrypted channel, we're making caching either difficult or impossible.
    We've had some discussions about having a non-secure, static-only content channel (where the resources are signed, or cryptographic hashes of the insecure content are sent over a secure link), but have made no headway yet...

  • DoS potential
The potential for DoS of the server (by increasing the amount of state) is no worse than it would be for TCP connections. In fact, since more of the state is managed in userspace, it is easier to detect and react to increases in resource utilization. The client side, however, does have a potential for DoS since the server may now attempt to use client resources which would not normally be considered to be large per stream state.
More investigation needs to be done about attack modes and remediation responses.
  • Stream capacity.
Today, HTTP has no knowledge of approximate client bandwidth, and TCP only slowly and indirectly discovers it as a connection warms up. However, by the time TCP has warmed up, the page is often already completely loaded. To ensure that bandwidth is utilized efficiently, SPDY allows the client to tell the server its approximate bandwidth. The HELLO message is part of this, but measurement, reporting and all of the other infrastructure and behavioral modifcations are lacking.
  • Server-controlled connection parameters.
The "server" -- since either side may be considered a "server" in SPDY, "server" refers here to the side which receives requests for new sessions -- can inform the other side of reasonable connection or stream limits which should be respected. As with stream capacity, the HELLO message provides a container for signaling this, but no work has yet been done to figure out what values should be set, when, and what behavioral changes should be expected when the value does change.
  • Prefetching/server push
If the client can inform the server (or vice versa) how much bandwidth it is willing to allocate to various activities, then the server can push or prefetch resources without a) impacting the activities that the user wants to perform; or b) being too inefficient. While this approach still has inefficiencies (the server still send sparts of resources that the client already has before the client gets around to sending a FIN to the server), it doesn't seem to be grossly inefficient when coupled with the client telling the server how much slack (bandwidth) it is is willing to give to the server. However, this means that the client must be able to dynamically adjust the slack that it provides for server push or prefetching, for example, if it sees an increase in PING time.
  • Re-prioritization of streams
Users often switch between multiple tabs in the browser. When the user switches tasks, the protocol should allow for a change in priority as the user now wants different data at a higher priority. This would likely involve creating priority groups so that the relative priority of resources for a tab can be managed all at once instead of having to make a number of changes proportional to the number of requests (which is 100% likely to be the same or larger!)
  • Uni-directional (but associated) streams
It may be a mistake to make a connection-analogue in the stream. It is perhaps better to allow each side to make a unidirectional stream, and provide data about which stream it may be associated with. This offers a much easier-to-comprehend association mechanism for server push.
  • Flow control
Each side can announce how much data or bandwidth it will accept for each class of streams. If this is done, then speculative operations such as server push can soak up a limited amount of the pipe (especially important if the pipe is long and thin). This may allow for the elimination of more complex "is this already in the cache" or "announce what I have in my cache" schemes which are likely costly and complex.

  • Use of gzip compression
The use of gzip compression should be stateful only across headers in the channel. Essentially, separate gzip contexts should be maintained for headers versus data.
  • DNS data
It is suboptimal that the browser must discover new hostnames and then look them up in cases where it is fetching new resources controlled by the same entity. For example, there shouldn't be a need to do another DNS lookup for a resource from static.foo.com. when the browser has already resolved xxx.foo.com. In these cases, it would seemingly make sense to send (over the SPDY channel) the DNS data (signed in such a way that the user-agent can verify that it is authoritative).
  • Redirect to a different IP while retaining the HOST header
For large sites or caches, it may be advantageous to supplement DNS frontend load balancing with the ability to send the user to a sibling that is less loaded, but also able to serve the request. This is not possible today with HTTP, as redirects must point to a different name (or use an IP, which amounts to the same thing), so that cookies, etc. are treated differently. This feature would likely be tied to the DNS data feature, or a more interesting verification mechanism would need to exist.
  • New headers
    • Request headers:
      • Never been to this site before header.  When the server receives this header, it is used as permission to open multiple, server-initiated streams carrying subresource content. If sent, then the server can freely push all the resources necessary for the page.
    • Response
      • Subresource headers

  评论这张
 
阅读(1342)| 评论(0)
推荐 转载

历史上的今天

评论

<#--最新日志,群博日志--> <#--推荐日志--> <#--引用记录--> <#--博主推荐--> <#--随机阅读--> <#--首页推荐--> <#--历史上的今天--> <#--被推荐日志--> <#--上一篇,下一篇--> <#-- 热度 --> <#-- 网易新闻广告 --> <#--右边模块结构--> <#--评论模块结构--> <#--引用模块结构--> <#--博主发起的投票-->
 
 
 
 
 
 
 
 
 
 
 
 
 
 

页脚

网易公司版权所有 ©1997-2017