Package org.littleshoot.proxy.impl

Class ClientToProxyConnection

java.lang.Object
io.netty.channel.ChannelHandlerAdapter
io.netty.channel.ChannelInboundHandlerAdapter
io.netty.channel.SimpleChannelInboundHandler<Object>
org.littleshoot.proxy.impl.ProxyConnection<HttpRequest>
org.littleshoot.proxy.impl.ClientToProxyConnection
All Implemented Interfaces:
ChannelHandler, ChannelInboundHandler
public class ClientToProxyConnection extends ProxyConnection<HttpRequest>

Represents a connection from a client to our proxy. Each ClientToProxyConnection can have multiple ProxyToServerConnections, at most one per outbound host:port.

Once a ProxyToServerConnection has been created for a given server, it is continually reused. The ProxyToServerConnection goes through its own lifecycle of connects and disconnects, with different underlying Channels, but only a single ProxyToServerConnection object is used per server. The one exception to this is CONNECT tunneling - if a connection has been used for CONNECT tunneling, that connection will never be reused.

As the ProxyToServerConnections receive responses from their servers, they feed these back to the client by calling respond(ProxyToServerConnection, HttpFilters, HttpRequest, HttpResponse, HttpObject) .

  • Field Details

    • CONNECTION_ESTABLISHED

      private static final HttpResponseStatus CONNECTION_ESTABLISHED

    • HTTP_ENCODER_NAME

      private static final String HTTP_ENCODER_NAME
      See Also:

    • HTTP_DECODER_NAME

      private static final String HTTP_DECODER_NAME
      See Also:

    • HTTP_PROXY_DECODER_NAME

      private static final String HTTP_PROXY_DECODER_NAME
      See Also:

    • HTTP_REQUEST_READ_MONITOR_NAME

      private static final String HTTP_REQUEST_READ_MONITOR_NAME
      See Also:

    • HTTP_RESPONSE_WRITTEN_MONITOR_NAME

      private static final String HTTP_RESPONSE_WRITTEN_MONITOR_NAME
      See Also:

    • MAIN_HANDLER_NAME

      private static final String MAIN_HANDLER_NAME
      See Also:

    • ABSOLUTE_URI_PATTERN

      private static final Pattern ABSOLUTE_URI_PATTERN
      Used for case-insensitive comparisons when checking direct proxy request.

    • serverConnectionsByHostAndPort

      private final Map<String,ProxyToServerConnection> serverConnectionsByHostAndPort
      Keep track of all ProxyToServerConnections by host+port.

    • numberOfCurrentlyConnectingServers

      private final AtomicInteger numberOfCurrentlyConnectingServers
      Keep track of how many servers are currently in the process of connecting.

    • haProxyMessage

      private HAProxyMessage haProxyMessage
      Keep track of proxy protocol header

    • numberOfCurrentlyConnectedServers

      private final AtomicInteger numberOfCurrentlyConnectedServers
      Keep track of how many servers are currently connected.

    • numberOfReusedServerConnections

      private final AtomicInteger numberOfReusedServerConnections
      Keep track of how many times we were able to reuse a connection.

    • currentServerConnection

      private volatile ProxyToServerConnection currentServerConnection
      This is the current server connection that we're using while transferring chunked data.

    • currentFilters

      private volatile HttpFilters currentFilters
      The current filters to apply to incoming requests/chunks.

    • clientSslSession

      private volatile SSLSession clientSslSession

    • mitming

      private volatile boolean mitming
      Tracks whether this ClientToProxyConnection is current doing MITM.

    • authenticated

      private final AtomicBoolean authenticated

    • globalTrafficShapingHandler

      private final GlobalTrafficShapingHandler globalTrafficShapingHandler

    • currentRequest

      private volatile HttpRequest currentRequest
      The current HTTP request that this connection is currently servicing.

    • clientDetails

      private final ClientDetails clientDetails

    • RespondCONNECTSuccessful

      ConnectionFlowStep RespondCONNECTSuccessful
      Tells the Client that its HTTP CONNECT request was successful.

    • bytesReadMonitor

      private final ProxyConnection<HttpRequest>.BytesReadMonitor bytesReadMonitor

    • requestReadMonitor

      private final ProxyConnection<HttpRequest>.RequestReadMonitor requestReadMonitor

    • bytesWrittenMonitor

      private final ProxyConnection<HttpRequest>.BytesWrittenMonitor bytesWrittenMonitor

    • responseWrittenMonitor

      private final ProxyConnection<HttpRequest>.ResponseWrittenMonitor responseWrittenMonitor

  • Constructor Details

  • Method Details

    • readHAProxyMessage

      protected void readHAProxyMessage(HAProxyMessage msg)
      Description copied from class: ProxyConnection
      Read an HAProxyMessage
      Specified by:
      readHAProxyMessage in class ProxyConnection<HttpRequest>
      Parameters:
      msg - HAProxyMessage

    • readHTTPInitial

      protected ConnectionState readHTTPInitial(HttpRequest httpRequest)
      Description copied from class: ProxyConnection
      Implement this to handle reading the initial object (e.g. HttpRequest or HttpResponse).
      Specified by:
      readHTTPInitial in class ProxyConnection<HttpRequest>

    • doReadHTTPInitial

      private ConnectionState doReadHTTPInitial(HttpRequest httpRequest)

      Reads an HttpRequest.

      If we don't yet have a ProxyToServerConnection for the desired server, this takes care of creating it.

      Note - the "server" could be a chained proxy, not the final endpoint for the request.

    • isRequestToOriginServer

      private boolean isRequestToOriginServer(HttpRequest httpRequest)
      Returns true if the specified request is a request to an origin server, rather than to a proxy server. If this request is being MITM'd, this method always returns false. The format of requests to a proxy server are defined in RFC 7230, section 5.3.2 (all other requests are considered requests to an origin server): 
               When making a request to a proxy, other than a CONNECT or server-wide
         OPTIONS request (as detailed below), a client MUST send the target
         URI in absolute-form as the request-target.
         [...]
         An example absolute-form of request-line would be:
         GET https://www.example.org/pub/WWW/TheProject.html HTTP/1.1
         To allow for transition to the absolute-form for all requests in some
         future version of HTTP, a server MUST accept the absolute-form in
         requests, even though HTTP/1.1 clients will only send them in
         requests to proxies.
      Parameters:
      httpRequest - the request to evaluate
      Returns:
      true if the specified request is a request to an origin server, otherwise false

    • readHTTPChunk

      protected void readHTTPChunk(HttpContent chunk)
      Description copied from class: ProxyConnection
      Implement this to handle reading a chunk in a chunked transfer.
      Specified by:
      readHTTPChunk in class ProxyConnection<HttpRequest>

    • readRaw

      protected void readRaw(ByteBuf buf)
      Description copied from class: ProxyConnection
      Implement this to handle reading a raw buffer as they are used in HTTP tunneling.
      Specified by:
      readRaw in class ProxyConnection<HttpRequest>

    • respond

      void respond(ProxyToServerConnection serverConnection, HttpFilters filters, HttpRequest currentHttpRequest, HttpResponse currentHttpResponse, HttpObject httpObject)
      Send a response to the client.
      Parameters:
      serverConnection - the ProxyToServerConnection that's responding
      filters - the filters to apply to the response
      currentHttpRequest - the HttpRequest that prompted this response
      currentHttpResponse - the HttpResponse corresponding to this data (when doing chunked transfers, this is the initial HttpResponse object that came in before the other chunks)
      httpObject - the data with which to respond

    • resetCurrentRequest

      private void resetCurrentRequest()

    • switchToWebSocketProtocol

      private void switchToWebSocketProtocol(ProxyToServerConnection serverConnection)

    • connected

      protected void connected()
      On connect of the client, start waiting for an initial HttpRequest.
      Overrides:
      connected in class ProxyConnection<HttpRequest>

    • timedOut

      void timedOut(ProxyToServerConnection serverConnection)

    • timedOut

      protected void timedOut()
      Description copied from class: ProxyConnection
      This method is called when the underlying Channel times out due to an idle timeout.
      Overrides:
      timedOut in class ProxyConnection<HttpRequest>

    • disconnected

      protected void disconnected()
      On disconnect of the client, disconnect all server connections.
      Overrides:
      disconnected in class ProxyConnection<HttpRequest>

    • serverConnectionFlowStarted

      protected void serverConnectionFlowStarted(ProxyToServerConnection serverConnection)
      Called when ProxyToServerConnection starts its connection flow.

    • serverConnectionSucceeded

      protected void serverConnectionSucceeded(ProxyToServerConnection serverConnection, boolean shouldForwardInitialRequest)
      If the ProxyToServerConnection completes its connection lifecycle successfully, this method is called to let us know about it.

    • serverConnectionFailed

      protected boolean serverConnectionFailed(ProxyToServerConnection serverConnection, ConnectionState lastStateBeforeFailure, Throwable cause)
      If the ProxyToServerConnection fails to complete its connection lifecycle successfully, this method is called to let us know about it.

      After failing to connect to the server, one of two things can happen:

      1. If the server was a chained proxy, we fall back to connecting to the ultimate endpoint directly.
      2. If the server was the ultimate endpoint, we return a 502 Bad Gateway to the client.
      Parameters:
      serverConnection -
      lastStateBeforeFailure -
      cause - what caused the failure
      Returns:
      true if we're falling back to a another chained proxy (or direct connection) and trying again

    • connectionFailedUnrecoverably

      private void connectionFailedUnrecoverably(HttpRequest initialRequest, ProxyToServerConnection serverConnection)

    • resumeReadingIfNecessary

      private void resumeReadingIfNecessary()

    • serverDisconnected

      protected void serverDisconnected(ProxyToServerConnection serverConnection)
      On disconnect of the server, track that we have one fewer connected servers and then disconnect the client if necessary.

    • becameSaturated

      protected void becameSaturated()
      When the ClientToProxyConnection becomes saturated, stop reading on all associated ProxyToServerConnections.
      Overrides:
      becameSaturated in class ProxyConnection<HttpRequest>

    • becameWritable

      protected void becameWritable()
      When the ClientToProxyConnection becomes writable, resume reading on all associated ProxyToServerConnections.
      Overrides:
      becameWritable in class ProxyConnection<HttpRequest>

    • serverBecameSaturated

      protected void serverBecameSaturated(ProxyToServerConnection serverConnection)
      When a server becomes saturated, we stop reading from the client.

    • serverBecameWriteable

      protected void serverBecameWriteable(ProxyToServerConnection serverConnection)
      When a server becomes writeable, we check to see if all servers are writeable and if they are, we resume reading.

    • exceptionCaught

      protected void exceptionCaught(Throwable cause)
      Description copied from class: ProxyConnection
      Override this to handle exceptions that occurred during asynchronous processing on the Channel.
      Overrides:
      exceptionCaught in class ProxyConnection<HttpRequest>

    • initChannelPipeline

      private void initChannelPipeline(ChannelPipeline pipeline)
      Initialize the ChannelPipeline for the client to proxy channel. LittleProxy acts like a server here. A ChannelPipeline invokes the read (Inbound) handlers in ascending ordering of the list and then the write (Outbound) handlers in descending ordering. Regarding the Javadoc of HttpObjectAggregator it's needed to have the HttpResponseEncoder or HttpRequestEncoder before the HttpObjectAggregator in the ChannelPipeline.

    • removeHandlerIfPresent

      private void removeHandlerIfPresent(String name)

    • isAcceptProxyProtocol

      boolean isAcceptProxyProtocol()
      Is the proxy server set to accept a proxy protocol header
      Returns:
      True if the proxy server set to accept a proxy protocol header. False otherwise

    • isSendProxyProtocol

      boolean isSendProxyProtocol()
      Is the proxy server set to send a proxy protocol header
      Returns:
      True if the proxy server set to send a proxy protocol header. False otherwise

    • closeConnectionsAfterWriteIfNecessary

      private void closeConnectionsAfterWriteIfNecessary(ProxyToServerConnection serverConnection, HttpRequest currentHttpRequest, HttpResponse currentHttpResponse, HttpObject httpObject)
      This method takes care of closing client to proxy and/or proxy to server connections after finishing a write.

    • forceDisconnect

      private void forceDisconnect(ProxyToServerConnection serverConnection)

    • shouldCloseClientConnection

      private boolean shouldCloseClientConnection(HttpRequest req, HttpResponse res, HttpObject httpObject)
      Determine whether or not the client connection should be closed.

    • shouldCloseServerConnection

      private boolean shouldCloseServerConnection(HttpRequest req, HttpResponse res, HttpObject msg)
      Determines if the remote connection should be closed based on the request and response pair. If the request is HTTP 1.0 with no keep-alive header, for example, the connection should be closed. This in part determines if we should close the connection. Here's the relevant section of RFC 2616: "HTTP/1.1 defines the "close" connection option for the sender to signal that the connection will be closed after completion of the response. For example, Connection: close in either the request or the response header fields indicates that the connection SHOULD NOT be considered `persistent' (section 8.1) after the current request/response is complete."
      Parameters:
      req - The request.
      res - The response.
      msg - The message.
      Returns:
      Returns true if the connection should close.

    • authenticationRequired

      private boolean authenticationRequired(HttpRequest request)

      Checks whether the given HttpRequest requires authentication.

      If the request contains credentials, these are checked.

      If authentication is still required, either because no credentials were provided or the credentials were wrong, this writes a 407 response to the client.

    • writeAuthenticationRequired

      private void writeAuthenticationRequired(String realm)

    • copy

      private HttpRequest copy(HttpRequest original)
      Copy the given HttpRequest verbatim.

    • fixHttpVersionHeaderIfNecessary

      private void fixHttpVersionHeaderIfNecessary(HttpResponse httpResponse)
      Chunked encoding is an HTTP 1.1 feature, but sometimes we get a chunked response that reports its HTTP version as 1.0. In this case, we change it to 1.1.

    • modifyRequestHeadersToReflectProxying

      private void modifyRequestHeadersToReflectProxying(HttpRequest httpRequest)
      If and only if our proxy is not running in transparent mode, modify the request headers to reflect that it was proxied.

    • isNextHopOriginServer

      private boolean isNextHopOriginServer()

    • modifyResponseHeadersToReflectProxying

      private void modifyResponseHeadersToReflectProxying(HttpResponse httpResponse)
      If and only if our proxy is not running in transparent mode, modify the response headers to reflect that it was proxied.

    • switchProxyConnectionHeader

      private void switchProxyConnectionHeader(HttpHeaders headers)
      Switch the de-facto standard "Proxy-Connection" header to "Connection" when we pass it along to the remote host. This is largely undocumented but seems to be what most browsers and servers expect.
      Parameters:
      headers - The headers to modify

    • stripConnectionTokens

      private void stripConnectionTokens(HttpHeaders headers)
      RFC2616 Section 14.10 HTTP/1.1 proxies MUST parse the Connection header field before a message is forwarded and, for each connection-token in this field, remove any header field(s) from the message with the same name as the connection-token.
      Parameters:
      headers - The headers to modify

    • stripHopByHopHeaders

      private void stripHopByHopHeaders(HttpHeaders headers)
      Removes all headers that should not be forwarded. See RFC 2616 13.5.1 End-to-end and Hop-by-hop Headers.
      Parameters:
      headers - The headers to modify

    • writeBadGateway

      private boolean writeBadGateway(HttpRequest httpRequest)
      Tells the client that something went wrong trying to proxy its request. If the Bad Gateway is a response to an HTTP HEAD request, the response will contain no body, but the Content-Length header will be set to the value it would have been if this 502 Bad Gateway were in response to a GET.
      Parameters:
      httpRequest - the HttpRequest that is resulting in the Bad Gateway response
      Returns:
      true if the connection will be kept open, or false if it will be disconnected

    • writeBadRequest

      private boolean writeBadRequest(HttpRequest httpRequest)
      Tells the client that the request was malformed or erroneous. If the Bad Request is a response to an HTTP HEAD request, the response will contain no body, but the Content-Length header will be set to the value it would have been if this Bad Request were in response to a GET.
      Returns:
      true if the connection will be kept open, or false if it will be disconnected

    • writeGatewayTimeout

      private boolean writeGatewayTimeout(HttpRequest httpRequest)
      Tells the client that the connection to the server, or possibly to some intermediary service (such as DNS), timed out. If the Gateway Timeout is a response to an HTTP HEAD request, the response will contain no body, but the Content-Length header will be set to the value it would have been if this 504 Gateway Timeout were in response to a GET.
      Parameters:
      httpRequest - the HttpRequest that is resulting in the Gateway Timeout response
      Returns:
      true if the connection will be kept open, or false if it will be disconnected

    • respondWithShortCircuitResponse

      private boolean respondWithShortCircuitResponse(HttpResponse httpResponse)
      Responds to the client with the specified "short-circuit" response. The response will be sent through the HttpFilters.proxyToClientResponse(HttpObject) filter method before writing it to the client. The client will not be disconnected, unless the response includes a "Connection: close" header, or the filter returns a null HttpResponse (in which case no response will be written to the client and the connection will be disconnected immediately). If the response is not a Bad Gateway or Gateway Timeout response, the response's headers will be modified to reflect proxying, including adding a Via header, Date header, etc.
      Parameters:
      httpResponse - the response to return to the client
      Returns:
      true if the connection will be kept open, or false if it will be disconnected.

    • identifyHostAndPort

      private String identifyHostAndPort(HttpRequest httpRequest)
      Identify the host and port for a request.

    • writeEmptyBuffer

      private ChannelFuture writeEmptyBuffer()
      Write an empty buffer at the end of a chunked transfer. We need to do this to handle the way Netty creates HttpChunks from responses that aren't in fact chunked from the remote server using Transfer-Encoding: chunked. Netty turns these into pseudo-chunked responses in cases where the response would otherwise fill up too much memory or where the length of the response body is unknown. This is handy because it means we can start streaming response bodies back to the client without reading the entire response. The problem is that in these pseudo-cases the last chunk is encoded to null, and this thwarts normal ChannelFutures from propagating operationComplete events on writes to appropriate channel listeners. We work around this by writing an empty buffer in those cases and using the empty buffer's future instead to handle any operations we need to when responses are fully written back to clients.

    • isMitming

      public boolean isMitming()

    • setMitming

      protected void setMitming(boolean isMitming)

    • recordClientConnected

      private void recordClientConnected()

    • recordClientSSLHandshakeSucceeded

      private void recordClientSSLHandshakeSucceeded()

    • recordClientDisconnected

      private void recordClientDisconnected()

    • getClientAddress

      public InetSocketAddress getClientAddress()

    • flowContext

      private FlowContext flowContext()

    • getHaProxyMessage

      public HAProxyMessage getHaProxyMessage()

    • getClientDetails

      public ClientDetails getClientDetails()