o bB@sddlmZddlZddlZddlZddlmZddlmZm Z ddlm Z ddl m Z m Z mZddlmZdd lmZdd lmZdd lmZdd lmZgd ZeeZdZdZedeZ ddZ!e"e!e e"e!e dZ#ee dZ$GdddeZ%Gddde%Z&ddZ'dS))absolute_importN)RecentlyUsedContainer)HTTPConnectionPoolHTTPSConnectionPool)port_by_scheme)LocationValueError MaxRetryErrorProxySchemeUnknown)six)urljoin)RequestMethods) parse_url)Retry) PoolManager ProxyManagerproxy_from_url)key_file cert_file cert_reqsca_certs ssl_version ca_cert_dir ssl_context key_password) key_schemekey_hostkey_port key_timeout key_retries key_strict key_blockkey_source_address key_key_filekey_key_password key_cert_file key_cert_reqs key_ca_certskey_ssl_versionkey_ca_cert_dirkey_ssl_context key_maxsize key_headers key__proxykey__proxy_headerskey_socket_optionskey__socks_optionskey_assert_hostnamekey_assert_fingerprintkey_server_hostnamePoolKeycCs|}|d|d<|d|d<dD]}||vr,||dur,t||||<q|d}|dur>> manager = PoolManager(num_pools=2) >>> r = manager.request('GET', 'http://google.com/') >>> r = manager.request('GET', 'http://google.com/mail') >>> r = manager.request('GET', 'http://yahoo.com/') >>> len(manager.pools) 2 N cKs8t||||_t|ddd|_t|_t|_dS)NcSs|SN)close)pr<r<rMsz&PoolManager.__init__..) dispose_func)r __init__connection_pool_kwrpoolspool_classes_by_schemekey_fn_by_schemer=)self num_poolsr7rYr<r<rMrXs zPoolManager.__init__cCs|SrSr<r]r<r<rM __enter__szPoolManager.__enter__cCs |dS)NF)clear)r]exc_typeexc_valexc_tbr<r<rM__exit__szPoolManager.__exit__cCsb|j|}|dur|j}dD]}||dq|dkr(tD]}||dq|||fi|S)a Create a new :class:`ConnectionPool` based on host, port, scheme, and any additional pool keyword arguments. If ``request_context`` is provided, it is provided as keyword arguments to the pool class used. This method is used to actually create the connection pools handed out by :meth:`connection_from_url` and companion methods. It is intended to be overridden for customization. N)r5r6portrP)r[rYr=rE SSL_KEYWORDS)r]r5r6rfrHpool_clsrJkwr<r<rM _new_pools  zPoolManager._new_poolcCs|jdS)z Empty our store of pools and direct them all to close. This will not affect in-flight connections, but they will not be re-used after completion. N)rZrar_r<r<rMraszPoolManager.clearrPcCsT|std||}|pd|d<|st|dd}||d<||d<||S)a Get a :class:`ConnectionPool` based on the host, port, and scheme. If ``port`` isn't given, it will be derived from the ``scheme`` using ``urllib3.connectionpool.port_by_scheme``. If ``pool_kwargs`` is provided, it is merged with the instance's ``connection_pool_kw`` variable and used to create the new connection pool, if one is needed. zNo host specified.rPr5Prfr6)r_merge_pool_kwargsrrAr>connection_from_context)r]r6rfr5 pool_kwargsrHr<r<rMconnection_from_hosts    z PoolManager.connection_from_hostcCs,|d}|j|}||}|j||dS)z Get a :class:`ConnectionPool` based on the request context. ``request_context`` must at least contain the ``scheme`` key and its value must be a key in ``key_fn_by_scheme`` instance variable. r5rH)r>r\connection_from_pool_key)r]rHr5pool_key_constructorpool_keyr<r<rMrms  z#PoolManager.connection_from_contextcCs|jj4|j|}|r|WdS|d}|d}|d}|j||||d}||j|<Wd|S1s;wY|S)z Get a :class:`ConnectionPool` based on the provided pool key. ``pool_key`` should be a namedtuple that only contains immutable objects. At a minimum it must have the ``scheme``, ``host``, and ``port`` fields. Nr5r6rfrp)rZlockrArj)r]rsrHpoolr5r6rfr<r<rMrqs    z$PoolManager.connection_from_pool_keycCs t|}|j|j|j|j|dS)a Similar to :func:`urllib3.connectionpool.connection_from_url`. If ``pool_kwargs`` is not provided and a new pool needs to be constructed, ``self.connection_pool_kw`` is used to initialize the :class:`urllib3.connectionpool.ConnectionPool`. If ``pool_kwargs`` is provided, it is used instead. Note that if a new pool does not need to be created for the request, the provided ``pool_kwargs`` are not used. )rfr5rn)rror6rfr5)r]urlrnur<r<rMconnection_from_urls zPoolManager.connection_from_urlc CsR|j}|r'|D]\}}|dur"z||=Wq ty!Yq w|||<q |S)a Merge a dictionary of override values for self.connection_pool_kw. This does not modify self.connection_pool_kw and returns a new dict. Any keys in the override dictionary with a value of ``None`` are removed from the merged dictionary. N)rYr=r@KeyError)r]overridebase_pool_kwargsrJvaluer<r<rMrls    zPoolManager._merge_pool_kwargsTc Kst|}|j|j|j|jd}d|d<d|d<d|vr"|j|d<|jdur7|jdkr7|j||fi|}n |j||j fi|}|oG| }|sL|St ||}|j dkrXd }| d } t| tsitj| |d } | jr||stt|d} | D]} | | jvr|d| dq|z | j||||d } Wnty| jr|YSw| |d <||d<td |||j||fi|S)a] Same as :meth:`urllib3.connectionpool.HTTPConnectionPool.urlopen` with custom cross-host redirect logic and only sends the request-uri portion of the ``url``. The given ``url`` parameter must be absolute, such that an appropriate :class:`urllib3.connectionpool.ConnectionPool` can be chosen for it. )rfr5Fassert_same_hostredirectr7NrPi/GETretries)r~)response_poolzRedirecting %s -> %s)rror6rfr5r7r=proxyurlopen request_uriget_redirect_locationr statusrA isinstancerfrom_intremove_headers_on_redirect is_same_hostrCr iterkeysr>rE incrementr raise_on_redirectloginfo) r]methodrvr~rirwconnrredirect_locationrr7headerr<r<rMr1sJ       zPoolManager.urlopen)rRNrSNrPNT)__name__ __module__ __qualname____doc__rrXr`rerjrarormrqrxrlrr<r<r<rMr}s     rcsLeZdZdZ  dfdd Zdfdd Zdd d Zdfd d ZZS)rax Behaves just like :class:`PoolManager`, but sends all requests through the defined proxy, using the CONNECT method for HTTPS URLs. :param proxy_url: The URL of the proxy to be used. :param proxy_headers: A dictionary containing headers that will be sent to the proxy. In case of HTTP they are being sent with each request, while in the HTTPS/CONNECT case they are sent only once. Could be used for proxy authentication. Example: >>> proxy = urllib3.ProxyManager('http://localhost:3128/') >>> r1 = proxy.request('GET', 'http://google.com/') >>> r2 = proxy.request('GET', 'http://httpbin.org/') >>> len(proxy.pools) 1 >>> r3 = proxy.request('GET', 'https://httpbin.org/') >>> r4 = proxy.request('GET', 'https://twitter.com/') >>> len(proxy.pools) 3 rRNc st|trd|j|j|jf}t|}|js#t|jd}|j|d}|jdvr-t |j||_ |p3i|_ |j |d<|j |d<t t |j||fi|dS)Nz %s://%s:%irk)rfrO_proxyr8)rrr5r6rfrrrA_replacer r proxy_headerssuperrrX)r] proxy_urlr^r7rrYrrf __class__r<rMrXs&          zProxyManager.__init__rPcsD|dkrtt|j||||dStt|j|jj|jj|jj|dS)NrQ)rn)rrrorr6rfr5)r]r6rfr5rnrr<rMros  z!ProxyManager.connection_from_hostcCs0ddi}t|j}|r||d<|r|||S)z Sets headers needed by proxies: specifically, the Accept and Host headers. Only sets headers not provided by the user. Acceptz*/*Host)rnetlocupdate)r]rvr7headers_rr<r<rM_set_proxy_headerss  zProxyManager._set_proxy_headersTc sNt|}|jdkr|d|j}||||d<tt|j||fd|i|S)z@Same as HTTP(S)ConnectionPool.urlopen, ``url`` must be absolute.rPr7r~)rr5rAr7rrrr)r]rrvr~rirwr7rr<rMrs  zProxyManager.urlopen)rRNNrrSr) rrrrrXrorr __classcell__r<r<rrMros rcKstdd|i|S)Nrr<)r)rvrir<r<rMrsr)( __future__r collections functoolslogging _collectionsrconnectionpoolrrr exceptionsrr r packagesr Zpackages.six.moves.urllib.parser requestr Zutil.urlrZ util.retryr__all__ getLoggerrrrg _key_fields namedtupler4rNpartialr\r[rrrr<r<r<rMs:           6 s W