Python requests-cache multiple expirations

How do I set up different expiration timers for specific api requests I make with requests using the requests-cache library? Example, I want one specific call to be cached for a minute while another separate call url method is cached for an hour. ...
more »

2017-10-04 00:10 (0) Answers

Enforce serial requests in asyncio

I've been using asyncio and the http requests package aiohttp recently and I've run into a problem. My application talks to a REST API. for some API endpoints it makes sense to be able to dispatch multiple requests in parallel. Eg. sending differ...
more »

2017-09-19 02:09 (0) Answers

Python pass parameters to URL

I have to use requests.Session.get to get a JSON from a URL . I need pass a obj as parameters. The Url is: http://c0.3.cn/stock?skuId=5217606&area=1_72_2799_0&venderId=1000000140&cat=670,671,672&buyNum=1&choseSuitSkuIds=&extr...
more »

2017-09-14 03:09 (1) Answers

How to handle rate limiting and caching?

So far I have achieved a very basic understanding of how the HTTP protocol works. My question isn't going to be a practical one, more of a (stupid) conceptual one. I searched for a policy concerning these types of things, and to my understanding this...
more »

2017-09-09 17:09 (0) Answers

Python mock REST API

I need to do unittest for a method that has 2 rest calls and another method call. To describe in details, it is as below: class A(): def methA(self, param1): val1 = methB(someParam) res1 = requests.get(url1, headers=headers, veri...
more »

2017-08-28 17:08 (0) Answers

RESTful file upload fails with large files

Greenhouse's harvest API requires that I include the base64 encoded content in my post request to attach a file to a user. I put together this POST request in python, which works for files of size 70kb. r = requests.post('https://harvest.greenhouse...
more »

2017-07-26 23:07 (0) Answers

SSL Client Authentication with Python requests

I'm using Python's requests library to perform client side authentication with certificates. The scenario is the following: CA1 has issued a certificate for an intermediate CA (CA2) and CA2 has issued my client's certificate CLIENT. The server I'm co...
more »

2017-07-21 11:07 (1) Answers

Downloading files from URL in Python

I have a CSV file that has 67000 rows of URL. Each URL leads to download other datasets in format of CSV, HWP, ZIP, etc. This is the code I have written: import cgi import requests SAVE_DIR = 'C:/dataset' def downloadURLResource(url): r = re...
more »

2017-07-20 10:07 (1) Answers

python requests: url with another sub-requests

When I execute manually this URL in my webbrowser I see in my network console that three other requests will be executed. it works. call: www.my.url/publish_something get this cmd get that cmd post that... How can I do it in Python requests? Th...
more »

2017-06-22 13:06 (1) Answers

python requests loading big page solution

I have this trouble when opening a big page about 82,000 lines with Python requests lib. Before I tried to use urllib2, but here was an error "IncompleteRead" Now with requests: r = requests.get(https://www.bhphotovideo.com/c/search?atclk=Model+Y...
more »

2017-05-22 09:05 (1) Answers

Loading Image Resource into Numpy Array

I'm trying to get a JPEG image resource from the web into a NumPy array image representation similar to the array returned by scipy.misc.imread. Instead of saving the image to disk, as in the example below: import requests from scipy import misc de...
more »

2017-05-18 22:05 (2) Answers

Add string from list to url for API get request

Hi I have an API get request that retrieves numbers that I put into a list. I then want to loop over that list, adding every number in the list to a separate new get request. The code I have so far is: for evt in eventList: main_api = 'https://a...
more »

2017-05-16 13:05 (1) Answers

python requests, what i am missing?

import requests a = 'http://www.wipo.int/branddb/en/?q={"searches":[{"te":"coca%20cola","fi":"BRAND"}]}' b = requests.get(a) print(b.text) Here is a simple webpage. If I paste url in browser and open the link, I get the results I want to see. If...
more »

2017-04-19 23:04 (1) Answers

Why does browser send requests for static files?

I have seen in Firebug that my browser sends requests even for all static files. This happened when I have enabled caching for static files. I also saw the server response with 304 status code. Now, my question: Why should the browser send requests ...
more »

2017-04-15 10:04 (1) Answers

Prevent http request caching at client side

I am trying to measure the response time of a Web Server , at client side but the response times are very small. So I think that the requests are caching at the client side. I have already tried adding appendices next to url but no change. ...
more »

2017-03-19 13:03 (0) Answers

Python connect to rest api

i want to connect to a rest api with the following code: data = requests.get("http://pbx/rest/login").json() head = {'loginType' : 'Internal','nonce' : data["nonce"], 'secret' : (hashlib.sha1((LOGIN_ID+data['nonce']+PASSWORD).encode('utf-8')).hexdig...
more »

2017-02-18 13:02 (1) Answers