GimmeProxy.com - free rotating proxy api
Gimmeproxy was heavily updated recently, check this article: http://256cats.com/gimmeproxy-api-major-update/
Recently I needed a lot of free proxy servers for one project. There was another requirement: proxies had to be rotated.
That means I needed to change proxy after several requests and needed many. I couldn’t use the same proxy for more than 400 requests per hour, so I had to manage some kind of timeouts.
That’s easy, I thought: get list of proxies somewhere, check them and choose one at a time by random. But it turned out to be not so easy.
First of all, obtaining free proxy list in txt or csv format was not so FREE. In fact a lot of websites offer that kind of service. They allow to check some of their proxies and then try to sell you full updated list for $3-20 per month. Others have some subscription-based API.
I had no idea what I was going to pay for. Some proxies from the free list just didn’t work at all, some didn’t allow to connect to my target website, even when they were advertised as ‘checked’ 3 minutes ago. Some did work though.
So I still had to check every proxy and set timeouts.
Long story short, I decided to build my own free rotating proxy api. It’s RESTful and called GimmeProxy.com.
GimmeProxy.com
For now it’s simple: it scrapes public lists of proxies from different websites, checks them and returns you random one in JSON.
It’s free and you don’t need to register or anything. Just think up and use any string that will be your user id and you are good to go. You get new random proxy with each request to the API.
Also you can set timeout in seconds. During that time you won’t get the same proxy again. Or you can set it to 0, if you don’t care.
For example: http://gimmeproxy.com/api/get/8bb99df808d75d71ee1bdd9d5da5672a/?timeout=0
Here user id is 8bb99df808d75d71ee1bdd9d5da5672a and timeout is 0 (i.e. no timeout).
Example response: {“ip”:”123.225.106.40:80”, “country”:”usa”, “type”:”http”, “curl”:”http://123.225.106.40:80"}
Ip, country and type are self-describing and curl is _CURLOPT_PROXY_ option of Curl library.
PHP Curl Proxy example
It’s simple to use gimmeproxy with PHP.
Node.js Curl Proxy example
At first install curl-request module.
If you are interested, please give it a try. Still some proxies might not work for your target servers, but you can always get another proxy.
Any comments and questions are welcome as usual.
Tech stack
This site was built with NodeJS, Express, Redis, Cheerio. I’ve used Redis as a database, especially it’s excellent Lua scripting feature.
Realtime stats was made with Socket.IO and Redis PUB/SUB.
To run all scripts I used PM2.