This blog is for Free Thought and occasionally, for the thoughts of the author of this blog! Technology and Open Source tools. Hacks from the world of Information Technology and my occasional rants.
If this blog helped you in any way, please donate a dollar here
Showing posts with label Code. Show all posts
Showing posts with label Code. Show all posts
Sunday, December 23, 2018
Monday, June 26, 2017
Fast download with Golang
Hey all, people keep saying about how great the concurrency is in golang. To be honest, I am pretty much a novice in this.
So just wrote a download accelerator in golang using it's much awesome parallel feature, the source of which can be found here: https://github.com/rghose/go-parallel-downloader
Here are a few results:
This was with GOMAXPROCS=2, threads = 5 with Chunk Size = 10 KB
So well, the chunk size of download did make a lot of difference, since the number of connections to download the thing would decrease, too many connections does not help in speed up
So just wrote a download accelerator in golang using it's much awesome parallel feature, the source of which can be found here: https://github.com/rghose/go-parallel-downloader
Here are a few results:
| URL Downloaded | Time taken by wget | Time taken by Golang |
| http://get.videolan.org/vlc/2.2.5.1/macosx/vlc-2.2.5.1.dmg | 1m3.375s | 0m26.609s |
This was with GOMAXPROCS=2, threads = 5 with Chunk Size = 10 KB
So well, the chunk size of download did make a lot of difference, since the number of connections to download the thing would decrease, too many connections does not help in speed up
Monday, December 5, 2016
Selenium ChromeDriver Install Script
Hello Folks,
As I was trying to instruct someone how to install Selenium Webdriver on OSX, I realised that there were no good scripts to automate this task. There were instructions, sure, but none of the were a copy and paste solution. Not that I encourage copy-paste, I found this rather inconvenient though.
So here's a script install selenium chromedriver for any unix osx or windows computers (assuming cygwin) -
In case, the github plugin did not work the URL is here:
https://gist.github.com/rghose/106ae71639c6f9ce9076a260a5614b94
Enjoy!
As I was trying to instruct someone how to install Selenium Webdriver on OSX, I realised that there were no good scripts to automate this task. There were instructions, sure, but none of the were a copy and paste solution. Not that I encourage copy-paste, I found this rather inconvenient though.
So here's a script install selenium chromedriver for any unix osx or windows computers (assuming cygwin) -
In case, the github plugin did not work the URL is here:
https://gist.github.com/rghose/106ae71639c6f9ce9076a260a5614b94
Enjoy!
Friday, January 8, 2016
One line youtube video downloader
Okay, so this might be illegal in certain places. I just decided to try this out for fun, and here we have it, download youtube videos with 1 line of shell script (and some perl)
The script looks like this:
The script looks like this:
curl -H 'Upgrade-insecure-requests: 1' -H 'Cache-control: max-age=0' -H 'Accept-language: en-US,en;q=0.8,bn;q=0.6' -H 'Accept-encoding: gzip, deflate, sdch' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8' -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36' "https://www.youtube.com/watch?v=II3L1noJlYQ" | gunzip | egrep -o 'https%3A%2F%2F[^\.]*\.googlevideo.com%2F[^,\]*' | perl -pe 's/\%(\w\w)/chr hex $1/ge' | head -n1 | xargs wget -O out.video
One line.
Get it here:
Sunday, November 2, 2014
Identifying the actor
So, here I am at Candies, Bandra looking at an all too familiar Bollywood actor, who was also a customer there. He played the villain or cop in many movies, I know him. He had his son with him as well. They bear an uncanny resemblance. I want to go up there and get a picture clicked with him. Wait, what was his name?
I asked my friend who was with me, he too seemed clueless. He also wanted to get a picture clicked, but what if he asked his name? We were terrified of that prospect, as we saw a young woman probably in her 30s, go up to him and click one herself. Damn. Shes knows his name for sure.
In our dilemma and brain racking to recollect his name, he was done with his buying and he left. Leaving me pondering over this problem of his identity. So I came home and tried searching Google, assuming it will read my brainwaves. No use. I could not find a Bollywood actor who visited Candies often, he was not a very well known figure like say, Shahrukh Khan for instance, but he was pretty well known. Now I decided to do this: http://en.wikipedia.org/wiki/List_of_Indian_film_actors!
Again, there were too many! So I made these:
a) Python script to get the images from an infobox element in a page here.
b) Shell script to automate this here.
Armed with these tools, I did this:
I made myself some coffee, came back to my laptop and casually started browsing the thumbnails from my file manager. Found him!
I asked my friend who was with me, he too seemed clueless. He also wanted to get a picture clicked, but what if he asked his name? We were terrified of that prospect, as we saw a young woman probably in her 30s, go up to him and click one herself. Damn. Shes knows his name for sure.
In our dilemma and brain racking to recollect his name, he was done with his buying and he left. Leaving me pondering over this problem of his identity. So I came home and tried searching Google, assuming it will read my brainwaves. No use. I could not find a Bollywood actor who visited Candies often, he was not a very well known figure like say, Shahrukh Khan for instance, but he was pretty well known. Now I decided to do this: http://en.wikipedia.org/wiki/List_of_Indian_film_actors!
Again, there were too many! So I made these:
a) Python script to get the images from an infobox element in a page here.
b) Shell script to automate this here.
Armed with these tools, I did this:
# wget http://en.wikipedia.org/wiki/List_of_Indian_film_actorsDeleted the last irrelevant line and fired up "./parse_names.sh list"
# grep '<li><a href=' List_of_Indian_film_actors | grep 'title' > list
I made myself some coffee, came back to my laptop and casually started browsing the thumbnails from my file manager. Found him!
Wednesday, November 17, 2010
Database connectivity with Java
Spent a lot of time today searching for ways to use Database connectivity on my Ubuntu box. I write the code but it just was not working!
Wrote this code to test for connectivity with this :
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.SQLException;
public class TestDatabase {
Friday, August 6, 2010
Captcha in PHP
Writing a custom captcha for your website is a fairly easy task with the PHP GD Library.
The GD Library is a graphics library that enables to draw (and generate) graphics. This output can be done on any of the standard image file formats.
In my example I choose to create a PNG file that will be displayed in my registration page. So the first piece of code I need is somthing like this:
The GD Library is a graphics library that enables to draw (and generate) graphics. This output can be done on any of the standard image file formats.
In my example I choose to create a PNG file that will be displayed in my registration page. So the first piece of code I need is somthing like this:
Subscribe to:
Posts (Atom)
